(This article was published in the November/December 2010 issue of interactions magazine.)

It seems likely that we find ourselves at an inflection point in the evolution of healthcare. While the situation has certainly been brought to a boil by recent American political events, the opportunities for change fit into a much larger context; they have the potential to truly transform the delivery of healthcare globally.

Unlike some, I don’t believe our current healthcare system is totally broken. I’ve conducted design research in quite a number of clinical settings and have consulted for businesses representing many different aspects of the healthcare industry, including provider networks, medical-device manufacturers, and even health insurance companies. I’ve seen magic worked on regular basis, and from a historical (and global) perspective, the standard of care in the developed world is astoundingly high. I am in awe of the abilities of doctors, nurses, techs, and other clinicians to consistently function at a very high level despite the fact they’re forced to work with archaic infrastructure in less than ideal environments. (As for the insurance companies, perhaps the best thing to say is that they function to make money but could be dramatically more successful as businesses if they changed their approach to things.)

It is at this level—the level of infrastructure—where these big opportunities for transformation exist. It isn’t that we don’t know what kinds of patient and clinician behaviors and medical interventions result in healthy outcomes; it’s that at a systemic level, we’re not doing a good job facilitating these behaviors and driving appropriate interventions. The right changes here will provide a conduit for evolutionary change to cascade throughout the system to achieve dramatic improvements in the quality and cost of healthcare. Which isn’t to say that it also isn’t incredibly important for medical knowledge to continue to evolve; it’s just that we already know enough to dramatically drive up quality and drive down costs.

Many of the opportunities to improve our healthcare system can fit into three big categories: proactively engaging individuals to take better care of themselves; providing better interventional care beyond the walls of the hospital; and improving care delivery inside hospitals through standardization and better collaboration between clinicians, patients, and families. All three of these strategies require new infrastructure and perhaps a shift in the definition, role, and activities that characterize the hospital.

The first two ideas are mostly about what happens outside the hospital. These are things that architects wouldn’t traditionally worry about when designing hospitals. But that kind of thinking has gotten us into our current predicament, where the current built “environment” for providing healthcare is sometimes an impediment to necessary change. If we step back and define a hospital as the nexus for healthcare in a community, we have a platform on which we can imagine the ideal infrastructure for keeping people healthy as possible in a cost-effective way.

In the May+June 2010 issue of interactions, Hugh Dubberly suggested designers ought to help reframe what healthcare is and how it is delivered, as well as to reframe what it means for design to help. I couldn’t agree more, and in this spirit, propose reconsidering what healthcare infrastructure is necessary to better care for people, how design should address this new notion of infrastructure, and what this all means for the institution of the hospital.

Proactive Engagement of Individuals

The first huge opportunity for healthcare transformation is around proactively engaging individuals with their own health. In case anyone has missed this striking fact, many of the most common chronic diseases, like diabetes, heart disease, and cancer, are not only a substantial cause of death (70 percent in 2006, according to the U.S. Agency for Healthcare Research and Quality), and a substantial source of healthcare costs ($500 billion in the U.S.in 2006), but they are largely preventable or controllable by lifestyle choices such as diet and exercise.

So not only is a person with diabetes who is able to proactively keep their blood sugar under control with diet and exercise more likely to have a “positive outcome” (i.e., live a long life, avoid the nasty complications of poorly controlled diabetes like loss of vision or amputation), but they are also likely to cost their health insurance payer a lot less than someone with poorly controlled diabetes. Put another way, you can buy an awful lot of proactive engagement with the cost of one ambulance trip to the hospital followed by even a short hospital stay.

But people without the symptoms of disease rarely encounter the healthcare system, and by the time someone faces Type 2 Diabetes or a first heart attack he or she has probably developed some habits that are pretty tough to break. So as obvious as this opportunity is, it’s a tricky challenge—how can we educate and motivate pretty much everyone to live in a healthy way as early as possible, and then how do we help people make radical adjustments to their behavior if either the first round of engagement didn’t work, or they’re genetically or otherwise predisposed to a chronic disease and they’ve fallen ill?

While this all sounds good in theory, people’s habits are notoriously difficult to change, and psychologists’ and economists’ substantial understanding of human motivations can seem meager in the face of rampant unhealthy behavior in our society. However, there are instances in which these strategies have been clearly shown to be highly effective.

Like many large companies, Safeway’s health insurance program is self-funded, meaning the employer pays all health insurance benefits (and therefore incurs most of the risk and benefits from all cost savings). Starting in 2005, Safeway has offered significantly discounted premiums for people with healthy behavior related to tobacco usage, weight, blood pressure, and cholesterol levels. During the first four years of the program, Safeway kept per capita healthcare costs flat, while most American companies’ costs increased 38 percent over the same period.

Safeway CEO Steven Burd recently described the thinking behind the Healthy Measures program: “Safeway’s plan capitalizes on two key insights... The first is that 70 percent of all healthcare costs are the direct result of behavior. The second insight, which is well understood by the providers of healthcare, is that 74 percent of all costs are confined to four chronic conditions (cardiovascular disease, cancer, diabetes and obesity). Furthermore, 80 percent of cardiovascular disease and diabetes is preventable, 60 percent of cancers are preventable, and more than 90 percent of obesity is preventable”. (For more about the Safeway Healthy Measures program, see this piece by Burd in the Wall Street Journal.

So it seems that with even the most basic motivators (money), it’s possible to both reduce costs and improve health for a relatively large and diverse population. The next obvious question is what infrastructure is required to proactively engage individuals to behave in healthier ways on a broader basis.

Research by behavioral economists has found that people tend to make good decisions where action and result are closely tied together (either through feedback or when consequences are obvious), when doing the right thing isn’t too difficult, and when a decision is made repeatedly (i.e., the individual is able to practice). People also tend to be highly motivated by making progress toward achieving goals, and by group social dynamics—fitting in with and impressing (or not disappointing) their friends, family, colleagues and neighbors. (For a good primer in behavioral economics, see Nudge by Richard Thaler and Cass Sunstein.)

It seems then that the starting point for behavior change is good information, and the starting point for the supporting infrastructure must be a robust health record that includes a wide range of information, including data from clinical systems (e.g., hospital electronic medical records), as well as more personal information about things like diet, exercise, and health data collected at home (weight, blood sugar, blood pressure). Of course this data must be accessible and easily maintainable for patients, their families, and the clinicians (or institutions) whom they choose to allow to create or consume data from the personal health record.

While there is strong evidence that just tracking personal data can have a significant impact on health-related behavior, what is done with and in response to the data is critically important. The most basic motivator is a good understanding of the potential consequences of individuals’ actions, so interfaces into this personal health record must help people understand what diseases and conditions they are at risk for based upon the clinical and personally tracked information. From here various levers are used to help people reduce the likelihood of these unhappy consequences with capabilities for goal setting and progress tracking, rewards for healthy behavior, and education about the potential ramifications of unhealthy behaviors tied in to various social media to provide community and support.

There are also huge opportunities to present data in such a way as to help people understand correlations between their actions, environmental factors, and their health. This can help individuals fine-tune treatment of a disease or condition or better understand an undiagnosed problem. While my assumption is that mobile- and Web-based software interfaces will be invaluable in delivering these experiences, we shouldn’t at all discount the role of clinicians—nurses in particular—in educating patients about their diseases and helping them establish and track goals to improve their health.

Sadly, for any of this to be effective, there is considerable work to be done both in translating medical (and legal) information for general consumption, and in educating the public on how to comprehend even basic medical facts. According to the 2003 National Assessment of Adult Literacy, fewer than half of American adults were judged to be proficient in health-literacy skills to the extent that they could accomplish tasks like finding the correct age range during which children should receive a particular vaccine, or using a chart that shows all childhood vaccinations and the ages at which children should receive them [5]. While interactive experiences can provide useful assistance to people struggling to understand their medical and health situations, our overall healthcare infrastructure must better account for the education of almost everyone, probably through integration into secondary-school curriculum. (Perhaps this would be an aspect of usefully reimagined “home economics” or “civics” classes.)

Personal-data acquisition is also a huge field of need and opportunity. Current solutions for tracking personal health data tend to be extremely siloed. While mobile devices provide a great new way of delivering sensing and monitoring capabilities, and while there are some nicely designed health-info trackers, exercise logs, and food diaries in the iPhone App store, almost all require a user to actively launch the application or interact with a reminder in order capture data. These all tend to be plagued with the same problem as paper trackers—the people who remember to use them regularly tend to be people who are already in good control of their health.

The real opportunity lies in figuring out how to integrate data tracking and education with experiences that people already find engaging—whether it’s social networking, television, games, or reading news. The tricky part is figuring out how to present these capabilities so that they provide enough friction to capture people’s attention, but not so much friction that they detract from the overall experience. And of course, they should be as lightweight to interact with as possible. I could even imagine insurers paying to advertise in media and software—behavior placements instead of product placements and even setting up incentive programs where media providers are rewarded for driving healthcare engagement (like paying for clicks in online advertising). This would seem to be a small expense in the face of the cost savings shown by Safeway.

Changing the Boundaries of the Hospital

Clearly, as healthcare institutions continue to reach out into the community to concern themselves with proactively motivating healthy behaviors, the boundary of the hospital will increasingly no longer be defined by physical walls. This trend also extends to interventional and nursing care.

Home healthcare is nothing new, and millions of people in the U.S. alone receive some form of home care (according to the February 2004 "National Home and Hospice Study", by the National Center for Health Statistics), but it is very expensive for patients and payers alike to get an in-person visit for basic healthcare monitoring for patients with chronic conditions like congestive heart failure. A number of products exist that walk patients through the process of collecting vital signs (in the case of congestive heart failure: blood pressure and weight), reminding them about their care regimen and uploading the vitals to a central nursing station where software tools can help nurses spot trends that may require in-person intervention.

A recent study by Kaiser Permanente and the American Heart Association has shown that patients with high blood pressure using home healthcare monitoring were 50 percent more likely to have their blood pressure under control. This alone represents a dramatic opportunity—hypertension is a leading predictor for heart attacks and strokes, both of which are highly deadly and expensive to our healthcare system as a whole (and therefore expensive to every one of us). Another recent study conducted by Kauffman Foundation and Brookings Institution economist Robert E. Litan predicts that the U.S. could save $197 million from implementing telehealth systems to help treat chronic ailments like diabetes, congestive heart failure, chronic obstructive pulmonary disease, and chronic skin ulcers.

Current home health monitoring products are mostly dismally designed. For example, the Health Buddy, a popular telehealth device, communicates to its (mostly elderly, sight-impaired, arthritic) users entirely with small, center-aligned text, which is clearly not as readable, let alone as engaging, as it could be. These products also suffer from the same issues that plague many existing solutions around proactive engagement: They aren’t already integrated into a person’s daily routine. For example, given the prevalence of television watching among the chronically ill, it seems that the cable-TV box would be a much more useful platform than a standalone “medical device,” which is as easy to ignore as the blood pressure pump itself.

The Mayo Center for Innovation (CFI) has been working on a program to design a “patient-centered medical home” that provides a better connection between patients and their healthcare providers beyond the in-person visit. According to the CFI, it is “exploring ways to implement unobtrusive systems into a patient’s daily routine—perhaps a coffeemaker or refrigerator that records blood pressure or glucose levels—without disrupting their busy lives”. While this certainly points in the right direction, it is the tip of the iceberg. Whether it’s in a coffeemaker, the television, or PC-based social media, there is a huge opportunity for designers to imagine how home healthcare monitoring and telehealth can more seamlessly fit into people’s daily lives.

Using technology to provide better care beyond the walls of hospitals and doctors’ offices need not be limited to monitoring chronic-disease patients. Many routine trips to a doctor’s office might easily be replaced by a videoconference or even an email exchange. This, of course, would be especially beneficial to patients in rural settings.

Centralization of specialized resources seems to generally be a good thing in the organization of a healthcare system. It’s more cost-effective and easier to standardize care (and thereby improve quality) to deliver cardiac care in a single large facility in a metropolitan area than in a number of small facilities spread around. However, a huge downside to this approach is the increase of travel time—and decrease in access—for patients. Telehealth provides great opportunities to mitigate these problems by allowing patients to access the expertise at specialized facilities from far away. And not only ought our new healthcare infrastructure better support remote clinician-patient interactions, but because of this trend to centralize specialization, healthcare providers will also increasingly need information systems to support non-collocated collaboration between clinicians.

A common concern about an increased reliance on telehealth is that the lack of face time will result in a reduction in quality of care. This thinking goes that it will be even easier for doctors to be less thorough, and they won’t benefit from their well-developed senses of observation. The counter-argument to this is that by reducing the amount of time spent on very routine visits, doctors will actually free up time to spend with patients whose conditions require their full attention.

Current payer policies are perhaps the biggest infrastructure challenge when it comes to telehealth. As often is the case with many kinds of preventive care, currently telemedicine and remote consultation and treatment are not well-covered by Medicare or private health insurance plans. Remote consultations between doctor and patient are reimbursable only if conducted over video, and preventative examination of remote monitoring data is not reimbursable. It’s clear that both public and private payers could reap significant cost savings (and care improvements) by better compensating physicians and hospitals for using telemedicine effectively, but are too bogged down in bureaucracy to make the necessary policy changes.

While telehealth is largely about extending the impact of the healthcare system into people’s homes, when we talk about redefining the boundaries of the hospital it is also important to consider how healthcare institutions can have a bigger impact in communities, perhaps even by bringing these communities into the hospital. While it may be common sense to many of us, a 2009 paper in the Archives of Internal Medicine showed a 38 percent lower incidence of Type 2 Diabetes among people who lived in neighborhoods with better resources for food and exercise [9].

Determining exactly what should be done as a result of this data is certainly a ripe opportunity for design thinking, and some would reasonably argue that access to healthy food and exercise is a civic good that is best handled at the municipal level. That said, if we truly want to orient our healthcare infrastructure in such as way as to result in better health at a lower cost, perhaps there is an opportunity for healthcare institutions to have a hand in providing these essentials to a community. It’s clearly outside their core expertise now, but as these institutions move in the direction of encouraging healthy behavior, is it really too much of a stretch to imagine that hospitals could have a hand in providing exercise facilities and farmers markets full of fresh veggies (one of the biggest drivers of diabetes reduction on the food side of the above-mentioned study)?

Changing Care Delivery

Even if we are successfully and proactively engaging people to live more healthily and we are better able to use telemedicine to avoid expensive trips to the hospital, a significant portion of medical care will still be delivered in hospitals and doctor’s offices. One of the most significant recent changes in healthcare is a move to standardize treatment upon what has been scientifically shown to produce the best outcomes. It turns out that this incredibly reasonable (and surprisingly new) approach does in fact tend to improve patient outcomes while reducing cost of care.

For example, at Intermountain Healthcare, a Salt Lake City-based system of hospitals and clinics, clinicians adopted practices to more tightly control of the glucose level of patients in intensive care units (ICUs), which “led to a statistically significant reduction in the rates of mortality in this patient group.” And by standardizing care for patients on ventilators, they ended up reducing the average time each patient was on a ventilator by more than a day. These adjustments reduced the rate of ventilator-associated pneumonia by 10 percent over two years, shortened the overall length of stay in the ICU, and reduced costs by more than $3,000 per ICU patient. (For more about IHC, see this article in the NY Times Magazine. Also, in the name of full disclosure, I should say Cooper has worked with IHC for the past couple years.)

As obvious as it might sound, on a practical level, actually practicing evidence-based medicine is harder than it might appear. The rate of new findings in medical science continues to increase, and for all but the sub-sub-specialists it is nearly impossible for a physician to read and retain all the research relevant to the patients that they see. For evidence-based medicine to be a reality, it requires that medical decisions be made in light of appropriate patient data, which is viewed in the context of up-to-date medical research (it also requires that substantial data be captured about the treatment and progress of each patient to feed back into research). Further, while all this data is absolutely critical to the endeavor, information systems must also be smart and help make people smart.

Human bodies and diseases are incredibly complex, and research findings are rarely black and white (and are always changing). In top form, human judgment (for example, as portrayed in Gary Klein’s Sources of Power or Malcolm Gladwell’s Blink) is capable of things well beyond any AI or decision engine (or at least one that’s ready for prime time on a hospital floor). But the one thing that computers do have on even the smartest clinicians is consistency. This means that information systems must aid in filtering through the increasingly voluminous data captured about each patient; help clinicians understand relationships between data, observations and protocols, care guidelines, and other standardized approaches to care; and ultimately assist in making good judgments about patient care.

Rapidly increasing medical knowledge has resulted in an increase in specialization. This means that most patients with a chronic disease or comorbidities are treated by teams of clinicians rather than one or two individuals, and our healthcare infrastructure must also support communication, collaboration, and workflow coordination between clinicians. This means not only more team-oriented physical environments, but also better tools for remote collaboration: a way of conducting remote “rounds”—complete with videoconferencing, patient video feeds, shared viewing of imaging, and medical data, all with next-to-zero fussing with technology.

And as we consider how infrastructure should support communication and collaboration, we mustn’t forget patients and their families. Recently, there has been a drive to better involve patients and their families in care decisions. At the Mayo Clinic, the SPARC innovation group has been experimenting with consulting spaces that move the doctor out from behind the big desk, onto equal footing with the patient in an attempt to engage the patient as a participant in care rather than a passive invalid. (Read more about this work in this article by Businessweek.)

Similarly, at the St. Louis Children’s Hospital newborn intensive care unit, parents are encouraged to contribute observations during rounds, be involved in treatment decisions, and participate in routine care activities so they’re prepared before taking an infant home. Not only does this type of family involvement reduce emotional stress, and in fact help coordinate communication between clinicians, but the American Academy of Pediatrics has also found that pediatric care that actively involves parents generally results in shorter hospital stays, fewer readmissions, improved staff satisfaction, and fewer lawsuits. (Read more about St. Louis Children's in the Wall Street Journal.)

The infrastructure required here is actually quite similar to what we have already discussed with regard to changing care delivery. Hospital architecture and information systems must not only account for communication and collaboration between clinicians, but must also include patients and families. They should be able to monitor and enter data, to review and discuss care plans and specific orders, to ask questions and provide suggestions to clinicians, and to be reminded and encouraged about things like medications and physical therapy.

Conclusions

There is no way that healthcare transformation can be achieved solely by legislation. While the necessary changes absolutely do require new policies to be put in place that better align the interests of healthcare providers (doctors and hospitals) and payers (insurance companies and government) with the health of people, this alone will not achieve the cost reductions and quality improvements we seek. What must be changed is the way individuals interact with the healthcare system, and I believe this change must be driven by healthcare institutions.

The institution of the hospital is ripe for reinvention. By taking a step back and considering what infrastructure best supports a healthy community, we can reframe the idea of the hospital in a way that makes it more effective, both in terms of medicine and economics. What if instead of limiting our notion of a hospital to the building where clinicians physically act upon patients, we expanded that definition to include the physical and virtual environment that provides all kinds of interactions and experiences that result in healthier people.

Of course there is no shortage of challenges in achieving this vision. Medicine is very complex, and transitioning to the digital  agehas been slow and not universally successful. As with everything built on a foundation of modern information technology, there is a huge potential for difficulty for the people involved—patients and clinicians alike. There are legacy systems, policies, and attitudes that threaten to unnecessarily slow the pace of progress. And at the end of it all, while digital infrastructure will be invaluable to the productive transformation of healthcare, the most important interactions will between people. As former Cleveland Clinic Chief Experience Officer, and patient-centered design pioneer Dr. Bridget Duffy put it, “I think there’s an understanding that as the higher tech we get, the more high touch we need to get”.

What do you think? Join the conversation in Comments

After years of mumbling excuses about the cobbler's children and how busy we've been, we're thrilled to announce the launch of our new site. It's taken almost a year from our initial design explorations, but we're really happy with where we've ended up.

While its been a very collaborative effort, it's also been refreshing to design without the usual cast of stakeholders. (In order to overcome the well-known nightmare that is a firm designing its own site, we almost completely eliminated creative reviews by anyone not directly involved in the project.)

We think the new site much better reflects our design sensibilities and the direction of the firm. It's still a bit of a work-in-progress. (For one, we plan on adding social bookmarking features in the Journal when we have a moment.) But we're interested to hear your feedback—let us know what you think in the comments section.

Credits

Design by Nick Myers and Dave Cronin, with help from Jayson McCaulliff, Doug LeMoine, Imon Deshmukh, Martina Maleike, and Daniel Kuo. Copy by Dave and Doug, with editorial assistance from Steve Calde and Suzy Thompson. Code by the amazing Elisha Cook and Andrew Hoag at blackdrumm, and photography by the very talented Emily Nathan.

What do you think? Join the conversation in Comments

I have to admit that I’ve been steering clear of talking about “design thinking” for a while now. A couple years back, when I first heard about what sounded like an exciting new angle on design strategy, I eagerly scoured the web to figure out what it was all about. At Cooper, we’ve always concerned ourselves with challenges beyond skin-deep ornamentation, and we particularly relish working for clients who value the insights that we can bring to their strategic business decisions. I’m interested in anything that gives us leverage to help businesses get beyond the assumptions that stand in the way of truly serving human needs.

So when I set off to learn more, I was a bit disappointed to discover that all the information I could find about “design thinking” appeared to prominently feature the Keeley triangle, some business success stories and not a lot more. (For those that aren’t familiar, Larry Keeley, an OG innovation strategist, devised the triangle as a way of expressing how successful businesses are balanced in the concerns about the desirability, technical feasibility and financial viability of their products.)

The Keeley Triangle. The d-school site appears to have been refreshed in the interim, but if I remember correctly, at one point, the home page featured a marker sketch of this diagram with the words “this is design thinking.”

To be clear, I have no argument with the Keeley triangle. It was part of the foundation of Alan’s arguments in The Inmates are Running the Asylum (Alan Cooper’s 1999 book about the challenges of creating great digital products), and throughout the years I’ve found it to be an incredibly useful device in explaining how design fits with business and technology concerns.

But I guess I feel like defining design thinking by the Keeley triangle alone is like explaining how to fly by stating the laws of physics. In a 1998 HBR article, one of the first articulations of design thinking, Tim Brown defined design thinking as “a discipline that uses the designer’s sensibility and methods to match people’s needs with what is technologically feasible and what a viable business strategy can convert into customer value and market opportunity.” I have very little to disagree with in this, yet I don’t find it particularly useful or interesting. And it really begs at least one big question—what part of “the designer’s sensibility”? The obsession over details? The ability to create incredibly disorganized Photoshop (or Fireworks) files? The propensity to wear black?

All this said, I certainly see promise in the vision and enormously appreciate the work that Brown and IDEO have done to popularize the idea that human-centered design methods are fantastic tools for improving all kinds of things—not just product skins and interfaces, and that businesses can get vastly more value when they ask designers to participate in the product (or service) conception process, rather than to just pretty-up an already-formed idea. So I was really excited when I finally got around to reading Roger Martin’s The Design of Business and discovered a conceptual model that has really helped me understand what part of the designer’s skillset is really useful for this big picture thinking.

Martin refers to this conceptual model as “the knowledge funnel.” The funnel starts with a mystery—for example, how to feed the newly emergent car-centric middle class of 1950s Southern California. Businesses then can create value by moving along the tunnel first to a heuristic, or simple idea about how to solve the mystery—a quick service hamburger stand; then to an algorithm, or the specific operational rules about how to achieve the heuristic—where the hamburger stands should be located, how they should be designed, what the menu should be, how to prepare every item on the menu, and how customers should be served.

Among other things, what emerges in Martin’s model of design thinking is that this “designer’s sensibility” that Brown speaks of is the ability to use an understanding of customers’ needs (as well as technology and business factors) to move inwards and outwards in this funnel by iterating through many different heuristics and algorithms to ultimately imagine and then validate a way of solving this mystery. Intrinsic to this ability is abductive reasoning— making logical leaps to imagine what might be true in the future.

These ideas really resonate with me, but I struggle with the notion that abductive reasoning abilities are unique to designers. Martin is dean at the Rotman School of Management at the University of Toronto, and his audience is largely business people. I understand why he wants to differentiate these sensibilities from the largely analytical skills that dominate modern business education. But when I first read and thought about the idea that abductive reasoning is “design thinking”, I had two reactions: first, this is what I’d thought business people were supposed to be doing all along; and second, I know plenty of designers who aren’t at all interested in or good at abductive reasoning beyond their medium of, for example, interaction design, visual interface design or industrial design.

Ultimately, I have grave concerns if imagining a better future becomes solely the province of designers or design thinkers, a world of business and political leaders will be absolved of their core responsibility—making things better. (Not that I’m suggesting either Brown or Martin propose this; in fact, they both very focused on how non-designers can learn to think like designers.) I also worry that the term “design” will lose relevance for all the other meanings we rely upon it to convey. As Michael Beirut recently put it, “Don't say design, say innovation, and when innovation doesn't work, make sure you saved some of that design stuff, because you're going to need it.”

Given the big challenges we face in terms of the economy, environment and society, I think it’s a great idea that everyone learns more about creatively engaging with mysteries through abductive reasoning. Still,there must be a better term than “design thinking” to describe it. Any ideas?

What do you think? Join the conversation in Comments

(Originally published in interactions magazine, I've expanded this a bit to include more examples.)

Musical instruments provide really intriguing examples of user interface design. While it can take years of training and no small amount of aptitude, an instrument in the right hands can provide highly nuanced control over the many aspects of sound that come together to form one of the highest forms of human expression. And even for those of us who will never achieve such heights of virtuosity, merely using such a "user interface" can result a great sense of enjoyment, immersion and fulfillment (what is often referred to as a state of "flow"). Music is almost universally important to human culture, but instruments are not strictly "useful" and it seems strange to think of them as mere tools. That said, from the first bone flutes and stone tools, the evolution of musical instruments has closely paralleled to that of more utilitarian technology. As inventor and futurist Ray Kurzweil puts it, "[musical expression] has always used the most advanced technologies available."

Not surprisingly then, as with so many other things, the dramatic increase of processor speeds has brought about a revolution in the way people use computers to make music. But, while computational power has been a critical enabling factor in this revolution, at least equally as important has been the ongoing evolution of the user interfaces of these new digital instruments.

The Novation Launchpad, a hardware controller specifically designed to work with Ableton Live running on a computer.

A recent history of musical technology and interactivity

As with the broader universe of technology, musical instruments have co-evolved with the practice of music. New technologies are often first introduced as a way of replicating and incrementally improving upon a previously established way of doing things, and then they may eventually point the way to something entirely new. In the same way the first cars were designed as "horseless carriages," synthesizers were largely first looked to as means to emulate the sounds of acoustic instruments, and it took decades before electronic sounds became aesthetically appealing in their own right. Starting with the concept of the instrument, of course we still have all the traditional kinds of sound-makers from percussion to strings to brass to woodwinds. And then we also have the first generation of plugged-in versions of these: electric guitars and pianos, for example. The next step from there was the synthesizer, which brings us a closer to where we are today.

Synthesizers are electronic instruments that employ a variety of techniques to allow users to explicitly control different qualities of the sound, including pitch, harmonic content, duration and how the sound changes as it is played. These are typically controlled by a piano keyboard for pitch, as well as other hardware controls (such as knobs, sliders, dials and buttons) for other aspects of the sound.

The Sequential Circuits Prophet 5, a classic analog synthesizer.

The first synthesizers, such as the Moog, used analog circuitry and a style of synthesis referred to as "subtractive synthesis" where an oscillator (or several) is used to generate a tone, and filters are used to remove harmonic content from that tone (hence are subtractive). While these machines are capable of creating deep, rich sounds, they have a somewhat limited sonic palette, and are not terribly effective at emulating more traditional instruments.

From a user interface perspective, synthesizers provided an interesting leap forward. Where a violinist must train for years to gain the dexterity to adjust the timbre of their sound, someone playing an analog synthesizer is able to make such an adjustment by twisting a knob or two. By making this control explicit and direct, it greatly flattens the learning curve (though to be fair it has proven no shortcut to the highly nuanced styling of a master instrumentalist).

In the 80's the first digital synthesizers were created. These machines employed a variety of computational techniques that were made possible by advances in microprocessor technology. While these sophisticated synthesis techniques were (sometimes) capable of more natural and expressive sounds, what was in essence the grafting of a computer onto a piano keyboard resulted in a dramatic increase in user interface complexity. The direct manipulation provided by the filter cutoff knob on an analog synthesizer was completely lost in favor of page after page of parameters presented on the small LCD screen of digital synthesizers like the Yamaha DX7

The programming screen of the Yamaha DX7

Most recently, there have been a couple significant evolutionary steps in the world of synthesis. As the processing power of personal computers increased through the 90's, it became possible to create synthesizers that run as applications in Windows or MacOS. These "softsynths" are particularly useful because it's possible to integrate them quite seamlessly into a computer-based production environment (often called a DAW or Digital Audio Workstation, discussed further below). Also, because they are able to rely on significant computing power and large graphical user interfaces, they can present both novel methods of synthesis (such as physical modeling, which uses principles of physics to model the sound emitted by things like plucked strings), and most significantly for this article, novel methods of control (more on this a bit later).

Native Instruments' FM8, a softsynth loosely based on the DX-7.

Closely related to the synthesizer is the sampler, which plays back recorded sound in a musical fashion. Some samplers are designed around short samples of a individual instruments, which, for example, allows one use a keyboard to control the sonic diversity of whole orchestra. Other samplers are useful for manipulating and looping longer phrases of recorded audio, and were first popularized by hip-hop producers in the late 80's.

Along with the evolution of the synthesizer and sampler came that of the sequencer. With its heritage in the paper rolls and mechanisms of the player piano, the idea behind a sequencer is a way of recording, playing back and editing a control signal that tells a synthesizer or sampler what notes to play, as well as other information about how they should be played. From an interaction perspective, this was quite revolutionary. A musician or producer could record the notes to be played in song, and while playing them back with the sequencer, continue to modify and adjust the sonic characteristics without ever having to make an audio recording.

The sequencer also provided musicians with a way of coordinating several synthesizers at once, but unfortunately, this often required sorting out piles of cables and settings between the various devices. Originally taking the form of a standalone hardware box (or part of a drum machine or sampler), the sequencer was the first part of the electronic music production environment to find its way onto the personal computer.

At first glance, mucking around with a sequencer is not a terribly musical activity, and can be a great distraction from the actual playing of music. This strikes at the heart of a typical conundrum faced by today's product designers: adding sophisticated and (potentially) useful capabilities to a person's toolset can also add the significant overhead of managing those capabilities. Many musicians complain that if their equipment setup is too complex it becomes easy to lose track of their musical ideas because they have to spend so much attention managing technology. However, for many electronic musicians and hip-hop producers, the sequencer itself has become an instrument in its own right—after hard practice it is possible to achieve an impressive level of virtuosity in live performance.

Hip producer and performer Exile using the sampler and sequencer capabilities of the Akai MPC 2000XL as a live instrument.

The final bits of the contemporary music technology equation are the components around the recording and manipulation of sounds. Where in an acoustic live setting, the qualities of sound are entirely determined by the musician, the instrument and the acoustics of the room, in the contemporary world of recorded music, the number of factors that influence the sound grow exponentially to include all flavors of recording and distribution technologies. While the importance of the musician has not necessarily diminished (though this is, I suppose, debatable), what is done with sound after it leaves the instrument has become dramatically more important. While this is not necessarily a new development—the concept of the studio as an instrument goes back at least to the 60's and the pioneering work done at Abbey Road, as well as by Brian Wilson and Phil Spector—it is at the core of recent evolutions in digital music technology.

These include: the mixer, which combines sounds from multiple sources while adjusting the volume of each source as well as shaping the sounds (by way of an equalizer); the multi-track recorder, which can record several sound sources at once while maintaining their independence (ultimately to be mixed together later); and effects devices, such as distortion and echo boxes, change the character of the played or recorded sounds. Once again, while there are certainly many prominent producers who stand by their 60's era analog mixing desks and still prefer to record to 1/2" tape, here again increased processing power has meant that all of these capabilities can be found on a personal computer.

In fact, today, software such as Ableton Live, Apple Logic, Steinberg Cubase and Digidesign ProTools provide the capabilities of synthesizers, samplers, sequencers, mixers, effects and recorders, all through a single, integrated environment. (These are commonly referred to as Digital Audio Workstations or DAWs). While these are not universally simple to use, in many cases the utility and usability of these applications is a big improvement over the technology that came before them (though there are still plenty of folks who debate this point). This fact, combined with their relative cost-effectiveness has meant that a lot of musicians of all stripes have brought computers into their bedrooms, studios, practice rooms and even onto stage, and in the process they are redefining the way that recorded music is made. No longer does a band need to work according to a schedule in an expensive studio to record their ideas, but rather they're able record as the feeling strikes wherever and whenever that happens to be. This transformation hasn't been universally easy or fruitful, but the computer's new status as a music instrument surely has some lessons for us.

Designing for fluency

Creating music requires a musician to achieve a sense of fluency with their tools. This is why they have to practice so much. It is incredibly detrimental to the musical experience to stop and think about how to operate a piece of equipment; the fleeting and elusive ideas behind a song are easily lost to technical distractions. For a musician to get an idea out of her head, the use of her instruments must be effortless. And this effortless isn't just a state of minimum exertion and maximum efficiency. The experience must allow for playfulness and spontaneity, the enablers of the best of human capabilities. So it seems that these musicians and producers who are successfully composing music and producing recordings using their computers are able to achieve a sense of fluency (or even virtuosity) that is an ideal we should have for all kinds of users of technology: from surgeons to stock traders. The question, then is what about these musical user interfaces contributes to this sense of fluidity.

Simplicity

One of the most important aspects of a good musical interface is simplicity. Guitars, pianos and trumpets are all quite straightforward in the way the present their capabilities. Obviously, the less interface a user is confronted with, the more mental bandwidth they can devote to hear and play music. With fewer choices, the user is both able to explore a nuanced way of using each aspect of an instrument, and is ultimately more able to play notes decisively and meaningfully.

To be clear, the for a user interface to be simple it is not necessarily true that it must also be basic or unsophisticated. Just because a drum is quite straightforward and direct in the way it's capabilities are presented, it is obviously capable of being played with nuance and subtlety, though this takes years of practice. For an instrument to facilitate a feeling of fluidity, it must be learnable, but not necessarily focused on novice use.

While not always successful, the drive for simplicity in the creation of digital instruments has always been there. Greg Hendershott, the creator of Cakewalk, one of the first computer-based sequencers, explains the driving force behind his design decisions:

My design philosophy for the early Cakewalk software was to make simple things simple and to have more complicated things require a little more effort. So anything that you’d be doing frequently… I tried to keep that very up-front in the user interface — a single keystroke — and things that were a little less frequent I’d tried to have buried in a dialog box or away from view. I think that’s one thing that we’ve tried to keep up, although that [gets] more challenging as you have more and more functionality and more features to the software.

Unfortunately, as Hendershott alludes, Cakewalk and other early DAW's quickly grew top-heavy under the weight of features, and it became so that these tools lost their simplicity started to impose their way of working on their users. Ultimately, a new player emerged with renewed dedication to simplicity.

In 2001, Live was released by Ableton, a small Berlin-based company focused on creating a simple tool for creative music performance. As Thomas Bangalter of French pop act Daft Punk puts it, "Live is by definition one of the most transparent and simple [pieces of] creative software I have ever used. Its interface is streamlined and its features, though with a minimal approach, offer endless possibilities. The whole process is easy, simple, and fun, yet with accessible possible sophistication and professional results."

There are several things that contribute to Live's feeling of simplicity. First, the feature set of the product is fairly limited. Where the more bloated DAW's provide every function that could possibly be required in any production situation, the Ableton design team has very intentionally omitted features that could contribute to complexity with adding to its capability as a creative performance tool. Music-related information (such as notes and control data) is best input through performance (e.g. via a keyboard), rather than numerically or through traditional notation in a score (which other DAW's provide).

The Session View of Ableton Live.

Live's user interface also contributes to its sense of simplicity. The entire interface is contained in two main screen states, one optimized for improvisational performance, the other for composition and more structured performances. A musician is typically able to spend an entire usage session in one view or the other, never with a thought to "navigation" (a uniquely unmusical concept). Both screens follow the same structural patterns, which provide access to files, instruments and all music-related parameters without dialog boxes or overlapping windows. Further contributing to the simplicity are the control elements (such as "knobs" and "faders"), which are rendered in a flat style that minimizes the amount of pixels devoted to ornamentation. This makes it easy for users to understand the state of all the controls through effective use of modeless visual feedback (discussed further below), without the visual noise of faux-3d rendering.

Redefinition of the instrument

Another important design decision made by the Ableton team was to respond to and accommodate the changing definition of a "musical instrument." Here we return to the idea of the studio as an instrument, first explored by rock and R&B producers in the 60's, and then fully embraced by hip-hop and electronic musicians in the 80's and 90's. As Robert Henke, member of seminal electronic group Monolake and co-founder of Ableton explains:

Software sequencers and hard-disk recording applications were originally designed as studio tools, replacing tape machines. Historically, they were more aimed towards sound engineers than towards musicians. The underlying idea of timeline-based editing is construction and sculpturing, not so much performing. As a result, those tools fail onstage or in any context where improvisation or interaction with musicians is essential. So all the software Gerhard and I wrote for our own purposes enabled us to interact with the music in real time.

To generalize, it seems that the lesson to be learned here is many product designers (and business people) become unnecessarily constrained by a rigid product definition based upon existing categories, rather than a holistic understanding of user needs and mental models. Taking a step back to observe broader context may present opportunities to deliver something that people will find useful and desirable.

Modeless, visual feedback

Since music happens in real-time, musicians must be able to quickly understand what's going on with their instruments. Most traditional instruments don't maintain state (and if they do, it's visibly and audibly noticeable). In the world of a computer-based production environment, it's quite easy for things to get so complex that it isn't obvious from the sound exactly what's going on with various components. This is where visual feedback can really help. Visual representations of knobs and faders are effective devices here—a knob not only allows direct manipulation, but also a way to quickly understand the current value of the parameter without touching the control or requesting the information (and are therefore modeless).

Recent innovations in musical user interfaces have broken from metaphors referring to our mechanical past to achieve many novel ways of providing visual feedback. Many of the instruments that ship with Reaktor, Native Instrument's modular audio application construction environment, provide ingenious graphical mechanisms for controlling sounds and musical compositions.

The Newscool "ensemble" for Native Instruments' Reaktor. Newscool features a generative sequencer based on John Horton Conway's Game of Life.

Reaktor's Skrewell ensemble.

Physical control

While an elegant visual interface can greatly contribute to a musicians' sense of fluency, focusing too much on the visual can be a problem. After all, music is an auditory experience. Visualizations of music are always an abstraction, and in many ways can stand in the way of a musician trusting their ears and their personal perception of tone and time, which is truly at the heart of music.

One of the things that helps allow musicians to take their eyes off the screen are physical control surfaces. It's very difficult to interact with an onscreen knob with a mouse if one isn't looking at the screen. Also, music is traditionally a dexterous two-handed affair—there's usually more than one thing going on at once, and it helps to use two hands to keep everything moving. Computer operating systems have historically only offered direct manipulation through the mouse pointer in a single place (while this has now changed with Windows 7 and OSX support of multi-touch, the software still hasn't quite caught up).

As legendary musician and composer Brian Eno explains, "I want to use the whole of my body, not just my index finger. One of the problems with a lot of software systems is that they expect you to type, but it uses a part of my brain that I don't always want to be in the music process. I don't want to shift between being a musician and being a secretary."

Quite a number of physical control surfaces have been developed to solve this problem. These range from surfaces with dedicated controls for common functions (such as faders for mixer channels and knobs for equalizers) to banks of knobs and faders that can be flexibly assigned to any aspect of a synthesizer, sequencer or mixer. Not only are there standard piano keyboards, but also pads for triggering samples and even gestural controllers (including interfaces with Nintendo's Wii Remote).

The Faderfox, a compact physical controller for use with laptop-based audio software. See the Novation Launchpad at the top of the article for another good example of a physical control surface. One of the most exciting control surfaces is Jazzmutant's Lemur which provides a multitouch screen (beating the iPhone to market by years) that can be flexibly configured to provide a number of different types of touch and gestural controls. It seems pretty much a foregone conclusion at this point that the capabilities of the Lemur will be largely available in some form on the iPad and other tablet platforms in the not too distant future, dramatically driving down the price of highly configurable and touch-based controls.

The Jazzmutant Lemur, a highly customizable multi-touch control surface.

Video of musician and sound designer Richard Devine demoing a Lemur-based controller for Native Instruments' Absynth.

Balance between structure and experimentation

One of the hallmarks of the best of modern music, from jazz to pop is a simultaneous reference to and departure from conventional forms. In most cases, either slavishly following the rules or blindly breaking them will bore the audience. Similarly, for musicians to work effectively, they require a fair amount of structure so there's not the need to reconsider first assumptions every time they want to play a note, but there's also the flexibility to reconfigure their instruments to chase that sound they hear in their minds.

Most DAW's allow for this with a straightforward architecture for recording and sequencing, that also accommodates more arcane, cross-modulated routings. Perhaps the best example of this balance between structure and experimentation comes with Native Instrument's Reaktor, which is a modular musical toolkit. The product ships with an arsenal of highly innovative synthesizers, samplers, drum machines and other musical inventions, but the inner workings of each are accessible and drastically modifiable through a visual construction environment. Users also build and share their own devices either from primitives or components borrowed from various other devices.

The underlying structure of Photone, a Reaktor ensemble.

What makes this experience quite effective is that one can start from something useful (and even inspirational) and make small adjustments to come closer to an imagine sound or way of working. This is a great approach for all kinds of interactive tools. Often customization is used as a proxy for design. The fallacy here is that while some people can and do tweak products to get them just right, the starting place is hugely important.

Looking forward

Clearly, the evolution of musical technology isn't over. If anything, it's accelerating and changing course. Dissatisfaction with the computer as a musical platform has led to renewed interest in musical instruments as physical devices. Yamaha's Tenori-On is perhaps the most dramatic of recent inventions in this arena. Designed by the renowned Toshio Iwai, this beautiful device provides an LED-lit grid of buttons as a novel user interface for musical improvisation and composition.

The Tenori-On.

Video demo of the Tenori On.

Even game consoles provide new avenues for innovation. Electroplankton, a game for the Nintendo DS (incidentally also designed by Iwai) allows users to create music by interacting with small onscreen "organisms", each of which has a unique control idiom and resulting sound.

Video demo of Eletroplankton.

And of course, the iPad and other medium-sized touchscreen tablets certainly offer incredible promise as a platform for musical instruments. Getting away from keyboard- and mouse-based conventions has the potential to be a huge enabler of more direct, flow-conducive experiences. If the iPad is really the computer for hanging out on the couch, then it might just be an ideal form factor for noodling on a musical instrument.

Smule's Magic Piano provides play-along guidance using the familiar Guitar Hero / Rock Band idioms, plus allows for a radial keyboard.

The application of these inventions to more utilitarian digital products may not be universally obvious, but there's a great potential to follow the lead of musical technology. Many attributes of a successful musical experience can be applied to all kinds of venues where critical decisions must be creatively made in real time. With appropriate application, the design strategies discussed here can help us create products that better allow for the kind of fluency and virtuosity that represent our best abilities.

What do you think? Join the conversation in Comments

Everyone knows that the iPhone is pretty great. The vast majority of my clients offer it up as their first example when I ask them, "What products on the market that represent the kind of experience you want to deliver?"

I mostly really like mine. But I've got to say there are a couple things about it that really bug me. Right up there after the fact there's no one-gesture way of switching between different email inboxes is the way the little red notifications circles work with the phone.

It's a bit confusing, plus requires unnecessary work

Whenever someone calls me, I don't answer, and the caller leaves a voicemail, a "2" is displayed in the little red circle over the Phone icon on the Home screen like this:

Maybe I'm kind of a simpleton, but doesn't that kind of make it seem like I've missed two calls? Or that I've got two voicemails?

And that isn't the worst of it. As confusing as that is, after using it for a while, I now mostly remember how it works (and even if I don't, it doesn't really cause me any real inconvenience.)

The really irritating part is when I go to the phone application, there are now two new red dots — one over Recent and one over Voicemail, like this:

Every single time, after I go listen to the voicemail, I have to click over to Recent to make that red dot with the number in it go away. Of course I know I missed the call, I've already listened to the voicemail. Why do I have to actively get rid of this extra dot?

You might be thinking "Relax, idiot. This isn't Ms. Pac Man. You don't have eat all the red dots." But I kind of do, don't I? Otherwise, the red dot starts to become useless. This might be fine with all your customers who bought an iPhone to replace their Razr, and those who don't have expectations of their phone as a productivity tool. But my mobile is actually a pretty important part of the way I manage my work and more importantly, my attention.

What if we changed things around just a bit?

The easiest fix is to just change the logic so that for any missed call, you only display one circle. If they leave a message, it's over Voicemail; if they don't, it's over Recent. (Which works for the transition between when it's just a missed call, and when they've left a message. The number just switches from Recent to Voicemail when a message is left.)

But it still kind of bugs me that a given phone call can be represented in two different places. It seems a lot simpler to have a single list of calls to scan through when I pick up my phone after a meeting. Maybe it could look something like this:

It would work like a combination of Recent and Voicemail...

...with filtering capabilities at the top, as in your Recent screen...

...and a dot next to incoming calls that have resulted in a voicemail message. (Clicking on the dot would bring up the Voicemail playback controls). I also propose changing the way the color coding works, so the red text, which formerly indicated a missed call, would now be used to signify unheard messages and unseen missed calls. This maps better to the color of the little red circle, which seems appropriate since the number in the circle is the now the count of red items.

Because I changed what red text means, we need some other indicator to differentiate missed calls. What if we did this with the icons? I didn't really have time to get into the pixels to solve this, but you get the idea with this placeholder.

You'll also notice that by combining Recent and Voicemail, I freed up some space in the navigation, which is convenient, because we need somewhere to put access to the outbound greeting. (I couldn't really decide between shaka, the devil's horns and a waving hand for the Greeting icon. Shaka seemed most in keeping with the playful iPhone experience.)