The Future of Surfaces in Healthcare and Beyond

Augmented reality displays, transparent touchscreens, shape-shifting buildings, digital tattoos with screens, living organisms as vibrating information panels, projectors turning any exterior into a control panel: the future of surfaces is as exciting as never before. How could medicine and healthcare benefit from the change in the usage of surfaces in the future?

Beauty and aesthetics of the surfaces around humans

In the cacophony of noise, colors and constant stimuli, humans of the 21st century are yearning for simplicity, cleanliness, and nature. Clear-cut shapes, single colors and natural materials. An escape from the vibrating, fully packed screens, the tumultuous urban environments, the mess of too many objects. No wonder that Marie Kondo and tidying up are so popular lately.

And that wish for simplicity, cleanliness and streamlined forms is how our smartphones got lost their keyboards and buttons in recent years – with other industries mimicking its minimalistic design. An empty surface which can turn into the greatest Swiss army knife ever devised, through interaction, connectivity, and contextual response, could be used almost everywhere. The latest vehicle interiors, our television sets, information screens in shopping malls – all mimic that surface. Thus, we started to contemplate whether and how healthcare adopted that design and what happens to surfaces in the future. What happens to that part of an object which gets into contact with the outside world, including humans, and may have nothing in common with the underlying structure, form or substance?

How do we imagine the surfaces of the future?

When contemplating about the future, let’s first turn to visionaries who already imagined futuristic exteriors, displays, and screens before. The see-through touchscreen with visualized data grabbed the attention of many filmmakers. In Star Trek, health data such as temperature, the state of the brain, lungs, cell rate or blood all appear on an easily decodable, transparent display from the earliest episodes of the series. While Captain Kirk and Dr. McCoy had to be content with the simple computer screen of today, Tom Cruise in Minority Report could go way beyond that. The movie still shaping the imagination of today’s display design containing transparent, floating, holographic screens which you could control through hand gestures and touch.

The combination of nature and technology shows a revolutionary use of surfaces in one of our favorite sci-fi movies, The Fifth Element. In the best scene, when the team tries to use the natural elements and opens up the covers with using wind, fire, water or earth, the visual display of the fluorescent lights combine with the simplest design.

In many respects, the recent Marvel-hit, Black Panther, moved beyond the universe of traditional displays. Although transparent touch-screens and displays emerging out of the blue appeared throughout the movie, electronic information also appeared on unexpected surfaces, such as the head part of the bed on which CIA Agent, Everett K. Ross was healed or on the wall behind the patient. In another remarkable sci-fi series, Westworld, the tri-fold tablet caught our eyes. This somewhat pocketable device is used by every Delos employee for any task that would require a smartphone or tablet today – not just modifying the behavioral programming of Westworld’s robotic hosts, but also mundane phone calls, GPS location tracking, and the like.

Smart wood and shape-shifting tabletop

Although in many cases inspiration comes from sci-fi movies, we agree with Christian Brown, architect and designer when he says that directors don’t necessarily show us the future, but what looks good on screen – be it huge buttons or over-sized letters on displays that the protagonists control with ridiculous haptic moves in front of see-through computers. Thus, while the direction is definitely transparent, minimalist, opaque design, perhaps holographic visualization and see-through glasses, there are many more trends that are not following sci-fi visions.

One of the direction which is visible today is that boundaries between objects start to become blurrier than ever. You cannot tell whether it’s just a simple shelf or the control panel for a spaceship. There are already projectors that turn any surface into touchscreens. Also, the Japanese start-up, Mui Lab, introduced its Mui Smart Display at CES earlier the month, which looks like a plank of wood when it’s not active. And it is actually lumber. A “smart” one. The concept brilliantly displays the latest design trends: the yearning to go back to nature, the need to stay on the technological train and to use any surface as a display.

As Daniel Leithinger of MIT Media Lab’s Tangible Media Group says, “the future of interface design is that we’ll be able to interface with everything, and the line between what we call a computer and what we don’t will eventually go away entirely. Tomorrow’s computers will be furniture, clothing, and more, and the ways we interact with them — and they with us — will be richer than we can possibly imagine.” Their team created the “Transform” project, a shape-shifting tabletop embedded with motion-detecting sensors that can change its shape or transfer physical objects across its surface. Imagine a future, where a kid throws a ball inside a room but that doesn’t bounce back from the surface of the table breaking plates and glasses to shiver, but a motion-sensing table actually catches it and pulls it inside.

Paperfeel tablet and moving buildings

The blurring boundaries between electronic objects and traditional articles for personal use also show elsewhere. For example, current technology is not only able to imitate the look, but also the feel of paper. Thus, you can get rid of all your notebooks and help save the rain forests of the Amazon, but at the same time not give up on the feel of paper. That’s what the reMarkable tablet offers. We suspect that something similar will appear in the case of news magazines and journals – comparable to what was already presented in Minority Report. Another breakthrough from the University of Cincinnati could even lead to rollable, flexible, low-cost, even disposable e-readers.

An additional example of revolutionizing texture, material and surfaces is the development of objects to grow, move around or morph into their environment. For example, one company that has found an alternative to conventional manufacturing methods is biotechnology startup BioMASON. They use a process that’s millions of years old. They inject sand with microorganisms, actually mimicking the way coral grows in nature over thousands of years, but reducing the process to mere days. And what about shape-shifting buildings, rotating skylines or emerging and disappearing staircases? These concepts are not (only) out of fairy tales and sci-fi movies: architects are creating a rotating skyscraper in Dubai where the view changes every single day. Plans for the 80-story residential building were first announced in 2008, and are expected to be implemented by 2020. Breathtaking, isn’t it?

AR projections on surfaces in medicine and healthcare

What does this all mean for the future of healing? Touchscreens, electronic displays, and tablets of all forms also appeared in healthcare in the last years, turning medicine from a paper-based into a screen-based profession. In the future, the GP’s personal computer will disappear into the desk as a single, perhaps rollable touchscreen with the option to visualize body parts or organs through immersive reality or in a holographic manner. While touchscreens are only capable of providing information in 2D, the advantage of VR/AR, mixed reality and holograms is their ability to project data in 3D – and that’s incredibly useful for medical professionals or students.

For example, the Anatomage Table is the world’s first virtual dissection surface perfect for teaching anatomy easily and spectacularly. The Hololens mixed reality device is also an excellent tool for studying the human body. But not only objects can be the space for projecting information. The start-up company AccuVein uses augmented reality by using a handheld scanner that projects over skin and shows nurses and doctors where veins are in the patients’ bodies. Moreover, in October 2018, the FDA gave the green light to Novarad’s OpenSight Augmented Reality System for pre-operative surgical planning, with which physicians can see both the 3D patient image from previous scans projected onto the patient as well as the patient physically in front of them. These examples mean that with the help of technology, physicians and nurses will also “use” patients’ bodies as surfaces for projecting information in the future.

Living organisms as displays

The idea of projecting information on the bodies of humans might go even further by creating screens and displays which combine with our clothes or even biological materials. Sensors, LEDs already appeared on clothing, but an extraordinary example of digital textiles is Caress of the Gaze, a unique, 3D printed garment from Iranian-American designer Behnaz Farahi that bristles under the gaze of the person looking at it. Another concept shows dresses that change based on their surroundings.

Sensors already found their way into clothing related to fitness and wellness, too. Sydney-based startup Wearable X’s Nadi X yoga pants come with built-in haptic vibrations that gently pulse at the hips, knees, and ankles to encourage you to move and/or hold positions. French fashion tech company, Spinali’s UV Protect swimsuit collection is equipped with a removable medallion-style waterproof sensor that aims to stop you staying too long in the sun. OMSignal’s smart bra, the OMbra, records distances run, breathing rates and heart rate, and even tells you when you’re recovered enough to head back to the gym. Sensoria’s second-gen connected socks want you to have the best run possible by offering data on your speed, distance, time as well as style. Also, the concept of SkinTrack goes even further by joining together screen and body part as an interface. The new technology developed by the Human-Computer Interaction Institute’s Future Interfaces Group expands your smartwatch’s “touchscreen” over your entire hand and arm – just by wearing a specially designed ring.

However, what if we even went a zero gravity moon step further. One day it might be possible to combine biological materials with screens, especially as the rise in bioengineering is upon us. Numerous companies are experimenting with 3D bioprinted tissues, creating synthetic skin, bones and other artificial replacements for body parts. Thus, coating organisms with LED lights might not even be that far away. London-based digital design studio Universal Everything even created the concept piece where a screen made of gel was applied to the side of a goldfish, relaying the health information of the fish. Although that sounds extraordinary, it might be superfluous – not like a digital tattoo on your arm displaying your vital signs.

There are already many experiments with the technology. In 2016, MC10 marketed its BioStampRC sensor, a waterproof, band-aid-like patch that sticks to the skin and monitors movement, muscle performance or heart activity. The tiny wearable even has a Bluetooth radio and a miniature battery. Imagine the nanodevice having a teeny-tiny LED display with all your relevant information in the future.

Hospital walls changing colors and acting as displays

Not only the bodies but also such traditional “display materials” as hospitals could be utilized in a more revolutionary way. Transparency, cleanliness, light colors and simple, even neutral or sterile design are also the trademark of health facilities (or at least they aim for reaching it). It is easy to visualize transparent touchscreens or augmented reality panels as information signs or maps in large hospitals. Interactive walls changing colors using various sensors could also play a role in medical professionals’ work – signifying a sudden emergency by turning red, for example.

Moreover, doctors or nurses might have the option to use voice commands or various hand gestures on the floating, transparent screen to give instructions or request a vaccine. Although that sounds science fiction at the moment (close to Minority Report), there’s already a gadget which operates similarly. The user puts Myo on their arm, and the person is able to control a computer from a distance with specific gestures. If there is a USB-stick connected to a laptop meters away, the Myo-user holding a presentation could change ppt-slides, zoom-in or zoom-out with just one gesture in the air. At the moment, the technology should still be developed to be used for such purposes as hospital touchscreen navigation, but we might eventually get there.

On the other hand, hospitals of the future might entirely disappear as we know them today as patients are moving towards becoming the point of care. Thus, we might only experience the new generation of interactive and shape-shifting surfaces in emergency rooms, at the GP’s office or in self-driving ambulances. That’s not gonna happen tomorrow. We should go way down the road to reach those scenarios, but the first time you casually encounter a surface-switching table and a VR wall, you should know, the future has arrived.