GE’s Augmented Reality Technology for Ultrasound Training

Training clinicians to properly scan patients with ultrasound probe, particularly in countries with a poor medical education infrastructure, can be a slow process.

While the cost and accessibility of ultrasound is dropping rapidly, the number of people qualified to use this equipment is not keeping up. A team at General Electric is now working on utilizing augmented reality, produced by Microsoft HoloLens, to teach sonography students how to locate and properly scan different organs.

The student wears the HoloLens, which projects a virtual image of the relevant organs onto a physical model of a human body. There’s also an ultrasound probe that can be moved in three dimensions to learn how to properly position it with respect to the patient.

“We position virtual organs in the field of view of the operator, overlaid on top of the mannequin,” said Ratnadeep Paul, lead engineer for augmented and virtual reality at GE Global Research, in a statement. “This allows the technician to position the probe on top of the correct organ. The placement of the virtual organs will be done by live tracking of the patient’s body and using our own proprietary artificial intelligence algorithms.” The team will also create a model of a pregnant woman to help learn how to scan a fetus.

Because the HoloLens is still pricey, the GE researchers also plan on investigating using smartphones and tablets as the interface to overlay a virtual image onto a real mannequin.