VR transforms health care: 3D human model coming to a school near you

GUEST:

Surrounded by darkness, the looming 20 foot skull is so close I can touch it. With the click of a mouse, the 3D model of the human head and neck pivots and I’m inside the eye socket examining this complex system from the inside out. It’s A viewpoint typically reserved for surgeons on the operating table; I’m amazed by the scale and detail of the mechanism that gives us the miracle of sight.

Above: Creative and medical teams examine the 3D Neck and Head project in SimVis’s ‘Lab 01’ immersive media space.

The finished product of three years of work and almost 1 million pounds, the 3D Head and Neck anatomy project has revolutionized the way we study anatomy. Glasgow’s School of Simulation and Visualization’s virtual reality model has been so well received by the medical community, that the school is now completing male and female versions of the entire human body to be launched in early 2018.

‘Best thing since Gray’s Anatomy‘

The 3D VR model of the head and neck has been rapidly adopted by universities and training programs across the U.K., with tens of thousands of interactions annually. According to Dr. Malcolm Skingle (CBE DSc Ph.D), the director and academic liaison for GlaxoSmithKline, “This is the best thing available to train the next cadre of medics and biological scientists. We don’t see this as a commercial opportunity but recognize it as being the best thing since Gray’s Anatomy.”

The School of Simulation and Visualization, or SimVis, partnered with leading hospitals and universities in order to complete the most accurate and detailed model of the head and neck ever constructed. The ultra-high resolution 3D virtual model is fully interactive and can be consumed across a variety of environments and platforms. Students can interact with the model in a group setting, similar to an IMAX theater, or study the model solo on their tablet, laptop or mobile device. The 3D model includes accurate visualizations of the skeleton, nervous system, blood supply, muscles, supporting tissues and lymphatic drainage.

The model also includes haptic or touch capabilities providing feedback through hand controllers that can simulate routine procedures, such as giving an injection. In the past, medical students would practice giving an injection on crude, plastic dummies or on each other. Today, students can perfect this skill safely and expertly without ever touching a patient.

Above: The 3D Neck and Head project include haptics or touch technology that simulates administering an injection to a patient

Blending of art and science

In order to create the medically verified model, the SimVis team worked alongside a clinical verification team of surgeons and anatomy professors including the director of the American Association of Anatomists, professor Anna Lysakowski.

Above: SimVis 3D modeller works to create texture maps for an anatomical model

The creation process brought together 3D modellers and artists, doctors, researchers and professors. This blending of the creative arts and sciences is what sets SimVis apart and allowed for such stunning results.

The workflow begins with actual dissections of human cadavers that are then photographed using advanced photogrammetry. This technique includes taking thousands of high-resolution photos from multiple angles and stitching them together to help form the precise 3D model. Modelling software, such as 3ds Max or Maya, is then used to construct the shape of the model virtually and overlay the exact look and feel through a process known as “texture mapping.”

Once this has been completed, the user interface is constructed and the content is formatted for various platforms with the help of the gaming engine, Unity. Rigorous tests are then conducted in order to validate the model’s accuracy.

Above: SimVis designer shows how real cadavers are photographed at various points and then stitched into the shape underlying the 3D virtual model

The road ahead

Capitalizing on the momentum of head and neck projects success, SimVis is preparing to launch the completed “3D Definitive Human” project early next year. The project is the culmination of a three year partnership between SimVis, the Scottish Funding Council, the Fulbright Commission and regional colleges and universities. The end result will be highly accurate, 3D models of the entire male and female human body.

Above: SimVis Director, Paul Chapman, on stage sharing a preview of The Definitive Human project launching in early 2018

Based upon the positive results and improved learning outcomes from the head and neck project, the implications for the “3D Definitive Human” project are limitless. According to SimVis director Paul Chapman, “These models will result in better diagnosis, improved training and education for medical professionals and will ultimately lead to much better health care for patients.”

SimVis is not alone in seeing the potential that immersive technology, such as virtual reality, has in shaping the future of health care. A US based start-up, Surgical Theater, uses virtual reality to create a model of the patient’s anatomy to help both doctors and patients better understand complex procedures. Microsoft has also been exploring medical case uses for their augmented reality headset the Hololens. One pilot project, dubbed “Project Serra,” experimented with using the Hololens to project MRI data, real-time on a patient being prepped for surgery. This would then allow for more precise targeting of tumors within the body.

VR projects like the “3D Definitive Human” are a great stride forward but we’re still in the nascent stages of truly leveraging the power of immersive technologies to help improve health care.  Combining highly accurate, VR models with actual patient data, the emerging capabilities of AR and AI, and new concepts such as “digital twins” will result in life altering outcomes. Chapman sums it up best, “We can’t even begin to imagine what comes next.”

Lisa Peyton is an immersive media strategist and media psychologist focusing on the business applications of new technologies.

Source: VR transforms health care: 3D human model coming to a school near you