How mixed reality is changing the game for healthcare, from performing live surgeries to delivering ultrasounds in 3D
2018 is already proving to be a transformative year for doctors and healthcare organizations through mixed reality
Last year around this time I had the chance to write a post about one of my favorite mixed reality uses cases – healthcare.
Since that time, we have continued to see remarkable momentum for mixed reality across numerous facets of the industry. This week at HIMSS our team had the privilege to once again be on the ground to see and experience how customers and partners are embracing mixed reality to transform the way they innovate, support, and help others.
Coming out of the show I wanted to take a moment to spotlight some of the innovation work we are seeing from organizations who are working to forge the future of healthcare.
CAE Healthcare
Joining us onsite at HIMSS this year was CAE Healthcare. CAE is an industry leader in the development of simulation-based technologies, curriculum, and resources that improve clinical performance. I talked about some of their earliest work with Microsoft HoloLens last year – well, they certainly haven’t slowed down!
CAE VimedixAR is the first ultrasound simulator to integrate HoloLens. Freed from the limits of a two-dimensional environment inside a monitor, healthcare professionals can display, enlarge, turn, and rotate realistic-looking anatomical parts, or command them to return into the manikin body. Users and those learning with this technology are able to witness (in real time) the ultrasound beam as it cuts through human anatomy.
https://vimeo.com/201466785
With CAE LucinaAR, clinical learners can view 3D holograms of a fetus as it descends the birth canal and gain an unprecedented view of anatomy as they learn to manage a shoulder dystocia delivery. CAE Lucina is a wireless childbirth simulator with validated, integrated maternal-fetal physiology for training on all the stages of delivery and the rare emergency scenario. With five pre-configured Simulated Clinical Experiences (SCEs), and new mixed reality modules coming soon, Lucina allows instructors at universities and hospitals to easily integrate simulation and augmented reality into their hands-on educational programs.
With their Abiomed Impella procedural training solution, CAE Healthcare is integrating HoloLens into its simulation-based training platform that allows physicians to practice ultrasound-guided placement of the world’s smallest heart pump. Overlaying the ultrasound manikin, probe and instruments, the CAE HoloLens framework will depict physiological responses and potential complications to help clinicians quickly master and deploy the device without risk to patients.
Adding to its physics-driven simulator for the Medtronic Micra™ Transcatheter Pacing System (TPS), the world’s smallest pacemaker, CAE Healthcare delivered a mixed reality training solution for up to 11 simultaneous learners on its Microsoft HoloLens platform. Learners can practice while viewing interactive holograms of a 3D beating heart into which the Micra delivery system is visible and connected in real-time.
Pearson delivers mixed reality learning tools for nursing professionals
At Bett this year we were thrilled to announce some of the great curricula our friends at Pearson were delivering to train nurses using mixed reality and the benefit of holographic patients. Today, I am happy to share more details about one of their extraordinary pieces of the curriculum – HoloPatient.
HoloPatient is a mixed-reality learning tool for nursing, allied health and medical schools that will deliver simulated patient experiences to more students, at a more manageable cost, and with a greater degree of consistency than ever before. This cutting-edge 3D immersive solution redefines how learners interact with their worlds and each other—bringing learning to life with startling clarity and breathing new possibility into teaching. Designed and conceived in collaboration with expert healthcare educators and simulation facilitators from around the world, HoloPatient uses Microsoft’s Mixed Reality Capture technology allowing tutors and students to engage in a range of learning and assessment activities previously only possible when using actors or by encountering patients in real life.
HoloPatient drives consistency across the delivery of learning content and assessment while also allowing nursing and medical schools to augment their existing delivery of world-class education and training with one of the most promising and engaging teaching and learning use-cases for mixed reality currently available.
The HoloPatient app will be released globally in April 2018, along with a virtual reality version later in the year. By the end of June 2018, Pearson will have released four other learning tools including Anatomy, Chemistry (building molecules in 3D), Math (Area and Volume of 3D objects) and History.
You can find out more at pearsonimmersive.com.
SphereGen
SphereGen is utilizing mixed reality to create innovative solutions designed to increase efficiency and revolutionize the learning process. SphereGen collaborated with St. George’s University to develop the Learning Heart. The Learning Heart is a HoloLens application that assists students in understanding the elements of the Heart. As a study aid and teaching tool, this application can be used individually or collaboratively by multiple HoloLens users. The holograms are fully three-dimensional and can be traversed around and viewed from any position and angle.
DICOM Director is a full product suite designed to enable communication and collaboration between different radiologists, doctors, and medical practices. The backend of this solution enables radiologists to view, transfer and share scans. This solution utilizes HoloLens as a viewer for radiological scans, such as MRIs, cat scans, and several others. Radiologists and doctors can remotely view their scans through the HoloLens and/or a Windows Mixed Reality headset. The viewing functionality enables the user to see the series of scans, an enlarged view of the selected scan (which they can scroll through), and the 3D model reconstruction from the scans.
https://www.youtube.com/watch?time_continue=26&v=L4FhmXsnHTg
Live surgery with HoloLens
One of the most mind-blowing moments in mixed reality occurred just a few months ago. On that day AP-HP, Paris French hospitals group, conducted a live surgery on YouTube, assisted by TeraRecon medical holographic platform leveraging Azure GPU and HoloLens. The surgeon was assisted by TeraRecon and their partners to develop the complete end-to-end solution from the patient surgery planning up to the real-time streaming of the real patent image data (pre-operative 3D CT scan) with its full volume rendering in the highest quality ever seen including segmentation of the CT scan. This live surgery was conduct using sharing experience to share the same patient data across multiple HoloLens with the same holographic space (operating room) as well as sharing the HoloLens view with the remote surgeon.
Visual3D
With a mission to empower doctors, Visual 3D Medical Science and Technology Development CO. LLC) is dedicated to the use of artificial intelligence, cloud computing, and mixed reality to enhance the capability of doctors. By way of research and development, Visual3D’s technology can promote doctor’s ability of perception and appliance of medical information; enhance the remote cooperative abilities of doctors in different places to build a scientific and technological platform for intelligent medical therapy.
Visual3D has already performed over 200 hip, knee, and spinal surgeries in China using HoloLens. In addition to using HoloLens in the operating theater, Visual3D is using HoloLens to assist doctors pre-surgery as both a tool to brief patients on their upcoming surgical procedures, as well as allowing doctors to pre-visualize and rehearse the procedure before entering the operating room.
Next Surgeries
Digital Pages, a leading Brazilian company in the lifecycle management of digital contents, has developed an interactive application for surgery planning using Microsoft HoloLens. The application offers students, surgeons and healthcare professionals a new perspective on the modeling and manipulation of physical and virtual 3D objects. Next Surgeries introduces the use of holographic projections in surgical planning, reducing the total time of surgeries, the exposure of the patient and staff to radiation and risk of infections, blood loss, etc. The application is available on the digital teaching platform RDP Learning integrated to HoloLens, bringing innovation and more security to the health sector.
I look forward to sharing more with you all soon as we continue to shine a light on many of the great mixed reality innovations being delivered by customers, partners, and developers of mixed reality!
Lorraine
How mixed reality is changing the game for healthcare, from performing live surgeries to delivering ultrasounds in 3D
