Augmented Reality Visor to Dramatically Improve Surgery

Employing new photonics technology, European scientists are developing a new Augmented Reality surgical visor in a bid to improve accuracy of interventions, showing anaesthetic and medical data while superimposing a patient's x-ray in perfect unison with their body, meaning surgeons never having to look away during an operation and surgery times reduced by over 20 minutes for every 3 hours.

The VOSTARS ('Video Optical See-Through Augmented Reality surgical System') medical visor is a head-mounted display (HMD) system that is capable of superimposing the patient's x-ray images in perfect 3D unison with their anatomy.

The visor also presents a patient’s anaesthetic data, heart rate, body temperature, blood pressure, and breathing rates, conveniently into the surgeon’s field of vision, in a drive to increase accuracy by focusing on the operation and reduce time by never having to look away.

The project forecasts a significant improvement of the intervention accuracy coupled with a reduction in times spent in an operation and under anaesthetic by at least 11 %.

Although Augmented Reality for surgical procedures has been talked about in academic and industrial research since the 1990s, a tool joining a surgeon's natural perception with patient data has not yet been widely implemented.

However it has been the use of photonics components, with the small, high-luminous micro display, the LED optical waveguide, and the array of microns to project a 2D x-ray image in front of the user that has been fundamental to realising this one time science fiction.

Project coordinator Dr Vincenzo Ferrari, biomedical engineering researcher at the Department of Information Engineering, at the University of Pisa, explains: "With this state-of-the-art, highly ergonomic visor, we intend to provide all the information required to improve surgery. The primary goal is to reduce not just surgery times, but also the time spent under anaesthetic and the cost involved in any operation."

"For the patient, this means saving off 20 minutes of every 3 hours of surgery and the guarantee of an extremely accurate intervention," said Dr Ferrari.

In the same way that a facing camera on a smartphone films moving images, the VOSTARS system works by capturing what the surgeon sees from a head-mounted camera. The system then 'merges' this footage of reality with the patient’s medical images, from CT, MRI, or 3DUS scans.

The central processor, using the most advanced registration techniques available for surgical navigation, then presents a real-time hybrid image on the visor 'dashboard' to the surgeon.

"Imagine driving with a sat-nav: we know how to drive and roughly where to go, but with real-time information, like speed, distance and time presented to us, we can take the most efficient route in the quickest time and perhaps more safely."

"However, rather than having that information on a small GPS screen, our important patient data, like the surgical target in the anatomy, anaesthetic info, breathing and heart rates for example, are all integrated instantly into the surgical visor.

"A clinician can move freely while still seeing the patient, the hybrid x-ray image and all of critical data all at once in a surgical 'dashboard' inside the screen. The surgeon, the patient and the procedure are all at one."

Hybrid
Scientists at VOSTARS are building a hybrid of the two existing AR headsets approaches that combines all the benefits of both a video see-through (VST) system and optical see-through (OST) head-mounted display.

Fabrizio Cutolo, engineer and expert in wearable augmented reality systems, believes neither VST nor OST alone was suitable for operating on a live patient: "When operating on a real person, it was clear to us that the benefits of OST and VST systems could be combined to make a hybrid device."

In 'optical see-through' (OST) systems, as in Microsoft Hololens, the user has a direct view of the natural environment with the computer-generated images superimposed on a user's field of vision using a semi-transparent mirror. This gives the user a more naturalistic experience, superimposing small amounts of the virtual onto the real world.

With 'video see-through' (VST) systems (as seen in the Occulus Rift headsets), the user is submerged in the virtual world, seeing through a closed head-mounted display (HMD) together with stereo cameras, experiencing life through screens. Since VST systems capture the video-image as seen by the user in real time it is good for aligning the real and virtual worlds.

"For something as critical as an operation, we had to have the naturalistic 'feel' of OST, while having the fluid interaction of the VST. Therefore a brand new device had to be made from scratch, rather than extend an existing technology. The VST-OST Augmented Reality hybrid was born."

Already 3 months into the 3 year project, VOSTARS aims to have a working prototype of the hybrid device ready for May of 2018. Initially being trialled on a number of procedures to the head, including Maxillofacial (jaws and face) surgery, neuro surgery, ENT (ear, nose and throat) and orthopedic surgery, the project hopes to be available to end users in 3 years, with mass production by 2022.

Coordinated in Italy at Department of Information Engineering and the EndoCAS Center for Computer Assisted Surgery (Pisa University), the VOSTARS project received a grant of €3,816,440 from Horizon 2020 via the Photonics Public Private Partnership.

Participants from four European countries include: (Germany) SCOPIS GMBH, Charité - Universitätsmedizin Berlin, Sankt Gertrauden Krankenhaus GMBH, Technische Universitaet Muenchen, Pilotfish GMBH; (Italy) Alma Mater Studiorum-Universita Di Bologna, Scuola Superiore Di Studi Universitare E Di Perfezionamento Sant'Anna; (United Kingdom) VREO Innovation; (France) Optinvent, Commissariat A L'Energie Atomique et aux Energies Alternatives.

For further information, please visit:
http://www.vostars.eu

About Photonics21
Photonics21 is the European Technology Platform (ETP) for photonics, a technology encompassing all of the products and processes around the emission, manipulation and detection of light. Photonics is integral to a wide range of industries that include the medical, healthcare, transport, manufacturing, and telecommunications sectors.

"Photonics21" was set up in December 2005 to bring the community of photonics researchers and industries together. The European Commission defined photonics as one of five European Key Enabling Technologies (KET's) in September 2009. Shortly after, the European Research & Innovation Program "Horizon 2020" invited Photonics21 to become a "Public Private Partnership" (PPP). The "Photonics 21 Association", a legal entity under Belgium law, became the private contract partner in November 2013 in a Public Private Partnership (PPP) in conjunction with the EU Commission.

Today Photonics21 represents more than 3000 personal members from across Europe and abroad. Our members are experts in the photonics industry, research organisations and universities who actively engage with us to develop a joint photonics strategy for future research and innovation in Europe.

With the global photonics market growing from €350 Billion in 2011 to €447 Billion in 2015, Photonics remains a strong industry. The European photonics industry, estimated to be worth €70 billion, has considerable global leadership positions and employs over 300,000 people directly.

With positive growth forecast, current industry trends like digitalisation, resource efficiency, individual and zero failure production will drive the photonics industry further.

Most Popular Now

Augmented Reality Visor to Dramatically …

Employing new photonics technology, European scientists are developing a new Augmented Reality surgical visor in a bid to improve accuracy of interventions, showing anaesthetic and medical data while superimposing a...

Read more

Interactive Health Apps May Inspire Heal…

Just like real doctors and nurses, online health tools with good - but controlled - communication skills can promote healthier lifestyles, according to researchers. However, if their tone is conversational...

Read more

Call for Papers: EHB 2017 - IEEE Interna…

22 - 24 June 2017, Sinaia, Romania. The 6-th edition of the International Conference on e-Health and Bioengineering, EHB 2017, will take place in the city of Sinaia, Romania. This year...

Read more

UK and Italian Health Tech Firms to Help…

Data sharing ambitions set out in newly published sustainability and transformation plans (STPs) have been given a boost by a new strategic partnership between data management specialist Stalis and Italy's...

Read more

From Health Apps to Nursing Robots - A G…

25 - 27 April 2017, Berlin, Germany. Around the world the subject of e-Health is steadily gaining in importance, whether it involves electronic patient files, online video consultations or any of...

Read more

Philips Teams Up with German Startup One…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, today announced a partnership agreement with German digital health company Onelife Health to jointly develop innovative connected health...

Read more

EU eHealth Competition 2017

The eHealth Competition is an initiative that rewards the best eHealth / mHealth solutions produced by European SMEs (Small and Medium Enterprises). Its objective is to support business success of...

Read more

Philips and LabPON Plan to Create World…

Royal Philips (NYSE: PHG, AEX: PHIA) and LabPON, the first clinical laboratory to transition to 100% histopathology digital diagnosis, have announced its plans to create a digital database of massive...

Read more

Virtual Reality Cognitive Training Game …

Greek researchers demonstrated the potential of a self-administered virtual supermarket cognitive training game for remotely detecting mild cognitive impairment (MCI), without the need for an examiner, among a sample of...

Read more

Technology Boost for Health and Social C…

Health and social care organisations aiming to be fully compliant with the government’s Personalised Health and Care 2020 plan, can now access electronic health record (EHR) and healthcare integration technologies...

Read more

Agfa HealthCare's Health Management Pla…

Agfa HealthCare announces today that its health management platform, including the XERO universal image viewer, has been selected to support the joint Radiotherapy Treatment Project of the Saolta University Health...

Read more

Philips Introduces Advanced Radiology So…

Royal Philips (NYSE: PHG, AEX: PHIA), a global leader in health technology, has unveiled new advanced radiology solutions at the 2017 European Congress of Radiology (ECR). In response to today's...

Read more
(HEALTH IT) SPACE - Take a look at who has just Joined