Bio-inspired endoscope provides 3D visible and near-infrared images simultaneously

December 22, 2020

WASHINGTON -- Researchers have developed a new bio-inspired medical endoscope that can acquire 3D visible light and near-infrared fluorescence images at the same time. It features an optical design that combines the high-resolution 3D imaging of human vision with the mantis shrimp's capability to simultaneously detect multiple wavelengths of light.

Endoscopes with 3D imaging capability can help surgeons precisely locate diseased tissue. Adding fluorescence imaging can make cancerous tissue light up for easier removal or highlight critical parts of the anatomy that need to be avoided during surgery.

In The Optical Society (OSA) journal Optics Express, Chenyoung Shi from the Chinese Academy of Sciences and colleagues describe and demonstrate the new multimodal endoscope. Although this is an early demonstration, the new endoscope is designed to directly replace existing endoscopes without requiring clinicians to learn how to use a new instrument.

"Existing fluorescence 3D endoscopes require surgeons to switch working modes during operation to see the fluorescence images," said Shi. "Because our 3D endoscope can acquire visible and fluorescent 3D images simultaneously, it not only provides more visual information but can also greatly shorten the operation time and reduce risks during surgery."

Aiding robotic surgery

Although it could be used for any endoscopic procedure, the researchers designed the new multimodal endoscope for robotic surgery systems. These systems help increase the precision and accuracy of minimally invasive procedures and can help surgeons perform complicated tasks in confined areas of the body. For robotic surgery, the enhanced visual information provided by the new endoscope could help a surgeon who may be in a different room from the patient clearly distinguish various types of tissue in the surgical field.

"Although today's robotic surgical systems require the surgeon to be close by, robotic surgery based on this multimodal 3D endoscope might one day allow surgeons to remotely perform procedures in faraway locations," said Shi. "This could help solve the problem of uneven distribution of medical resources and benefit people who live in areas with relatively poor medical conditions."

The new multimodal endoscope achieves high-resolution 3D imaging using two optical systems to form a binocular design much like that of human eyes. However, in this case, the optical design can accommodate both visible light like human eyes and the near-infrared wavelengths required for fluorescence imaging. This light is detected by a sensor inspired by the compound eyes of mantis shrimp, which not only detect multispectral information but also recognize polarized light. The sensor detects multiple parts of the electromagnetic spectrum by using pixels with different spectral and polarization responses.

To obtain high-quality 3D images, the binocular optical system must have two optical systems with exactly the same parameters. "This places stringent requirements on the processing accuracy of optical components," said Shi. "We were able to accomplish this accuracy using precision optical processing and combined this with chip-based spectroscopy technology to make this multimodal 3D endoscope possible."

Combining visible and fluorescence images

To test the new endoscope, the researchers analyzed its resolution, fluorescence imaging capability and ability to simultaneously obtain 3D images with near-infrared and visible color information. The endoscope performed well and achieved a resolution as high as 7 line pairs per millimeter with visible light-- the same as the best 3D endoscopes used today -- and 4 line pairs per millimeter under near-infrared illumination.

They then used the endoscope to acquire visible color and near-infrared ?uorescence images of three concentrations of indocyanine green. This near-infrared fluorescent label is approved by the FDA and used to label tumor tissues. Although the three samples could not be distinguished by the human eye, they could be clearly distinguished using the multimodal 3D endoscope. The researchers also tested the endoscope's 3D imaging performance by using it to image a toy with many crisscross parts. The endoscope was able to produce 3D images that did not cause eye fatigue, even after a long period of viewing.

The researchers plan to use the 3D endoscope to perform additional biological and clinical imaging. They also plan to incorporate more wavelengths and the ability to sense polarization to provide even more visual information.
Paper: L. Liu, C. Shi, T. Wang, H. Zhang, L. Jing, X. Jin, J. Xu, H. Wang, Bio-inspired multimodal 3D endoscope for image-guided and robotic surgery, Opt. Express, 29, 1, 145-157 (2020). DOI:

About Optics Express

Optics Express reports on scientific and technology innovations in all aspects of optics and photonics. The bi-weekly journal provides rapid publication of original, peer-reviewed papers. It is published by The Optical Society (OSA) and led by Editor-in-Chief James Leger of the University of Minnesota, USA. Optics Express is an open-access journal and is available at no cost to readers online at OSA Publishing.

About The Optical Society

Founded in 1916, The Optical Society (OSA) is the leading professional organization for scientists, engineers, students and business leaders who fuel discoveries, shape real-life applications and accelerate achievements in the science of light. Through world-renowned publications, meetings and membership initiatives, OSA provides quality research, inspired interactions and dedicated resources for its extensive global network of optics and photonics experts. For more information, visit

Media Contact:

The Optical Society

Related Robotic Surgery Articles from Brightsurf:

Decision conflict before cancer surgery correlates with lower activity after surgery
Nearly one-third of cancer patients who decide to undergo surgery for their condition may have second thoughts, and this decision conflict may lead to less favorable treatment outcomes in both the near- and long-term, according to a team of investigators at Massachusetts General Hospital and Ariadne Labs.

Robotic surgery in the COVID-19 era: Urologists take on the challenges
Robotic surgery plays a major role in modern management of prostate cancer, bladder cancer, and other conditions treated by urologists.

Robotic surgery may improve outcomes in mouth and throat cancer
Robotic surgery for patients with early stage, oropharyngeal squamous cell cancer is associated with improved health outcomes, including better long-term survival, according to a Cedars-Sinai study published Thursday in JAMA Oncology.

Examining association between weight loss before bariatric surgery, risk of death after surgery
Researchers looked at whether a patient's body weight and weight loss before bariatric surgery were associated with risk of death within 30 days after surgery using data from nearly 500,000 patients in the US and Canada.

Most deaths related to noncardiac surgery occur after surgery and after discharge from hospital
It's not the operating room that is risky for patients undergoing noncardiac surgery; it's the recovery period.

Robotic surgery for throat cancer not superior to radiation therapy, study finds
In 2012, scientists at Lawson Health Research Institute launched the world's first clinical trial comparing robotic surgery to radiation therapy for the treatment of oropharyngeal cancer (cancer at the back of the throat).

Children requiring thyroid surgery have better outcomes at high-volume surgery centers
New research recently published in the Journal of Pediatric Surgery found that post-operative success rates of pediatric thyroid patients, particularly children who require a thyroidectomy, correlate with the institution's patient volume.

Do negative public attitudes toward weight loss surgery stop some patients from having surgery?
Most patients who qualify for weight loss surgery don't have the procedure despite its safety and effectiveness.

Decoding robotic surgery skills
Researchers from the Keck School of Medicine of USC are looking to technology to help deconstruct expert surgeons' robotic surgery skills so they can create an objective, standardized way to train the next generation of surgeons.

Robotic surgery as effective as open surgery for bladder cancer
Robotic surgery is as effective as traditional open surgery in treating bladder cancer, according to a landmark study published in the journal Lancet.

Read More: Robotic Surgery News and Robotic Surgery Current Events is a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to