Play Button

Gaming Tech and 3-D are Changing the OR

State-of-the-art technology can streamline surgical procedures

Gaming Tech and 3-D are Changing the OR

8 Minute Read

A 72-year-old man suffering from nasal polyps and chronic sinus inflammation lay unconscious on the operating table. Blue surgical towels veiled nearly every inch of his body, but his face was exposed—his head tilted back and nose pointed toward the ceiling.

Martin Citardi, M.D., inserted a long endoscope through the patient’s nasal passage. The glow of a monitor illuminated the darkened room. Everyone’s eyes were glued to the video camera feed on the screen, which looked more like a video game than a real-time view of the winding journey through the patient’s sinus cavity.

Memorial Hermann-TMC surgeons use Stryker’s Scopis TGS to perform sinus surgery.

Citardi and his team at Memorial Hermann-Texas Medical Center are the first in the country to use augmented reality (AR) technology for minimally invasive sinus procedures.

AR technology, which overlays computer-generated images onto a user’s view of the real world, gained mainstream popularity after Pokémon Go first launched two years ago. At its peak, 45 million users were hooked on the gaming app.

But AR isn’t just for gamers anymore. Medical device companies are beginning to use AR technology to superimpose preoperative medical imaging over real-time video of a patient’s anatomy, creating a composite view for surgeons.

Typically, Citardi would simultaneously analyze a 2-D CT scan on a light box prior to surgery and mentally visualize the patient’s 3-D anatomy, going back and forth between the two. But when Citardi uses Stryker’s Scopis Target Guided Surgery (TGS), an AR navigation system, all the relevant imaging is combined into a single field.

He marks up preoperative images to identify which areas to dissect and which critical structures to avoid, then overlays the annotated scans directly on top of the endoscopic view of the surgical area to create a clear, video game-like image of the pathway to follow.

The Scopis TGS is analogous to a 3-D GPS navigation system: Doctors can track the surgical instrument’s position relative to the “map” of a patient’s preoperative imaging.

“Surgeons are visual creatures. Putting everything together on one screen is appealing,” Citardi said. “A big part of minimally invasive surgery is getting directly to the target and avoiding collateral damage. The ability to have those overlays facilitates that process. It takes even more of the mental gymnastics out of it.”

Since X-rays were first discovered in 1895 by German mechanical engineer and physicist Wilhelm Conrad Röntgen, medical imaging technology has made great strides. Positron emission tomography (PET) scans and nuclear medicine imaging were introduced in the 1950s. In the early 1960s, Ralph Meyerdirk and Bill Wright developed the first commercial ultrasound scanner, specifically designed for medical use.

In 1979, G.N. Hounsfield and A.M. Cormack won the Nobel Prize for their invention of computed tomography (CT) scans that captured cross-sectional images of the body. Also in the 1970s, American physician Raymond Damadian invented the magnetic resonance imaging (MRI) scanner.

These breakthroughs in medical imaging have become mainstays in medicine, helping doctors to accurately diagnose patients and surgeons to meticulously plan complicated operations. Yet the visualization of that information remained archaic.

“It sounds intuitive to have all the relative information on one screen, but we’re long overdue,” Citardi said. “Scopis is an important step on that path to image convergence.”


Scopis’ AR-assisted surgical navigation isn’t the only trending technology in the operating room. Surgeons in the Texas Medical Center recently used 3-D technology to perform intricate procedures.

In March, pediatric surgical specialists at Texas Children’s Hospital used the Orbeye for pediatric plastic, urology and congenital heart surgeries.

Developed by SOMED, a joint venture between major imaging powerhouses Olympus and Sony, Orbeye is a new freestanding videomicroscope that displays high-resolution 3-D imaging of surgical procedures on an expansive 55-inch 4K monitor with stunning clarity. It’s like bringing the IMAX movie theater experience into the operating theater.

“You put the glasses on and it literally almost comes out at you on the screen. It was like watching [a movie] in 3-D,” said Chris Pederson, M.D., a pediatric plastic surgeon at Texas Children’s who has used the Orbeye to perform reconstructive microsurgery on babies.

Texas Children’s pediatric plastic surgeon Chris Pederson, M.D., wears Orbeye 3-D glasses, instead of magnifying loupes, allowing him to look straight ahead at the 55-inch screen to perform surgery. (Photo courtesy of Texas Children’s Hospital)

Pederson called the device a “game-changer,” especially for microsurgery, including pediatric and reconstructive surgery.

“We’re able to see better and do better. We take care of kids with smaller spaces and smaller anatomy, so we need this technology,” said Chester Koh, M.D., pediatric urologist and director of the Pediatric Robotic Surgery Program at Texas Children’s, who spearheaded the effort to bring the Orbeye to Houston. “This is not just another piece of technology. It’s a needed tool that we’re using that will help us take care of kids better.”

Chester Koh, M.D., uses the Orbeye 3-D video microscope in March 2018. The small camera can be easily adjusted to view hard-to-reach areas. (Photo courtesy of Texas Children’s Hospital)

Typically, microsurgeons perform the initial part of the procedure using standard magnifying loupes to help them see tiny tissue and blood vessels, then roll in a cumbersome microscope to complete the next step of the surgery. Surgeons often spend hours with their heads slumped over, straining their necks and looking down at the operating table through either loupes or a microscope lens. But the Orbeye is designed to eliminate that discomfort.

By combining both the optical capabilities of the loupes and microscope into a single camera and screen, surgeons can swap out their traditional lens for 3-D glasses that allow them to stand upright.

“The original microscopes were invented much like a microscope you’d find in a lab. You’re hunched over and you have to get the eyepieces just right. They’re fine, but you have a very narrow field of view,” Pederson said. “With the Orbeye, you don’t have to lean over to look in the microscope. You suddenly have this giant field of view [on] this whole wall behind everyone. Hopefully, the next generation of surgeons won’t have neck and back problems like I do.”

Because the Orbeye is so compact, it doesn’t occupy as much space in the operating room and requires less time to set up.

The camera—the size of a soda can—is attached to a flexible arm that can be easily maneuvered and angled in different positions based on what surgeons need to see. The large monitor allows everyone in the room to view the surgical procedure and coordinate more effectively.


“We move body parts around from one place to another … and are often operating in places that are difficult to get both surgeons to see at the same time,” said reconstructive plastic surgeon Jesse Selber, M.D., director of clinical research and associate professor in the department of plastic surgery at The University of Texas MD Anderson Cancer Center, who used the Orbeye in April to perform a breast reconstruction surgery. “With the microscope, you have two people connected [on opposite sides] to it. The Orbeye would allow both surgeons to be on the same side of the bed or people to be in different body positions that the microscope might not allow you to do due to physical constrictions.”

The idea of integrating 3-D visualizations in the operating room isn’t new. Different iterations of 3-D operative imaging have appeared throughout the years, each version fueling the surgical community’s general excitement and optimism around the growing applications of this technology. But Orbeye, which won gold in the digital imaging category at the 2018 Edison Awards last month, could help standardize 3-D imaging in the OR.

“Screen-based magnification has long promised to deliver high-resolution 3-D magnification, but has never been good enough to replace lens-based optics until now,” Selber said. “Since screen optics have improved much more rapidly than lens-based optics in recent years, I expect this technology to eclipse microscope technology over the next several years.”

MD Anderson reconstructive plastic surgeon Jesse Selber, M.D. (right), uses the Orbeye to perform breast reconstruction on April 18, 2018.

SOMED is currently pending approval by the U.S. Food and Drug Administration for Orbeye’s infrared light technology, which penetrates tissue more than normal light, allowing surgeons to view underlying tissue.

“This machine may have some potential in making something invisible visible to the surgeon,” said Iki Adachi, M.D., a congenital heart surgeon at Texas Children’s. “There is significant potential in how the machine can change the way we operate very drastically.”

‘Be there in the planning’

AR technology and 3-D visualization are transforming the operating experience for surgeons, and medical device companies are paying attention to this growing trend. According to market research agency Arizton, the global market for such technologies in health care is projected to surge from $0.6 billion in 2017 to nearly $3.04 billion in 2023—a compound annual growth rate of more than 31 percent.

“Technology just gets better and better in terms of new generations of robots and new generations of microscopes,” Koh said. “The key is to be at the forefront—not wait for someone to build it, but be there in the planning, in the testing, to benefit our patients as soon as possible.”

Of course, any emerging technology draws skeptics.

“Any time you introduce new technology, there’s a million people who will line up to say there’s no reason to do anything any differently than the way we either do now or have always done,” Selber said. “It’s the reason why people used horses and buggies for 10 years after automobiles were invented and available.”

As medical technology continues to evolve and hospitals race to gain competitive advantage over one another, patient care must continue to be what drives innovation forward, Citardi said.

“We always have to find ways to do better by our patients. It’s as simple as that,” he said. “If everyone thought good enough was good enough, we’d still be doing the same things the same way as we did decades ago. There’s always room to make things better.”

Back to top