Rewriting the Brain—and the Bedside—with Immersive Technology
From the operating room to the ICU, immersive tech is reshaping care. Researchers and clinicians are leveraging holographic visualization and VR to tackle some of neuro- and critical-care’s toughest challenges—from guiding electrode placement in deep brain stimulation with HoloLens-based 3D mapping, to recreating PTSD triggers in controlled virtual environments to refine brain-stimulation therapies, to transforming early ICU mobilization with game-like experiences that boost patient engagement. Together, these tools provide richer physiologic and behavioral data, enable more precise clinical decisions, and open the door to personalized interventions that accelerate recovery and improve outcomes across neurologic, psychiatric, and critical-care populations.
Transcript
00:00:04 The very first time I ever got to see a holographic headmounted display was actually back in like 2013 14 something like that. So a long time ago and uh it was a prototype that Microsoft had built of the original hole lens and it blew me away. I had no idea that this was even possible and when I saw it, it was an immediate aha moment that this has got wonderful opportunities in neurosurgery.
00:00:33 My lab uses technologies like virtual reality to simulate the real world and the experiences that people would have naturally in both good and bad moments uh such as you know symptoms in in the bad moments, symptoms in disorders such as PTSD, like a flashback. And then once we understand how the brain reacts during these moments in a controlled setting like in the lab, we can then
00:00:54 develop treatments such as brain stimulation that can intervene with those moments and you know prevent these symptoms. We have long been excited about the potential of using augmented and virtual reality for physical rehabilitation. There are a lot of studies about the advantages of this for rehabilitation. We were looking for clinical partners and when we discovered
00:01:15 that Duen's team uh has established a clinical need for this technology and are looking for partners, we were extremely excited. The goal is to help neurosurgeons get electrodes into the right part of the brain. And it's a very complicated three-dimensional problem that doesn't work that great when you try and put it into a 2D visualization. The core of
00:01:44 Holo SNS is actually really just fancy ways to visualize MRI patient specific brain imaging data. But what we also do that makes it special is we can overlay those anatomical imaging data sets with electrophysiology data or advanced neuroanatomical data sets. There are many uh neurologic and psychiatric disorders now where patients
00:02:13 are being treated with these devices that are placed in certain brain areas and can record data. they have, you know, basically a very unique window into these brain areas. So, patients with PTSD, many of them have a hard time going out into crowded environments such as, you know, even just the grocery store or Target for instance. And the value here of virtual reality is that we
00:02:36 can create those environments. So we have a grocery store, we have various other environments where we can uh essentially immerse these these participants in these environments and we can also uh introduce various triggers and cues that would normally you know exacerbate their symptoms. It could be a loud sound. It could be you know something else aversive. And then
00:02:56 we could study the brain and also the body. Uh we have all these cameras that can measure their movements and we can study those things together to try to get a picture as to what's going on. uh when a patient uh is hospitalized in an intensive care unit setting for 3 days or more, they encounter the effects of immobility that results in physiologic complications that increase
00:03:24 their length of stay and their mortality. To combat these uh effects of prolonged immobility, the care team is involved in helping patients with an activity progression protocol. But this protocol can be very arduous, boring, and patients are not often able to engage. We had the idea of using virtual reality uh and a gaming perspective to help uh make it more exciting and
00:03:51 interesting for patients to be active early. We uh we wanted to create a beat saber-l like experience that uh people get very engaged with that they uh move uh without even thinking about that this mobility is is beneficial for them. uh but we did want to create the experience that is right for the level of activity that is sustainable in the ICU and as well as we really wanted to tailor the
00:04:20 levels of activity to different um um patients with different mobility restrictions. A lot of challenges in this work are about making sure that you meet the needs of the patients and the clinicians. So there are multiple challenges that arise in that regards. For instance, what movement is actually beneficial for the patients or even down to the level of uh what devices we can
00:04:44 use for tracking patients hands. It's been a slow grind cuz it's really uh engineering intensive to make it work in a clinically relevant environment. Um, but when it does work, it's awesome and the clinicians absolutely love it because it gives them a new way of seeing things and and I would say that we've even made, you know, kind of important clinical decisions based on
00:05:18 the technology that would have been very difficult, if not impossible to do with regular navigation technologies. And so we can capture all of that activity and try to see if we can predict clinical response, if they're getting better, if they're getting worse, if they have symptoms during the day, if they have nightmares at night. So it's a really rich uh in
00:05:40 lab and at home data set that we ultimately are using to develop a new therapy and treat patients with PTSD, Parkinson's, and so on.
>> So for almost a hundred years, we've known that this is a problem not moving for a long period of time. And with the use of VR, we hope to really revolutionize the way that patients in intensive care unit settings can move
00:06:03 earlier and leave the hospital sooner.