Revolutionizing Stroke Rehabilitation

A stroke can change a person’s life in an instant. 

The results can mean death or surviving with physical limitations that can require months or even years of rehabilitation, which can be time-consuming, expensive and emotionally draining for both patients and their families. 

It is a reality that affects more than 800,000 people each year in the United States alone and even more globally. With such a vast and pressing need for effective rehabilitation solutions, researchers with the UC Noyce Initiative are exploring innovative ways to make therapy more accessible and personalized. 

Using AI and Augmented Reality in Stroke Recovery With support from UC Noyce, a team of researchers from UC Berkeley, UC Irvine and UC San Francisco is tackling this challenge head-on. Their project, “Scenic MR: A Personalized, Privacy-Preserving Mixed Reality Platform for Home-Based Patient Rehabilitation,” aims to transform stroke recovery by combining augmented reality (AR), artificial intelligence (AI), and clinician-driven customization. This innovative approach will allow patients to receive engaging, interactive therapy sessions from the comfort of their homes, improving access to rehabilitation and enhancing recovery outcomes. 

A First-of-Its-Kind Approach to Stroke Rehabilitation 

ScenicMR is not just another medical app—it is an intelligent, adaptable rehabilitation system designed to provide personalized therapy to patients from the comfort of their homes. By using AR headsets like the Meta Quest, Scenic MR allows patients to follow interactive, AI-assisted therapy sessions while receiving real-time guidance and feedback. 

“We have completed development of our prototype AR app based on our Scenic MR technology and are about to initiate a clinical study at UC San Francisco involving patients with stroke,” explains Sanjit Seshia, Ph.D., lead principal investigator (PI) on the project, Cadence Founders Chair and professor in the Department of Electrical Engineering and Computer Sciences at UC Berkeley. “In addition to evaluating our system, we will conduct interviews with patients to better understand their challenges with at-home rehabilitation and use their insights to inform the next version of our Scenic MR system.” 

Bridging the Gap Between Clinicians and Technology 

One of the biggest barriers to adopting AR-based therapy is the difficulty in customizing rehabilitation exercises for each patient’s unique physical deficits and goals, and then implementing them in a home environment. Traditionally, this would require skilled programmers to design individualized therapy programs—an impractical and costly approach. However, the ScenicMR project solves this challenge through a groundbreaking innovation: clinicians could create and modify patient-specific rehabilitation programs without any programming knowledge. Using the Scenic probabilistic programming language, created at UC Berkeley in Professor Seshia’s group, along with generative AI, clinicians would customize rehabilitation exercises tailored to a patient’s mobility, progress and needs —ensuring that each therapy session remains both engaging and clinically effective. 

“This project can transform the delivery of effective, innovative and personalized health interventions, especially for those who live in rural areas with limited access to care or those with socio-economic limitations,” UCSF co-PI Cathra Halabi, M.D., said. 

Professor Yasser Shoukry, Ph.D., co-PI of the project from UC Irvine, added, “We are collaboratively developing a framework to automate data collection, analysis and decision-making in home-based rehabilitation while ensuring compliance with clinical practice and privacy constraints.” 

The Future of ScenicMR and Stroke Rehabilitation

With clinical trials set to begin at UCSF, the Scenic MR team is eager to validate the effectiveness of their technology and refine its capabilities based on direct patient feedback. Their long-term vision is to expand this AI-powered rehabilitation model to other conditions beyond stroke, making home-based therapy more accessible for patients with limited mobility, transportation barriers, or financial constraints.While the project is still in its early phases, its potential to democratize rehabilitation and empower stroke survivors with personalized, at-home therapy solutions is undeniable. 

“Because of the UC Noyce Initiative support, Scenic MR is moving forward to transform stroke rehabilitation,” Seshia said, “and help make personalized, AI-driven therapy more accessible.” 

Funding has facilitated the hiring of two postdoctoral scholars, one graduate student and ten undergraduates to be involved in the project, thereby training the next generation of researchers and giving them hands-on experience in the intersection of technology and healthcare. Funding has also facilitated the involvement of clinician researchers who care for patients recovering from stroke.This unique collaboration is fostering a new research area that merges artificial intelligence, programming systems, cyber-physical systems, human-computer interaction, neurology, neuro-recovery, cybersecurity and privacy.