Companion: Enhancing Physical Therapy through Playful Interaction
Companion is a mixed-reality projection-based installation that transforms physical therapy sessions for young patients into interactive, playful storytelling experiences. This project explores how emerging technologies, such as mixed-reality, can be leveraged to enhance patient engagement and improve the overall therapeutic experience.
Problem Space
Research has shown that on an organizational level, better hospital designs can lead to contained costs and more effective care, decreased levels of anxiety and stress, as well as accelerated recovery and shorter hospitalizations in patients. With this in mind, I was interested in exploring scalable physical therapy tools and interactions to make physical therapy less stressful for younger patients, through the enrichment of the environment and experience itself.
User Needs
According to a 2021 research paper, traditional hospital environments often fail to provide engaging, patient-friendly spaces for physical therapy, particularly for younger patients. The lack of dedicated, child-centric exercise areas can make physical therapy sessions feel monotonous and stressful. These environments are typically sterile and utilitarian, which may contribute to increased anxiety and slower recovery times for patients.
Initial Research
I began by analyzing existing technologies and solutions that use creative visuals and interactive elements to engage users. This research phase was crucial in identifying potential strategies for designing an immersive, multisensory experience within a hospital setting. I conducted a SWOT analysis on several relevant examples, which helped inform the direction of my design approach.
Concept Development
Building on the insights from my research, I developed the initial concept of a mixed-reality projection system that reacts to user movements. The core idea was to create an environment that not only supports physical rehabilitation but also enriches the patient’s experience by integrating storytelling elements. This concept aimed to make physical therapy sessions more engaging and less intimidating for young patients.
The Companion installation reimagines physical therapy as an interactive, narrative-driven experience. It utilizes a mixed-reality setup, including projectors, speakers, and cameras, to transform any flat wall into a dynamic storytelling canvas. Young patients can select their favorite children’s stories and act out the roles of various characters, with Companion responding to their movements by playing corresponding audio cues. This approach not only makes physical therapy more enjoyable but also fosters a sense of agency and immersion for the patients.
To better understand how patients would interact with Companion throughout their hospital stay, I developed a detailed journey map. This map outlines the patient’s day, highlighting key moments where Companion enhances their experience, from waking up to completing their physical therapy exercises. The journey map illustrates how audio and visual feedback are integrated into the therapy sessions to create a seamless, engaging experience.
Finalized Concept
Journey Map
Final Prototype
The current prototype, developed using PoseNet, successfully detects and classifies poses, triggering visual and auditory responses accordingly. While the visuals are still in the early stages, the prototype effectively demonstrates the core interactions that Companion facilitates. Future iterations will focus on refining these interactions to better meet the needs of young patients in a clinical setting.
Reflection and Future Directions
Project Reflection
Working on Companion has been an exciting journey, allowing me to explore new technologies and apply them in a healthcare context. This project was particularly challenging as it was my first major coding endeavor, involving the implementation of machine learning models and the development of interactive visual and auditory elements. Through this experience, I gained valuable insights into the intersection of design and technology in the medical field.
Future Enhancements
Moving forward, I plan to redevelop the code from scratch to enhance its functionality and scalability. Future iterations will explore more nuanced interactions, such as adjusting audio speed and volume based on the patient’s movement speed, to provide a more responsive and customized experience. Additionally, I aim to develop features that allow users to train their own models and create personalized audio sets, enabling healthcare professionals to tailor the experience to individual patient needs.
The true impact of this project lies in its scalability and adaptability, which extends beyond the initial user group. By applying UX principles, I’ve ensured that Companion can be tailored to various settings, transforming not only clinical environments but also enhancing everyday experiences for broader audiences. This project has the potential to evolve into a mobile solution, bringing therapeutic and engaging interactions to diverse contexts, from schools to home care, ultimately enriching lives on a larger scale.
Competencies
Experience Design, Mixed-Reality Design, UX Research, User Engagement, Prototyping, Interaction Design
Project Duration
Spring 2022 (6 weeks)
Opportunity
Research presented by Stony Brook Medicine indicates that well-designed healthcare environments and therapeutic play can reduce stress, enhance patient well-being, and lead to faster recovery. By reimagining physical therapy spaces to incorporate engaging, multisensory experiences, we can create a more supportive environment that addresses both the physical and emotional needs of young patients.
Experimentation and Prototyping
Using p5.js and PoseNet, I created an initial prototype that could detect and classify different poses into four categories. This early experimentation allowed me to test the feasibility of using pose detection as a means of interaction in a therapeutic setting. The prototype demonstrated how specific movements could trigger corresponding visual and auditory feedback, laying the foundation for the final design.
Visuals and Infographics
Next Steps
Visual elements are in progress and will be updated upon completion of the new code iteration. These updates will include polished visuals, enhanced journey maps, and additional infographics to better communicate the project’s impact.