Year: 2023
Role: Technical Direction, Product Design
Client: Studio Elsewhere for Harvard Medical School Research Group (MGH)
Media: Web Application, Three.js, various AI models (Midjourney, Chat GPT API, Elevenlabs)
Team: Torin Blankensmith (Software Engineer), Jiyoon Moon (Software Engineer), Eric Anderson (UX Research, Audio Engineering), Yuheng Zhu (3D Artist)
Tend is a web-based platform that couples multi-sensory stimulation with gaming methodology to improve outcomes for people living with moderate to advanced dementia and their are partners (dyads). As Technical Lead and Product Designer, I spearheaded the development of an immersive virtual environment called "Isle of Tend" - an interactive island where dyads explore together, engaging in choice-based behavioral therapy through therapeutic gaming experiences designed to reduce caregiver stress, improve mood, and strengthen their relationship.
Dementia affects over 6 million Americans and their care partners, creating significant emotional and physical burnout and relational strain. Traditional behavioral interventions can help dyads, but are particularly challenging for moderate to advanced dementia cases due to mismatched cognitive abilities between partners. Our Harvard research partners reached out to us to help them find a more engaging approach. For this intervention couple multi-sensory stimulation with gaming technology as a novel approach - creating a multi-sensory stimulation game platform that could engage dyads in choice-based behavioral therapy together.
The challenge was multifaceted: translating this research hypothesis into a platform that could meaningfully engage both users despite varying levels of cognitive function, while addressing the practical constraints of our elderly user base and the diverse relationship dynamics within dyads. Since the overwhelming majority of dyads are elderly, we needed a platform that was easily accessible, inexpensive, and familiar for them to use. Additionally, we had to consider the distinct challenges faced by different types of care relationships - aging spouses navigating their own health concerns, adult children balancing caregiving with their own family and career responsibilities, and other family members or friends who each bring unique emotional and practical challenges based on where they are in their own life stages.
We chose a browser-based solution because it leveraged cloud-hosted assets and browser-native technologies like Three.js for 3D rendering, enabling us to create an engaging, rich interactive experience on a familiar platform without requiring users to download specialized software or learn new devices.
Working closely with Harvard researchers, we translated an evidence-based conceptual model into the platform design. The model targets three core mechanisms: sustained attention through engaging activities that maintain focus for both partners, positive emotions via experiences designed to generate joy and connection, and active engagement through interactive elements requiring meaningful participation from both users. These mechanisms were designed to drive specific outcomes including increased relationship satisfaction and decreased distress in dyads, decreased stress in caregivers, and decreased stress for persons living with dementia.
The team conducted a comprehensive user-centered design process across multiple phases, beginning with focus groups involving clinicians, dyads, and caregivers to understand core needs and establish design requirements. This was followed by eight iterative workshop rounds with 10 dyads for continuous prototype refinement, incorporating rapid data analysis to quickly inform design decisions. The process culminated in an open pilot study to validate usability and feasibility. Throughout each phase, I facilitated user feedback sessions, synthesized insights into actionable design requirements, and collaborated with our multidisciplinary team to implement changes that balanced therapeutic efficacy with user experience.
Platform's key experiences
The platform featured five distinct interactive portals - Sound Garden, Poetry Palms, Photo Safari, Observation Tower, and Lighthouse Point - each designed as click-to-enter experiences with preview functionality that allowed users to understand what they would encounter before committing to an activity. One of the most thoughtful design considerations was our approach to the user interface, where we employed skeuomorphic design principles to create navigational elements that resembled sea glass pebbles. This design choice made the digital interface feel familiar and tactile, reducing cognitive load while maintaining the coastal island aesthetic that unified the entire experience.
Each portal contained a bioexperiential module that combined multi-sensory storytelling with adaptive narrative responses, interactive digital art creation with tactile and visual feedback, musical interaction experiences, and meditative environments with carefully calibrated sensory stimulation. The platform integrated choice-based behavioral therapy directly into gaming mechanics, allowing dyads to engage naturally occurring interactions. Technical implementation focused on creating a responsive web platform that supported real-time collaborative features, enabling shared activities between partners with varying cognitive abilities. The system included comprehensive accessibility features such as audio prompts for visually impaired users, voice recognition capabilities, and simplified navigation pathways that could accommodate different interaction preferences and abilities.
Sound Garden utilized camera-based motion tracking to create a collaborative music-making experience where both partners could contribute through movement and gesture. Photo Safari leveraged AI storytelling tools to guide dyads through collaborative narrative building - participants chose from curated photographs and used speech-to-text to describe elements, with the system generating story segments and prompting new questions to help build an evolving narrative that could continue as long as desired before compiling into a complete story. Poetry Palms offered a poem building experience that began with a rhyme and multiple continuation options, branching across over 50 possible directions with Midjourney-generated images that evolved to match the poem's progression, ultimately compiling participants' choices into a beautiful, personalized poem. The Observation Tower provided click-based exploration where dyads used a virtual magnifying glass to discover secrets hidden within images of different island areas, with each location containing conversation cards designed to spark meaningful dialogue between partners. Lighthouse Point served as a meditative sanctuary, offering dyads a quiet space for reflective time together in a calming environment. Technical implementation focused on creating a responsive web platform that supported real-time collaborative features, enabling shared activities between partners with varying cognitive abilities. The platform's architecture was built around adaptive content delivery that responded to user engagement patterns, ensuring that experiences remained appropriately challenging and engaging for both partners.
