Fritzi; Social Voice Assistant
Conversational Social Voice Assistant for Nursing Home Residents
Learn more
Team Members
Khoshnaz Kazemian, Christian J. Spengler, Bareen Miraj
Tools Used
Fusion 360, Figma, Raspberry Pi OS Lite, motionEye, OpenVoiceOS, Microsoft Azure, OpenAI
Duration
October 2023 to March 2024
University
Siegen University
______
Project Context
As the global population of seniors continues to grow, addressing the unique challenges faced by older adults, particularly those living in nursing homes, has become increasingly important. This project aimed to design and develop “Fritzi,” a conversational voice assistant tailored specifically for elderly residents. The primary goal was to reduce social isolation and promote engagement through natural, human-like interactions facilitated by artificial intelligence. By integrating advanced conversational capabilities, Fritzi was envisioned as a companion that could enhance the quality of life for users by fostering connections and providing an avenue for daily interactions.
______
Reseach Questions
The research questions that guided our project were:
- Can an AI-based social companion fill the gap left by a shortage of skilled caregivers and become a surrogate conversation partner for older adults?
- How will older adults appropriate this technology into their daily lives?
Our study aimed to explore these questions through a user-centered design approach, involving older adults in the design process and evaluating their interactions with the prototype in a real-world setting.
______
Problem Discovery
The primary challenges identified in the project centered around the experiences of older adults in care settings. Many residents face social isolation due to a lack of meaningful interactions, which can lead to feelings of loneliness and depression. Additionally, the complexity of modern technology often alienates elderly users, as many voice assistants are designed with a task-oriented approach rather than for conversational engagement. This disconnection means that elderly users may struggle to adopt and effectively use existing technology, exacerbating feelings of isolation.
Through initial discussions and research, several key problems were highlighted:
- Technological Adoption: Many older adults find modern devices complex and intimidating, particularly those not designed with their specific needs in mind.
- Social Isolation: The loneliness often felt by nursing home residents can be mitigated through increased opportunities for conversation and connection, which a conversational AI could provide.
- User Experience Issues: Existing voice assistants frequently lack the natural, engaging conversational flow that older adults desire, resulting in a frustrating experience.
______
Our Approach
We adopted a Research through Design (RtD) approach, which allowed us to combine research and design in an iterative process. The central idea behind RtD is that design itself becomes a method of inquiry. By creating and testing various prototypes of the voice assistant, we were able to learn through each iteration, gradually refining the technology to better suit the needs of older adults.
The RtD approach facilitated a cyclical process: each stage of development—problem identification, conceptualization, prototyping, testing, and reflection—generated insights that informed the next stage of the project. This method ensured that the design of Fritzi was deeply rooted in the real-world experiences of older adults.
______
Project Goals
Enhance Social Interaction
The primary goal of the project was to develop a voice assistant that could serve as a social companion for older adults, facilitating conversations and providing a source of engagement beyond simple command-based interactions.Create an Accessible and Intuitive Design
We aimed to create a device that was easy to use for individuals with minimal experience in using technology. The assistant had to be simple and intuitive, with minimal learning required, allowing older adults to use the device independently.Bridge the Caregiving Gap
In settings where human caregivers may not always be available, we sought to create a voice assistant that could help address practical needs (like reminders, information retrieval, or entertainment) while also providing companionship in an unobtrusive manner.
______
Our Process
Step 01
Engaging Older Adults in the Design Process
______
Co-Design Workshop
Our co-design workshop with older adults in the senior home was structured as a participatory design session, focused on understanding the preferences and expectations of the elderly concerning the visual design and interactional aspects of the voice assistant technology. This early engagement was critical in shaping the design decisions and ensuring that the technology would meet the real needs of its users.
Workshop Goals:
- Understand the visual preferences of older adults for a voice assistant, including the desired form factor, size, and aesthetic appeal.
- Gather insights into how older adults would prefer to interact with the device (e.g., voice commands, buttons, or other interaction methods).
- Explore participants’ comfort levels with AI and technology, identifying barriers and facilitators to adoption.
The workshop lasted 1.5 hours, covering an introduction to voice assistants, exploring real-world examples like ChatGPT, and discussing potential applications. We had seven participants initially, but five left halfway through, leaving only two engaged participants. This dropout was due to logistical issues, as the nursing home failed to ensure the presence of all interested residents, some of whom were absent due to illness.
Despite the reduced engagement, we collected data through observation, note-taking, and audio-visual recordings. Due to the low participation, we couldn’t gather sufficient design feedback, such as wireframes or sketches, and decided to organize a follow-up focus group with a smaller, more targeted group to facilitate deeper discussions. This experience highlighted the need for flexibility in participatory design and the importance of precise participant selection.
Key Insights:
- Visual Design: Participants expressed a strong preference for designs that felt friendly and non-intimidating. Some even suggested that the device have a slight anthropomorphic shape to foster a sense of companionship. This feedback directly influenced our decision to create a design that was approachable and visually comforting.
- Simplicity of Interaction: The elderly users placed a high value on ease of use. They preferred a design with minimal buttons and straightforward interactions. This reinforced the importance of reducing the complexity of the interface, focusing on one-button activation for conversations.
- Familiarity with Voice Technology: While many participants were unfamiliar with voice-activated technology, they were curious about its potential benefits. They expressed an openness to using a device that could help them perform everyday tasks and provide companionship, signaling that, with the right design, the voice assistant could be well-received.
______
Focus Group: In-Depth User Feedback
Following the co-design workshop, we conducted a focus group with a smaller group of nursing home residents to gain more detailed feedback on the assistant’s potential use and functionality. This session allowed us to dive deeper into how the voice assistant could integrate into their daily routines and further refine the design based on their insights.
To facilitate the session, we prepared printed materials and mock-ups, as well as a live demonstration using a projector. This hands-on demonstration helped participants grasp how the assistant would function.
Focus Group Goals:
- Understand the specific use cases participants envisioned for the voice assistant in their daily lives.
- Explore how the assistant could help with practical needs (like checking the weather or setting reminders) as well as provide social interaction.
Tangible Materials in Elderly Workshops
Key Insights:
- Desired Use Cases: Participants were particularly interested in using the assistant for informational tasks such as checking the weather, setting reminders, or knowing the daily schedule. They also expressed excitement about using the device as a conversation partner during times when they felt socially isolated.
- Continuous Conversation: A key theme that emerged was the desire for the voice assistant to support continuous interaction without the need to repeatedly press a button for each prompt. This insight informed later iterations of the assistant’s interaction model, as we aimed to create a more natural conversational flow.
Step 02
Prototyping and Design
______
Hardware and Software Selection
Given the insights from our research, we needed hardware and software that could reliably deliver on the promise of voice interaction without overwhelming the user with complicated technology. We chose the Raspberry Pi 4 Model B (8GB RAM) as the main computing device, along with the Waveshare WM8960 Audio HAT for audio processing. This setup allowed us to create a powerful yet compact system with sufficient RAM and CPU power to handle voice processing tasks efficiently.
For software, we selected OpenVoiceOS (OVOS), an open-source platform for creating voice-controlled interfaces, due to its flexibility and ability to be customized to meet our project’s unique requirements. To ensure reliable Speech-to-Text (STT), Natural Language Processing (NLP), and Text-to-Speech (TTS) functionalities, we integrated Microsoft Azure services, which provided high-quality, responsive voice interactions with minimal latency.
Technical Equipment Inside The Fritzi
______
Design Features
3D Modeling Design
We used Fusion 360 to model the Fritzi, creating a design that was modular and easy to assemble, with a head inspired by human-like robots to create a friendly and approachable form. The 3D-printed shell housed the Raspberry Pi and speakers, ensuring the device was compact but robust enough for daily use.
3D Design in Fusion 360
3D Printing in FabLab
Simple Interaction Model
Users could interact with Fritzi by pressing a single button on a modified USB numpad to activate the listening mode. This design choice eliminated the need for a wake word (like “Alexa”) and simplified the interaction process, as older adults expressed discomfort with repeated wake commands.
Step 03
Real-World Deployment
______
Deployment at a Nursing Home
After refining the prototype based on feedback from the workshop and focus group, we deployed Fritzi at a nursing home for a 10-day testing period. The goal was to observe how real users interacted with the assistant in their daily lives, collect data on usage patterns, and gather feedback on functionality and usability.
Prior to the deployment, we placed informational posters around the facility to inform residents about the upcoming introduction of Fritzi . We placed Fritzi in a central waiting area where residents frequently passed by. Informational posters explained how to use the assistant, and we set up a large display behind the device with instructions on interacting with it.
Challenges Faced During Deployment:
- WiFi Connectivity: The nursing home’s weak WiFi infrastructure resulted in delays when Fritzi processed requests. In some cases, the assistant was unresponsive due to poor connectivity, which frustrated users and decreased trust in the device’s capabilities.
- Interaction Flow: The need for residents to press a button each time they wanted to speak interrupted the conversational flow, particularly for those who were unfamiliar with technology or hesitant to engage in multiple-step processes.
- Environmental Noise: The placement of the assistant in a noisy area meant that background chatter sometimes interfered with Fritzi‘s ability to accurately capture and process voice inputs.
Despite these challenges, Fritzi successfully engaged several residents and staff members, providing meaningful insights into the use of voice assistants in real-world environments.
Placement of Detailed Instruction Poster Behind Fritzi
Results & Future Directions
______
Results
The deployment of Fritzi, our voice assistant, in a nursing home provided valuable insights into its usability and effectiveness. Over the course of 10 days, residents and staff interacted with the assistant, revealing several key outcomes:
- Positive Reception: Participants appreciated the simplicity of voice interactions, finding it helpful for tasks like checking the weather or setting reminders. It also showed potential for social engagement, though some users primarily used it for practical tasks.
- Challenges with Interaction Flow: The need to press a button for every interaction disrupted the conversational flow. Users desired a more natural, continuous interaction, without the need for repeated activation.
- Technical Issues: Connectivity problems due to poor WiFi caused delays in responses, frustrating users. Additionally, some button malfunctions affected the overall experience.
Overall, the testing confirmed that voice assistants have significant potential in nursing home environments but highlighted areas that need refinement for smoother interactions.
______
Future Directions
To improve upon our initial design and address the challenges identified during testing, we plan to focus on the following in future iterations:
- Improved Connectivity: Integrating a mobile hotspot or other connectivity solutions to ensure reliable internet access and reduce response delays.
- Continuous Interaction Model: Developing a system that allows for ongoing, natural conversations without the need to press a button for each interaction.
- Enhanced Error Handling: Implementing better feedback mechanisms, such as visual or audio cues, to let users know when the assistant is processing or encountering connectivity issues.
- Personalization: Adding features like speaker recognition to offer more personalized interactions and enable the assistant to remember previous conversations.
By addressing these areas, we aim to create a more intuitive and reliable voice assistant that can better support the social and practical needs of elderly users in care environments.
