A significant portion of the over 65 population experience a fall at least once a year in their home environment. This puts a faller under significant health risks and adds to the financial burden of the care services. This project explores a proof of concept implementation of a fall alert system that uses MiRo (a small mobile social robot designed and manufactured by Consequential Robotics) in the home environment.
We take advantage of MiRo’s pet-like characteristics, small size, mobility, and array of sensors to implement a system where a person who has fallen can interact with it and summon help if needed. The initial aim of this proof of concept system described here was to act as a demonstration tool for health professionals and housing association representatives, gauging their needs and requirements, driving this research forward.
Once the faller is located, MiRo turns its head and angles its neck to get the best possible view of the faller through two cameras mounted behind its eyes. It the proceeds to ask them to acknowledge they are conscious. This acknowledgement interaction can happen in a variety of different modalities; the faller can either reply vocally to MiRo saying they are “OK”, or they can touch MiRo’s body – MiRo is equipped with capacitive sensors running its entire body length and top of its head.
After it interacts with the faller, the MiRo tries to ascertain if they are moving using computer vision algorithms implemented in OpenCV. These algorithms take every frame generated by the MiRo’s camera and compares it to the next frame. Differences between subsequent images are rendered in contours, and for each difference found, a rectangle is super imposed upon the contours. In order to only select big movements, like those generated by a person moving, only contours of a certain size are considered. If they are moving, MiRo continues to monitor until they get up and logs it as a ‘fall but successful recovery’. However, if they are not moving, or if they do not respond to MiRo’s earlier request for interaction, a notification is send to the appropriate responder.
The aim of this proof of concept implementation is twofold. Firstly, to investigate the use of a social robot as an agent for assessing and evaluating the situation immediately after a fall. Secondly, to gauge how health professionals and housing association representatives would respond to the idea of using small social robots for interacting with fallers in their home environment.
Late-Breaking Report published in ACM/IEEE International Conference on Human-Robot Interaction 2020 (HRI'20) can be found here.
Below is a Youtube video we prepared summarising this project.