Novel evolution to operate the Microsoft Pixelsense in neuro-rehabilitation.


The fun.tast.tisch. project aims to develop an innovative software system for application in neuro-rehabilitation, using the Samsung SUR40 with Microsoft PixelSense (SUR40) as the preferred tabletop. This technology was chosen because it allows not only the recognition of fingers and hands, but also physical objects placed on the surface - exciting prospects in a rehabilitation context.
However, it is most important to emphasize that the new technology is not intended to replace, but rather to support conventional therapy settings (e.g. in taking over administrative tasks and offering decision support). According to an earlier feasibility-study, the Microsoft tabletop provides convincing benefits for use in neuro-rehabilitation (please note that in this particluar case, the predecessor of the SUR40 was used):

  • The table itself is perceived as a familiar object, which helps patients to overcome their timidity when using the computer for their trainings. The table allows therapists to support and motivate the patient easily and still maintain eye contact.
  • The multitouch technology, with a recognition of approximately 50 touches, enables patients to train bimanually as well as with multiuser interaction.
  • Multiuser interaction enables several users to interact at the same time. The tabletop display supports mutual and social interaction as a medium of communication among neurological patients.
  • Object recognition allows for the usage of physical objects directly in the training applications. This enables addressing the patient's "tactile sense" within the training. Individuals with impaired fine-motor skills and cognition abilities benefit from using tangible objects, e.g. rebuilding 3D shapes.

Now, SUR40 with its slim shape, also allows wheelchair users to train on the table, an advantage that was not available in the previous model. The training system developed in the context of the project has to fulfill the respective therapeutic requirements or rather, has to embed existing therapeutic approaches and practices, in order to provide benefits for the patients and therapists. The main target-group of fun.tast.tisch. consists of neurological patients with limited basal and cognitive abilities based on acquired neurological damages of the central nervous system.
The innovative aspect of the project is not limited to the technology used, but by the application itself in the field of neuro-rehabilitation. There are only a few other evaluated concepts and successfully implemented projects in related areas, thus one of the key aims of this unique project is to create and evaluate new human-machine interaction designs for natural user interfaces. This will ensure a highly intuitive, efficient and userfriendly handling for the end-user. To achieve this requires interdisciplinary skills and creative methods such as design thinking and user-centered design right from the beginning of the conceptualization phase (see more details in the Tangram section). Not only should up to ten modules containing different exercises be conceptualized and implemented, but they should also be examined in clinical practice. These trials should ensure a better market acceptance while providing benefits to therapeutic organization as well as to patients.

This project is funded by BMVIT / BMWFJ through the development programme 'COIN – Cooperation & Innovation'.

"After my stroke,
I couldn't concentrate very well anymore.
The game on the touch-screen table was a fun way to train in therapy." 
(Johann B., 60 years)
"In the beginning,
I really had to work at it, but over time, the training became easier." 
(Hermine S., 54 years)