E-TAN: a technologically-enhanced platform to assess visuospatial cognition with tangible interfaces.
Our approach is to replace the paper-and-pencil cognitive assessment applying ICT potential and using Tangible User Interfaces (Ishii, 2008).
The goal is to allow to the patient the execution of the task using tangible objects, but allowing a quick digitalization of the results and standadising the procedure.
The E-TAN is a technology-enhanced version of the Baking Tray Task (BTT) and allows the assessment and training of visuo-spatial and attentional abilities, especially for the case of unilateral spatial neglect (USN).
USN is a disorder usually caused by cortical or subcortical injury in the right brain hemisphere, that results in a variety of visuo-spatial and attentional deficits. The BTT allows a hands-on and accurate evaluation of the degree and type of spatial deficit through spatial movements and the manipulation of objects, usually in the form of cubes. Participants are requested to evenly dispose 16 cubes on a surface, “as if they were buns on a baking tray”.
Neglect patients often group the cubeson the right part of the tray, but quantification of performance is difficult or impossible.
The standard scoring of the BTT test is usally a litle bit laborious, for example, the result transferred on A4 paper sheets and performance is evaluated clinically counting the cubes in each half of the tray (see picture above). L-R differences greater than 2 are a sign of neglect.
The E-TAN, gamifies the patient’s experience with the help of a scenario and a tutoring virtual agent representing a cook.
E-TAN consists of an integrated system of concrete objects that participants can manipulate.
The E-TAN platform is a prototype derived from the enhancement of BTT, a tool devised to evaluate of spatial cognition (Cerrato & Ponticorvo, 2017; Cerrato, Ponticorvo, Bartolomeo & Miglino, 2018; Cerrato, Ponticorvo, Gigliotta, Bartolomeo & Miglino, 2019; Gentile, Cerrato & Ponticorvo, 2019).
The tangible physical interfaces in E-TAN are disks 4-cm in diameter.
A 30 fps camera placed on top of the board at a fixed distance, and connected to a laptop computer, can detect the disks when they disposed on a predefined board/surface, thanks to ArUco Markers (Garrido-Jurado, Muñoz-Salinas, Madrid-Cuevas & Marín-Jiménez, 2014).
ArUco Marker tags are often used in augmented reality systems (Cerrato, Siano & De Marco, 2018). They consist of a pattern of black and white tiles containing compressed information, similar to QR codes (see picture below).
Thus, E-TAN detects the spatial coordinates of the objects arranged on the board, and offers the opportunity to standardise the data collection and to store the subjects’ performance in both local and online databases. E-TAN also allows one to record the order and timing in which the objects are placed on the board, thus permitting the analysis of the temporal sequence of subjects’ performance.
- Cerrato, A., Ponticorvo, M., Gigliotta, O., Bartolomeo, P., & Miglino, O. (2019, June). The assessment of visuospatial abilities with tangible interfaces and machine learning. In International work-conference on the interplay between natural and artificial computation (pp. 78-87). Springer, Cham.
- Cerrato, A., Ponticorvo, M., Bartolomeo, P., & Miglino, O. (2018, September). Btt-scan: An ecological and technology enhanced tool to assess visual neglect. In Cognitive processing (Vol. 19, pp. S36-S36). TIERGARTENSTRASSE 17, D-69121 HEIDELBERG, GERMANY: SPRINGER HEIDELBERG.