CZ37254U1 - A device for the development of cognitive functions - Google Patents

Description

Field of technology

By its very nature, the device falls into the field of such human activities as education and healthcare, neurorehabilitation, social care, etc.

Current state of the art

So far, a phenomenon in the fields of medicine and psychology has used TUI (tangible user interface) technology to enable the blind to work with graphs and diagrams using the so-called MICOO (Multimodal Interactive Cubes for Object Orientation) objects, as described in the article by the authors Manshad, M.S.; Pontelli, E.; Manshad, S.J. MICOO (Multimodal Interactive Cubes for Object Orientation): A tangible user interface for the blind and visually impaired. In ASSETS’11, Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, Dundee Scotland, UK, 24 to 26 October 20IE, Association for Computing Machinery: New York, NY, USA, 2011; pp. 261 to 262 or objects (cubes, cones) as stated by the authors McGookin, D.; Robertson, E.; Brewster, S. Clutching at straws: Using tangible interaction to provide non-visual access to graphs. In Proceedings of the Conference on Human Factors in Computing Systems, Atlanta, GA, USA, 10 to 15 April 2010; volume 3, pp. 1715 to 1724. In both cases, the objects have reference markers that are detected by an infrared camera or a web camera, as well as by a software (SW) application implemented on the basis of an "opensource" algorithm - "reacTIVision" or and "ARToolkit", which information they process. Audible and haptic feedback is then used for the user.

Another interesting solution is the implementation for stimulating the learning process using physical objects (cards) with RFID tags for children with ADHD. In this case, RFID tags are read using an RFID reader in a mobile device. The information is processed and the child is provided with feedback in the form of sound, praise, and motivational text. This is a PC and projector solution. As described in the article by De Ua Guía, E.; Uozano, M.D.; Penichet, V.M.R. Educational games based on distributed and tangible user interfaces to stimulate cognitive abilities in children with ADHD. Bro. J. Educ. Technol. 2015, 46, 664 to 678.

In the article Jadan-Guerrero, J.; Jaen, J.; Carpio, M.A.; Guerrero, U.A. Kiteracy: A kit of tangible objects to strengthen literacy skills in children with Down syndrome. In IDC 2015, Proceedings of the The 14th International Conference on Interaction Design and Children; Association for Computing Machinery: New York, NY, USA, 2015; pp. 315 to 318, in which it is described that children with Down syndrome use plastic cards with pictures (of animals, fruits, houses, landscapes), plastic letters with RFID markers for the development of literary reading skills. Markers are scanned by an RFID reader connected to a PC or tablet, the information is also processed by this device and audio feedback is provided to the user. In another article by Haro, B.P.M.; Santana, P.C.; Magana, M.A. Developing reading skills in children with Down syndrome through tangible interfaces. In Proceedings of the ACM International Conference Proceeding Series, Singapore, 7 August 2012; pp. 28 to 34 cue cards are also used, but the situation is captured by a web camera. Subsequently, the image is processed and evaluated by the image recognition algorithm. This system consists of a table, a projector, a web camera on the shoulder and a multi-touch screen.

Another example is the use of a tangible environment for the detection of motor disorders and delays in motor development in children as described in the article by Martin-Ruiz, M.U. Foundations of a Smart Toy Development for the Early Detection of Motoric Impairments at Childhood. International J. Pediatr. Res. 2015, l, lie5. "Smart" cubes with a buzzer and UED diodes for feedback are used to detect motor disorders. Each cube contains 2 UDR (Eight Dependent Resistor) on the wall

-1 CZ 37254 UI sensors. Inside each cube there is an accelerometer, a gyroscope, a compass to determine the cube's position. A tilt sensor in each cube allows the cube to sleep when tilted, the cube wakes up, turns on. The cube allows you to record motor data and subsequently analyze this data. Another example described in the article Rivera, D.; Garcia, A.; Alarcos, B.; Velasco, J.R.; Ortega, J.E.; Martínez-Yelmo, I. Smart toys designed for detecting developmental delays. Sensors 2016, 16, 1953, is a monitoring of children's development using cubes that technically consist of a microcontroller, LiPol battery, protection circuits, wireless RF technology, sensor system (accelerometer, LDR), switch, LEDs, buzzer. The cube collects data and controls the tablet, stimulating the child's activity. Motor disorders are monitored through data analysis.

Another possibility, as stated in the article Lee, K.; Jeong, D.; Schindler, R.C.; Hlavaty, L.E.; Gross, S. L; Short, E.J. Interactive block games for assessing children's cognitive skills: Design and preliminary evaluation. Queue. Pediatrician. 2018, 6, 111, is the use of SIG dice for playing so-called TAG games. Each SIG cube has an integrated system of sensors such as: 6 optical sensors, 3-axis accelerometer, ZigBee module for wireless transmission, timer, microprocessor. By playing games with these cubes, the system enables data to be obtained and the cognitive experience of children to be evaluated. Another system to monitor and assess the social and cognitive skills of children aged 3 to 10 years in children with autism, as described in the article Al Mahmud, A.; Soysa, A.I. POMA: A tangible user interface to improve social and cognitive skills of Sri Lankan children with ASD. International J. Hum. Comput. Shame. 2020, 144, 102486. The technical concept of this approach represents: tangible objects (animals, fruits, vegetables, shapes), a tablet with a SW application. Each plastic object has a conductive foam attached to the underside, which has a typical shape specific to each object, the so-called touch pattern. SW presents an application with games (shapes, fruits, vegetables, animals) and 6 levels. The user completes the task, places the object on the touch tablet and uses touch pattern recognition to evaluate the correctness.

For educational and rehabilitation purposes of 3 to 71-year-old children, the so-called Activity board 1.0 was used, as described in the article Di Fuccio, R.; Siano, G.; De Marco, A. The Activity Board 1.0: RFID-NFC WIFI Multitags Desktop Reader for Education and Rehabilitation Applications. In Proceedings of the WorldCIST 2017: Recent Advances in Information Systems and Technologies, Terceira Island, Portugal, 30 March to 2 April 2017; pp. 677 to 689. These are tangible objects with RFID tags (letters, numbers, cubes, dolls, etc.) that are detected and identified by this activity board. The technical concept consists of an RFID reader that reads RFID tags, an RFID antenna, a microcontroller, a Li-Pol battery with USB charging. SW evaluates everything and provides feedback.

The following solutions are known from the patent literature. WO 2015/113441 AI System and method for recognizing objects placed together using sensors, where RFID tags on each object are used to recognize individual objects (cubes, figures, cards) and using RFID technology they are scanned and then SW evaluated in the superior tablet system, mobile, PC. Another technical solution are WO 2018/025067A1 - An educational toy and KR 101713085 Bl, Contents providing system by smart block for object recognition using NFC technology (NFC tags placed on objects, NFC antenna). Or they represent a combination of both NFC and RFID technologies, as stated in EP 3228370 Al Puzzle system interworking with external device. Another principle for object recognition is identification using a camera according to US 7474318 B2 - Interactive system and method. The camera scans the space where the cubes are manipulated, where each cube has a printed image. Using the processing, it images according to the coordinates of the markers, the distance is recalculated.

The device described in US 9108114 WL-Tangible User Interface and System Thereof then represents cubes where each wall contains 5 inductive proximity sensors (1 transmitter and 4 sensors). The TX (transmitter) is inside the cube and transmits at a distance of 5 mm when the RX receiver picks up the signal. These pairs are opposite each other. Each cube contains a microcontroller with ZigBee wireless communication, memory and a rechargeable Li-Ion battery. The output modes of the cube are 6 RGB LEDs and a mini speaker for reporting feedback, new tasks or melodies. There is a color LCD display on each wall of the cube. Here is a big difference with our technical solution. Regarding the purpose of use

-2 CZ 37254 UI for cognitive rehabilitation phenomenon US 9014614 B2- Cognitive assessment and treatment platform utilizing a distributed tangible - graphical user interface device. The principle of this solution is only the use of already commercially available Sifteo cubes, for which cognitive rehabilitation games were designed. Sifteo cubes consist of accelerometer, battery, button, CPU, Bluetooth, UCD screens.

The gaming station and its technical concept is described in WO 2012/145542 Al -Manipulable cubes base station patent. The cubes contain a battery, memory, controller, displays, and using Wi-Fi and Bluetooth, information about the movement, proximity, arrangement and orientation of nearby cubes is transmitted using an accelerometer. The station includes proximity sensors for detection and integration directly with the cubes. Objects with RFID tags (letters, numbers, cubes, dolls, etc.) that are detected and identified by this activity board. The technical concept consists of an RFID reader that reads RFID tags, an RFID antenna, a microcontroller, a Ui-Pol battery with USB charging. SW evaluates everything and provides feedback.

The essence of the technical solution

The device for the development of cognitive functions consists of the following parts: a docking station and playing blocks or is supplemented with a playing mat. Both the playing blocks and the playing mat are equipped with optoelectronic elements for optical communication between the playing blocks and/or between the playing mat and the playing blocks.

For the purposes of this application, a cube is a spatial geometric figure - e.g. cuboid, cube, pyramid, sphere, etc. The cube has a shell or walls. Each face of the dice or its shell is equipped or equipped with magnets. One of the walls of the dice or its shell provides access to the internal components of the dice. For the purposes of this application, electronic elements are, for example: electronic elements for backlighting, electronic elements for optical communication, electronic elements for wireless communication and control, as well as electronic elements for controlling displays. Physically, these electronic elements can be embodied, for example, as Infra Red transmitters (IR transmitter), Infra Red receivers (IR receiver), digital controller (FPGA - Field Programmable Gate Array), colored UED diodes (RGB EED - Red Green Ble Light Emitted Diode), display, vibration motor, speaker and microcontroller with wireless communication module. Dice displays allow you to display colors, but also any graphic elements, such as characters, symbols, shapes, images, etc. Dice displays also allow you to display the charge status of the respective dice.

The wall or shell of the dice consists of: a frame with handles for placing electronic elements, an opening for placing the display, the display itself and possibly also for placing the protective glass. Electronic elements can be attached to a printed circuit board (PCB board). In addition to the electronic elements, inside each game cube there is also a rechargeable battery, a speaker and a vibration motor for somatosensory activation of the user. Part of at least one wall or shell of the playing cube are conductive contacts that are used for powering the docking station. Variant charging solution, enables charging of the game cube with the wireless charging system from the docking station.

In the case of the variant with a playing mat, the mat has the following parts - the upper part consisting of an illuminated grid with individual playing fields, a speaker of the mat, a cover glass of the playing mat and a lower part where the electronics of the playing mat are located, while the upper and lower parts are connected together in a detachable manner. In the case of this variant, the docking station is part of the playing mat area. The playing field has the shape of the abutting surface of the playing cube. Each playing field consists of: a frame for holding electronic elements, a display and electronic elements. Each playing field is equipped with magnets.

Magnets in the walls of the dice and at the same time in the playing fields of the mat ensure the exact alignment of a dice on another dice and/or on the playing field. So it is possible to stack the cubes

-3 CZ 37254 UI not only on the surface of the playing mat, but also outside it - on top of each other, i.e. in height, but also in space. It is therefore possible to build a building in the shape of the letter T from the playing cubes. In this case, some playing cubes do not touch the playing mat, but are connected to each other and/or to the playing mat, via magnets and optoelectronic communication, via other cubes.

A docking station has as many slots as there are cubes in the device. In the area of the docking station, conductive contacts are placed from above. The docking station can be equipped with indicator lights to signal the charging of the dice and/or the individual playing dice have these indicators. Under the space of the docking station there are magnets for holding the dice. There is also a connector for powering the docking station and, in the case of the variant with a pad, also the pad. Furthermore, connectors for communication and service/diagnostics are located under the docking station area, or a battery for powering the docking station battery can also be located in this area. Furthermore, additional signaling for the full charge of the playing mat can be placed here, namely visual and/or acoustic.

All the magnets in the device, i.e. in the dice, the game mat and the docking station, are strong enough to keep the dice in place or on top of each other even if the user pushes them or the game mat is tilted. The magnets thus prevent the cubes from moving freely, such as sliding.

In the case of the variant with a playing mat, the device is activated and the playing mat sends messages via optical communication to each playing field of the mat. Each playing field of the mat activated in this way has its own unmistakable identification data and transmits a signal to determine whether there is a playing cube on it and/or in its vicinity. If so, then this "addressed" cube responds to the game board with a message containing its basic identification data (e.g., cube number) and the identification data of the corresponding cube wall that is adjacent to the "addressing" playing field pads. This information therefore defines to the playing mat the location of the playing cube on the playing mat and also which specific wall the playing cube is turned to the mat and/or to another playing cube, in both vertical and horizontal directions.

The information obtained in this way, about the location of the cube/cubes on the playing mat, including the touching walls with the playing mat or other cubes, is processed by the digital controller into the mat occupancy data structure. This data structure of the occupancy of the playing pad can be transferred to a superior control system located in an external device, such as a computer, mobile phone, tablet, etc. In a variant solution, the superior control system is located in some peripheral device such as a playing pad or docking station. The superior control system evaluates the distribution of the dice on the playing mat, based on the initial input of the game. In the event of a change to the entry, it sends instructions for changing the visualization elements of the walls of the individual dice to the playing mat. The game pad then passes these instructions to the relevant cube(s) to make this change to the visualization elements of the respective wall(s). The superior system also makes it possible to send instructions to the playing mat, which can also change the visualization elements, either for one playing field or for several playing fields at the same time. Thus, it is possible to create all the same, for example, colored blocks of more than one playing field on the playing mat.

In the case of the variant without a playing mat, the function of detecting the location of individual cubes is taken over by the first activated playing cube, the distribution of the playing cubes in space is then determined in the same way as in the case of the variant with a playing mat, with the only difference that instead of activating the playing fields of the mat, they are activated walls of individual dice. Information about the placement of individual dice on top of each other and/or next to each other and their rotation is further processed by the digital controller into a data structure - placement of individual dice on and/or next to each other. This dice layout data structure can be passed to a superior control system that evaluates the dice layout based on the default game input. If the task is fulfilled, it can send instructions for changing the visualization elements of the walls of individual dice to the first activated dice or to several dice directly. In the case of the variant with the first activated dice, these instructions are passed on to the remaining dice, which will carry out this change in the visualization elements of the respective wall or walls.

-4 CZ 37254 UI

For the successful development of the user's cognitive functions, it is important that all data from the game transferred to the superior control system are stored in memory and can be further analyzed and thus indicate the development trends of the user. In this way, information can be stored in the memory, for example, about the user's accuracy, speed, duration of the "game" or the trajectory with which the user placed the cubes on the playing surface, or whether he assembled, for example, the corresponding specified shape from the game.

Clarification of drawings

All views below refer to example embodiment 1

Giant. 1 - Playing cube - example of structural arrangement

Giant. 2 - Detailed view of the dice wall

Giant. 3 - Top view of the device - docking station and play mat

Giant. 4 - Axonometric view of a variant solution with a playing mat, docking station and cubes Fig. 5 - Diagram of the internal electronics of the dice

Examples of implementing a technical solution

Example 1

In this case, the playing set consists of 6 dice 10 with six faces. The playing mat 8 consists of 24 playing fields 11. The docking station 7 consists of 6 places. The entire playing set is controlled wirelessly via a tablet. Consisting of mains power. The playing cube 10 is square in this case.

Example 2

Example 2 differs from example 1 in that the playing set consists of: two playing cubes 10. playing mat 8 with four playing fields 11 and docking station 7 with two places. The whole thing is powered by batteries. The assembly is controlled wirelessly, via a mobile phone.

Example 3

Example 3 differs from the previous examples in that the playing set consists of: eight playing cubes 10. playing mat 8 has 24 playing fields 11. docking station 7 has eight places. The entire assembly can then be controlled via a "smart watch". The whole assembly can be powered from the mains or from a backup battery.

Example 4

Example 4 represents a playing set that has 8 playing cubes 10 and 8 places in the docking station 7 and at the same time has twelve playing fields 11. The power supply is handled in the same way as in example 3. The playing set is controlled via a laptop.

Example 5

Example 5 represents a game setup where the game pad 8 has 32 playing fields 11. six playing dice 10. 6 charging points in the docking station 7. The power supply is handled in the same way as in example 3. Control is performed via a PC.

-5 CZ 37254 UI

Example 6

Example 6 differs from the previous examples in that the playing cube 10 is in this case a prism with a triangular base.

Example 7

Example 7 differs from the previous variants in that the superior system, which manages information about the occupancy of the playing pad 8, user information and information about entering the game, is located in the memory of the playing pad 8.

Example 8

Example 8 differs from the previous examples in that the entire game, including control peripherals, is placed in a case.

Example 9

Example 9 differs from the previous examples in that the game pad 8 is not part of the assembly and its function, i.e. determining the location of the individual game blocks 10, is taken over by the first activated game block 10.

Example 10

Example 9 differs from the previous examples in that the device does not include a playing mat 8. The playing cubes 10 are not all of the same shape. In this case, it is a "building block" containing basic simple geometric shapes such as cubes, cuboids, cylinders, pyramids, cones, spheres, hemispheres, but also more complex multi-walls, whose abutting surfaces fit into each other.

Industrial applicability

The device is primarily intended for medical facilities, homes for the elderly, educational institutions, but it is also possible to use it in households for rehabilitation and/or development of cognitive abilities of people of different ages. The device can also be used in the entertainment industry for playing games.

Claims (13)

1. A device for the development of cognitive functions, characterized by the fact that it consists of: a docking station (7) and playing blocks (10), whereby the playing blocks (10) are equipped with electronic elements for optoelectronic communication with each other and/or between each other and the docking station (7) and is optionally made up of a play mat (8). 2. Device for the development of cognitive functions according to claim 1, characterized in that the playing cube (10) represents a 3D geometric body, has walls or a shell equipped with magnets (3), and one of the walls is intended for access to the internal equipment of the playing cube ( 10), in the case of a shell, access is provided through an opening in the shell or the shell of the dice (10) is removable. 3. Device for the development of cognitive functions according to claim 2, characterized in that each game cube (10) contains electronic elements for backlighting, for optical communication, for wireless communication and control and for controlling displays (4E). 4. Device for the development of cognitive functions according to claims 2 and 3, characterized in that each wall of the playing cube (10) consists of a frame (2) with handles for placing electronic elements, as well as an opening for placing a display (4E), a display (4E ) and/or from the protective glass (1), while the walls of the dice (10) are adjacent to each other. 5. Device for the development of cognitive functions according to claims 2 to 4, characterized in that the internal equipment of each playing cube (10) includes a rechargeable battery (5), a speaker (4G) and a vibration motor (4F). 6. Device for the development of cognitive functions according to claims 2 and 5, characterized in that at least one of the walls of the playing cube (10) is equipped with conductive contacts for power supply from the docking station (7) or a system for wireless power supply from the docking station (7) . 7. Device for the development of cognitive functions according to claim 1, characterized in that the device also includes a play mat (8), which is equipped with the same optoelectronic elements for communication as the docking station (7) and the game cube (10). 8. Device for the development of cognitive functions according to claims 1 and 7, characterized in that the docking station (7) is independent or is located at a defined place of the play mat (8), while the play mat (8) has the following parts - the upper part consisting of an illuminated grid (6) with individual playing fields (11), a speaker and a cover glass (9) and also has a lower part where the electronics of the playing pad (8) are located, while the upper and lower parts are connected together in a detachable manner. 9. Device for the development of cognitive functions according to claims 1, 7 and 8, characterized in that the shape of the playing field (11) and the place in the docking station (7) corresponds to the shape of the bearing surface of the playing cube (10) and is equipped with the same electronic elements as the wall/shell of the playing cube (10), with magnets (3) placed on the sides or perimeter of the playing field (11). 10. Device for the development of cognitive functions according to claims 2 and 9, characterized in that the magnets (3) located in the walls/shell of the playing blocks (10) and in the playing fields (11) of the playing mat (8) ensure the precise alignment of the playing blocks ( 10) in the playing field (11) and/or on another playing cube (10) both in the vertical and horizontal direction and also to prevent the falling of the playing cubes (10). 11. Device for the development of cognitive functions according to claims 1, 8, characterized in that the number of places in the docking station (7) corresponds to the number of playing dice (10). 12. Device for the development of cognitive functions according to claims 1, 8 and 10, characterized by the fact that in the space of the docking station (7) conductive contacts or a wireless charging system and charge signaling lights are located on top, and magnets (3) are located on the bottom for attachment dice (10). -7 CZ 37254 UI 13. Device for the development of cognitive functions according to claims 1, 8 to 12, characterized in that the docking station (7) has a connector for power supply and connectors for communication and service/diagnostics and a battery of the docking station (7) from below, as well as signaling for a full charge, both visual and acoustic.