WO2025076629A1

WO2025076629A1 - System and method for detecting changes in cognitive function over time

Info

Publication number: WO2025076629A1
Application number: PCT/CA2024/051346
Authority: WO
Inventors: Julius FROHLICH; Jared BOASEN; Mari SILD; Jeremy FROHLICH
Original assignee: Language Research Development Group LRDG inc
Current assignee: Language Research Development Group LRDG inc
Priority date: 2023-10-12
Filing date: 2024-10-11
Publication date: 2025-04-17
Anticipated expiration: 2026-04-12

Abstract

A system and method for detecting changes in cognitive function over time, the method comprising displaying a digital trail making test (TMT) canvas on a user device comprising a touch-enabled display, the digital TMT canvas comprising a plurality of sequential targets; receiving user input via the touch-enabled display, the user input corresponding to a user completing the digital TMT by drawing a path on the touch-enabled display to connect the plurality of sequential targets; determining a plurality of metrics characterizing the user's completion of the digital TMT; comparing the plurality of metrics with historical metrics, the historical metrics comprising past metrics characterizing previously completed digital TMTs; and detecting a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined or statistically significant amount.

Description

SYSTEM AND METHOD FOR DETECTING CHANGES IN COGNITIVE FUNCTION OVER TIME

CROSS REFERENCE

This application claims the benefit of, and priority to, U.S. provisional patent application 63/589,736 filed October 12, 2023, and entitled “SYSTEM AND METHOD FOR DETECTING CHANGES IN COGNITIVE FUNCTION OVER TIME”, the content of which is hereby incorporated by reference in its entirety.

TECHNICAL FIELD

The technical field generally relates to assessing cognitive function. More particularly, the disclosure relates to system and method for detecting changes in cognitive function over time by analyzing and tracking results of an iterative self-administered Trail Making Test.

BACKGROUND

Cognitive assessments, serving as a bedrock of global clinical care, play a pivotal role in exploring the impact of cognitive impairments on everyday tasks across diverse age groups. In light of this, there has been an increasing effort towards the digitization of these essential tools, aimed at overcoming the inherent drawbacks associated with traditional paper-based models. A prime illustration of this digital transformation is embodied in the adaptation of the Trail Making Test (TMT).

The TMT was originally developed in 1944 for the US Army to screen for traumatic brain injury. Using a pen and paper, it involves searching for 25 visual targets, and then connecting them together in sequence by drawing a line between them. Over the past 60 years the TMT has amassed extensive scientific research support for its ability to distinguish between healthy populations and those with cognitive functional impairment, leading it to become an essential clinical screening tool. However, it has suffered from numerous practical limitations such as administrator-dependent measurement accuracy and the need to have a professional administrator present. Moreover, many aspects of cognitive function expressed through the act of drawing on the page were impossible to capture and analyse. Furthermore, with only two versions of the test in existence, an individual could not take the test iteratively without their performance artificially improving due to practice effects.

Digital versions of the TMT have been developed which partially address these limitations, thereby permitting better measurement accuracy than the original and possibility of remote administration. However, no digital implementations of the TMT currently available either commercially or for research purposes comprehensively address the entire suite of the original TMT’s shortcomings. Most importantly, none of them allow individual test takers or group administrators to longitudinally track cognitive function securely, remotely, and iteratively in individuals or groups of users over time, let alone detect acute or long-term changes in cognitive function in these same individuals or groups.

SUMMARY

According to one aspect, there is provided a computer-implemented method for detecting changes in cognitive function over time. The method comprises displaying a digital trail making test (TMT) canvas on a user device comprising a touch-enabled display, the digital TMT canvas comprising a plurality of sequential targets; receiving user input via the touch- enabled display, the user input corresponding to a user completing the digital TMT by drawing a path on the touch-enabled display to connect the plurality of sequential targets; calculating a plurality of metrics characterizing the user’s completion of the digital TMT; comparing the plurality of metrics with historical metrics, the historical metrics comprising past metrics characterizing previously completed digital TMTs; and detecting a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined amount.

In at least one embodiment, the method comprises calculating moving averages and standard deviations of the historical metrics, and detecting the possible change in cognitive function when at least one of the plurality of metrics differs from its moving average by at least two standard deviations or by a statistically significant amount.

In at least one embodiment, the method comprises displaying the historical metrics on a graphical user interface (GUI), and highlighting metrics that differ from their moving averages by the at least two standard deviations or by a statistically significant amount.

In at least one embodiment, the method comprises displaying an alert on the touch- enabled display of the user device when the possible change in cognitive function is detected.

In at least one embodiment, the historical metrics comprise past metrics characterizing digital TMTs previously completed by the user.

In at least one embodiment, the historical metrics comprise past metrics characterizing digital TMTs previously completed by other users or a group of users.

In at least one embodiment, the method comprises comparing the plurality of metrics with historical metrics from a predetermined number of previously completed TMTs.

In at least one embodiment, the method comprises comparing the plurality of metrics with historical metrics from TMTs completed during a predetermined previous period.

In at least one embodiment, the plurality of metrics is associated with a first historical period comprising a plurality of first past metrics, the first historical period being distinct from a second historical period comprising a plurality of second past metrics; further wherein comparing the plurality of metrics comprises comparing the plurality of first past metrics with the plurality of second past metrics.

In at least one embodiment, the method comprises calculating averages and standard deviations of the first past metrics, calculating averages and standard deviations of the second past metrics, and detecting the possible change in cognitive function when an average or standard deviation of a metric during the first historical period differs from an average or standard deviation of the metrics during the second historical period by a predetermined threshold, including a statistical significance threshold.

In at least one embodiment, the digital TMT canvas comprises the plurality of sequential targets arranged according to a layout, further wherein displaying the digital TMT canvas comprises displaying the plurality of sequential targets according to a layout that differs from layouts of the previous TMTs completed by the user.

In at least one embodiment, the method comprises pre-generating a plurality of layouts, wherein displaying the digital TMT canvas comprises selecting one of the pre-generated layouts, and displaying the plurality of sequential targets arranged according to the selected pre-generated layout.

In at least one embodiment, the method comprises pre-generating the plurality of layouts on a backend server; and transmitting the plurality of pre-generated layouts from the backend server to the user device.

In at least one embodiment, the plurality of pre-generated layouts are transmitted from the backend server to a plurality of user devices for completion by a plurality of users, the method further comprising comparing the plurality of metrics characterizing the user’s completion of the digital TMT with sequential targets displayed according to one of the plurality of pre-generated layouts with historical metrics comprising past metrics characterizing other users’ completion of digital TMTs with sequential targets displayed according to the same one of the plurality of pre-generated layouts.

In at least one embodiment, pre-generating the plurality of layouts comprises associating a unique serial number with each layout; and wherein displaying the digital TMT canvas comprises selecting one of the pre-generated layouts having a serial number that differs from serials numbers of previous TMTs completed by the user, and/or selecting a specific pre-generated layout across a plurality of user devices to permit metrics comparison across a plurality of users based solely on the specific pre-generated layout.

In at least one embodiment, the plurality of metrics are recorded from coordinate data and temporal data indicative of where and when the user touched the touch-enabled display to draw the path to connect the plurality of sequential targets.

In at least one embodiment, the plurality of metrics comprises at least one of: a time spent to draw each correct segments, excluding a time spent in the sequential targets, and excluding a time from test start until the first sequential target is touched; a time spent to draw correct and incorrect segments, excluding the time spent in the sequential targets, and excluding the time from test start until the first sequential target is touched; a mean across all correct segments of the time spent to draw each correct segment, excluding a time spent in the sequential targets, and excluding a time from test start until the first sequential target is touched; a mean across all correct and incorrect segments of the time spent to draw correct and incorrect segments, excluding the time spent in the sequential targets, and excluding the time from test start until the first sequential target is touched; a mean across all correct segments of the difference between the path drawn and an ideal segment, normalized by the length of the ideal segment; a mean across all correct and incorrect segments of the difference between the path drawn and the ideal segment, normalized by the length of the ideal segment; a mean of distance drawn by the user over time to complete all the correct segments, normalized by the sum total distance of all ideal segment paths; a mean of distance drawn by the user over time to complete all the correct and incorrect segments, normalized by the sum total distance of all ideal segment paths; a standard deviation of the mean of distance drawn by the user over time to complete all the correct segments, normalized by the length of the ideal segment; a standard deviation of the mean of distance drawn by the user over time to complete all the correct and incorrect segments, normalized by the length of the ideal segment; a mean of ideal segments distance over time to complete all correct segments; a mean of ideal segments distance over time to complete all correct and incorrect segments; a mean of a standard deviation in an instantaneous velocity calculated for every coordinate data pair across all correct segments; a mean of a standard deviation in an instantaneous velocity calculated for every coordinate data pair across all correct and incorrect segments; a time from test start until the first sequential target is touched by the user; a sum of the time spent by the user in the sequential targets over all consecutive sequential targets; a sum of the time spent by the user in the sequential targets over all consecutive and non-consecutive sequential targets; and a time spent by the user to complete the TMT from test onset until a last sequential target is correctly reached; and wherein the correct segment corresponds to the path drawn by the user between two consecutive sequential targets, the incorrect segment corresponds to the path drawn by the user between two non-consecutive sequential targets, and the ideal segment corresponds to a straight line connecting a center of two consecutive sequential targets. In at least one embodiment, the plurality of metrics further comprises at least one of: a time average score; and a speed average score.

In at least one embodiment, detecting the possible change in cognitive function comprises determining whether at least some of the plurality of metrics deviate from expected metrics using a machine learning model.

According to another aspect, there is provided a system for detecting changes in cognitive function over time. The system comprises a user device, a backend server and a user interface. The user device comprises: at least one processor; a communications module; a touch-enabled display; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the user device to: display a digital TMT canvas on the touch-enabled display, the digital TMT canvas comprising a plurality of sequential targets; receive user input via the touch-enabled display, the user input corresponding to a user completing the digital TMT by drawing a path on the touch- enabled display to connect the plurality of sequential targets; record a plurality of metrics characterizing the user’s completion of the digital TMT ; and transmit the plurality of metrics via the communications module. The backend server comprises: at least one processor; a communications module; persistent storage storing historical metrics comprising past metrics characterizing previously completed digital TMTs; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the backend server to: receive the plurality of metric from the user device via the communications module; compare the plurality of metrics with the historical metrics, and generate an indication of a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined or statistically significant amount; and store the plurality of metrics in the persistent storage to update the historical metrics. The user interface obtains results associated with a given TMT for a given user, wherein the results are transmitted to the backend server.

The backend server further comprises a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising: retrieving test results associated with the given user; generating metrics; and transmitting to the user interface the metrics of the given user.

According to another aspect, there is provided a non-transitory computer-readable storage medium storing instructions that, when executed by a computing system having one or more processors, cause the computing system to perform a method for detecting changes in cognitive function over time, comprising: displaying a digital trail making test (TMT) canvas on a user device comprising a touch-enabled display, the digital TMT canvas comprising a plurality of sequential targets; receiving user input via the touch-enabled display, the user input corresponding to a user completing the digital TMT by drawing a path on the touch-enabled display to connect the plurality of sequential targets; calculating a plurality of metrics characterizing the user’s completion of the digital TMT ; comparing the plurality of metrics with historical metrics, the historical metrics comprising past metrics characterizing previously completed digital TMTs; and detecting a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined amount.

According to another aspect, there is provided a computer-implemented method for detecting changes in cognitive function over time. The method comprises: displaying a neuropsychological test canvas on a user device comprising a touch-enabled display, the neuropsychological test canvas comprising a plurality of targets; receiving user input via the touch-enabled display, the user input corresponding to a user completing the digital neuropsychological test by interacting with targets on the touch-enabled display in compliance with neuropsychological test instructions; calculating a plurality of metrics characterizing the user’s completion of the neuropsychological test; comparing the plurality of metrics with historical metrics, the historical metrics comprising past metrics characterizing previously completed neuropsychological tests; and detecting a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined amount.

According to yest another aspect, there is provided a system for detecting changes in cognitive function over time. The system comprises a user device, a backend server and a user interface.

The user device comprises: at least one processor; a communications module; a touch- enabled display; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the user device to: display a neuropsychological test canvas on the touch-enabled display, the neuropsychological test canvas comprising a plurality of targets; receive user input via the touch-enabled display, the user input corresponding to a user completing the neuropsychological test by interacting with targets on the touch-enabled display in compliance with neuropsychological test instructions; record a plurality of metrics characterizing the user’s completion of the neuropsychological test; and transmit the plurality of metrics via the communications module.

The backend server comprises: at least one processor; a communications module; persistent storage storing historical metrics comprising past metrics characterizing previously completed neuropsychological tests; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the backend server to: receive the plurality of metric from the user device via the communications module; compare the plurality of metrics with the historical metrics, and generate an indication of a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined or statistically significant amount; and store the plurality of metrics in the persistent storage to update the historical metrics.

The user interface obtains results associated with a given neuropsychological test for a given user, wherein the results are transmitted to the backend server.

The backend server further comprises a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising: retrieving test results associated with the given user; generating metrics; and transmitting to the user interface the metrics of the given user. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1 is an example of a Trail Making Test (TMT) canvas of type A, in accordance with a first layout;

Figure 2 is an example of a TMT canvas of type B, in accordance with a second layout;

Figure 3 is a block diagram of a system for detecting changes in cognitive function over time, in accordance with an embodiment;

Figure 4 is a schematic of a system for detecting changes in cognitive function over time in a multi-user environment, in accordance with an embodiment;

Figure 5 is a flow chart of a method for detecting changes in cognitive function over time, in accordance with an embodiment;

Figure 6 is an exemplary table of records of completed TMT in a multi-user environment, in accordance with an embodiment;

Figure 7 is the example TMT canvas of Figure 1 , showing an ideal path, in accordance with an embodiment; and

Figures 8A and 8B are views of a graphical user interface associated with an application for detecting changes in cognitive function over time, in accordance with an embodiment.

DETAILED DESCRIPTION

It will be appreciated that, for simplicity and clarity of illustration, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements or steps. In addition, numerous specific details are set forth in order to provide a thorough understanding of the exemplary embodiments described herein. However, it will be understood by those of ordinary skill in the art that the embodiments described herein may be practised without these specific details. In other instances, well-known methods, procedures and components have not been described in detail so as not to obscure the embodiments described herein. Furthermore, this description is not to be considered as limiting the scope of the embodiments described herein in any way but rather as merely describing the implementation of the various embodiments described herein.

The system and method described herein may be implemented in computer program(s) executed on processing device(s), each comprising at least one processor, a data storage system (including volatile and/or non-volatile memory and/or storage elements), and at least one input and/or output device. “Processing devices” encompass computers, servers and/or specialized electronic devices which receive, process and/or transmit data. As an example, “processing devices” can include processing means, such as microcontrollers, microprocessors, and/or CPUs, or be implemented on FPGAs. For example, and without limitation, a processing device may be a programmable logic unit, a mainframe computer, a server, a personal computer, a cloud-based program or system, a laptop, a personal data assistant, a cellular telephone, a smartphone, a wearable device, a tablet, a video game console or a portable video game device.

Furthermore, the system and method of the described embodiments are capable of being distributed in a computer program product comprising a computer readable medium that bears computer-usable instructions for one or more processors. The computer-usable instructions may also be in various forms including compiled and non-compiled code.

The processor(s) are used in combination with a storage medium, also referred to as “memory” or “storage means”. Storage medium encompasses volatile or non- volatile/persistent memory, such as registers, cache, RAM, flash memory, ROM, diskettes, compact disks, tapes, chips, as examples only. The type of memory is, of course, chosen according to the desired use, whether it should retain instructions, or temporarily store, retain or update data. Steps of the proposed methods are implemented as software instructions, stored in computer memory and executed by processors.

In the present description, the term "user" refers to an individual (i.e., a patient) whose cognitive function is to be assessed and to whom the TMT is iteratively administered.

The term “administrator” refers to an individual having access to results of TMTs administered to one or more users. For example, the administrator can be medical staff, such as researchers or clinicians, that have been authorized by the one or more user to have access to their data. In embodiments described herein, the TMT is self-administered in that users can carry out the TMT without requiring the presence or explicit direction from an administrator.

As will be described in further detail below, the proposed system and method, according to one aspect of the present description, aim to detect changes in a user’s cognitive function based on results of successive digital TMTs self-administered by the user over time. The proposed system and method are adapted to record a plurality of results of successive digital TMTs and to provide indication of change in cognitive functions by comparing the results of a specific individual or a set of multiple digital TMTs with historical results recorded in the past.

In an embodiment, the system and method permit self-administration of infinite variants of the TMT to registered app users. A diverse array of metrics is recorded for each completed test for each user and stored in a secure server. A moving average and standard deviation are then calculated based on the last X tests which is designated as the user’s “normal” range of cognitive performance. Acute test score deviants can be identified according to whether they fall outside two standard deviations of the user’s “normal” range. Occurrence of such deviants will trigger a notification system to alert the user that they have scored outside their normal range, affording them the opportunity to initiate an intervention against the detected change in their cognitive state, as will be further detailed below. Moreover, weekly, month to month, quarterly, biannual, and annual shifts in the moving average and standard deviation will also be statistically compared using the first and last X, Y, and Z test scores of those respective periods. Significant shifts (p < 0.05) in test scores will also trigger the aforementioned notification system in the same manner.

With reference now to Figures 1 and 2, a digital implementation of the TMT is shown according to an embodiment. The TMT generally involves presenting a canvas to a user on a touch-enabled display. The canvas includes a plurality of randomly-arranged and sequentially-labelled targets, and the user is directed to draw a continuous path (i.e. without lifting the tactile pen or finger) to connect the targets in order as quickly as possible. In some implementations, the user must draw a path that does not intersect itself. A total of 25 numbered targets are presented to the user in accordance with the number of targets from the original paper and pencil version of the test. However, the number of targets can be varied, either larger or smaller, in different implementations.

The TMT can generally include two parts: part A (TMT-A) and part B (TMT-B). In TMT-A, the targets are labelled using numbers only (e.g., from 1 to 25), and the user is directed to connect the targets in numerical order (e.g., in ascending order). In TMT-B, the targets are labelled using numbers and letters or phonetic symbols (e.g., from 1 to 13 and from A to L in English, or from 1 to 13 and from to L in Japanese), and the user is directed to connect the targets while alternating between numbers and letters in consecutive order (e.g., 1-A-2-B-3-C, etc.).

In another embodiment of TMT-B (not shown), the targets can comprise numbers (e.g., from 1 to 15), each of them being displayed twice, one in black with a white background and one in white with a black background. The user is directed to connect the targets in numerical order (e.g., in ascending order) but by alternating between white background and black background.

Figure 1 shows an example of a TMT-A canvas [60], where a plurality of sequential targets [62] are represented as circled numbers ranging from 1 to 25. The plurality of sequential targets [62] are randomly arranged and displayed on the TMT-A canvas [60] according to a layout [66], The user is directed to draw a continuous path [64] (e.g., using a tactile pen or a finger) to connect each target [62] in increasing numerical order, starting from number 1 , until number 25 is reached. In the illustrated embodiment, the path [64] drawn by the user is displayed on the canvas [60], and targets [62a] connected by the path [64] are visually distinguished from targets [62b] that have not yet been connected. It is appreciated, however, that other configurations are possible.

Figure 2 shows an example of a TMT-B canvas [60’], where a plurality of sequential targets [62’] are represented as circled numbers ranging from 1 to 13 and circled letters ranging from A to L. The plurality of sequential targets [62’] are randomly arranged and displayed according to a layout [66’]. The user is directed to draw a continuous path [64’] to connect each target [62’] in increasing order, alternating between numbers and letters, until number 13 is reached. In other words, the user is directed to draw the path [64’] from number 1 to letter A, then from letter A to number 2, then from number 2 to letter B, then from letter B to number 3, then from number 3 to letter C etc. Again, the path [64’] drawn by the user is displayed on the canvas, and targets [62a’] connected by the path [64’] are visually distinguished from targets [62b’] that have not yet been connected. It is appreciated, however, that other configurations are possible.

While in the embodiment described above the TMT is displayed using Arabic numbers and Latin alphabet, it is understood that TMT can be adapted in any language (as suggested above), such as Greek, Cyrillic, Hebrew, Arabic, Kurdish, Persian, Chinese, Korean, Japanese alphabets, or any other alphabet. Similarly, any other numbers than Arabic ones can be used.

Referring now to Figure 3, there is illustrated an embodiment of a system [10] for detecting changes in cognitive function over time. Broadly described the system [10] comprises a user device [20] and a backend server [40], It is appreciated that other components can be provided in other embodiments, as it will be described in more detail hereinbelow.

The user device [20] refers to any device that is adapted for administering a digital TMT to a user and measuring corresponding metrics. In the present embodiment, the user device [20] is an iPad® configured with a software application for displaying TMT canvases to a user, receiving user input in the form of paths drawn by the user on the digital touchscreen, and calculating metrics characterizing the user’s completion of the TMT. It is appreciated, however, that other computing devices may be possible, such as a personal computer, laptop, tablet, smart phone, or the like.

The user device [20] can include at least one processor [26] and memory [27], with the memory having stored thereon program code comprising machine-readable instructions that when executed by the at least one processor [26], cause the user device [20] to administer the TMT to a user and carry out at least some steps of the method described hereinbelow. For example, the program code can configure at least one processor to calculate a plurality of metrics characterizing a user’s completion of a TMT, as will be described in more detail hereinafter. In some embodiments, the user device [20] can further include persistent storage, for example for storing pre-generated TMT canvases and/or for storing metrics of TMTs previously administered via user device [20],

The user device [20] can furthermore include an output module in operative communication with the at least one processor [26] to present output to the user. The output module can comprise a display [22] operated by the at least one processor [26] and configured to present visual output to the user via a graphical user interface (GUI) [24], More specifically, the output module can be configured to display TMT canvases to the user, along with visual elements associated with administration of the TMT, such as a path drawn by the user, visually distinguishing targets correctly connected by the user, a current test duration, etc. The output module can further be configured to display at least some results associated with a completed TMT, such as the test completion time in seconds, and/or a notification to alert the user if they have scored outside of a normal range. In some embodiments, the output module can further be configured to present elements to assist users in self-administering the TMT, such as controls allowing a user to login and identify themself, controls allowing the user to begin, pause, and/or end tests, and elements instructing the user how to correctly complete the TMT, such as a video tutorial and/or hints while completing the TMT, such as indications when the user has incorrectly connected targets or has drawn a path that intersects with itself. In the present embodiment, the output module comprises a touch-enabled display [22] integrated as part of the iPad® tablet for presenting the visual output to the user. It is appreciated, however, that in other embodiments, other displays can be used as needed. Moreover, it is appreciated that the output module need not be limited to presenting visual output. For example, in some embodiments the output module can be configured to present other output to the user to facilitate administering TMTs and/or providing results therefrom, including but not limited to audio output (e.g., such as sounds and/or voice) and tactical output (e.g., such as haptic feedback).

The user device [20] can further include an input module in operative communication with the at least one processor [26] to receive input from the user. The input module can comprise a touch device operated by the at least one processor [26] and be configured to receive tactical input from the user corresponding to the user’s interaction with the GUI. More specifically, the input module can be configured to receive user input corresponding to paths drawn by the user to connect targets displayed via the GUI. The input module can further be configured to receive user input corresponding to users operating controls displayed by the GUI, such as the controls allowing a user to login and identify the user, and the controls allowing the user to begin and/or end tests. In the present embodiment, the input module also comprises the touch-enabled display [22] integrated as part of the iPad® tablet. In this fashion, users can provide input directly on the same display where the GUI is provided, for example using their finger or using a tactile pen. When drawing paths to connect targets during administration of a TMT, this can more closely mimic the experience of traditional paper-based TMTs. Although a touch-enabled display is used in the present embodiment, it is appreciated that other touch or tactile-based input devices can be used as needed, such as a touchpad or trackpad, a mouse, trackball, joystick, keyboard, etc. Moreover, it is appreciated that in some embodiments the input module can be configured to receive other types of input from users.

In some embodiments, the user device [20] can further include a communication module [28] in operative communication with the at least one processor [26], to allow communication between the user device [20] and one or more other devices, for example to receive pre-generated TMT canvases, to receive historical test results and/or alerts, and/or to transmit raw data and calculated metrics from each completed TMT.

The backend server [40] refers to any processing device capable of generating digital TMT canvases, storing historical metrics associated with completed TMTs, capable of making comparisons in historical metrics to detect changes in cognitive function over time, and/or capable of providing an administrator access to TMT results and corresponding metrics, and/or permitting an administrator to manage users. In the present embodiment, the backend server [40] is a server that is separate from and/or remote relative to the device that is used to administer the TMT to the user (e.g., the iPad® tablet). It is appreciated, that other configurations are possible. For example, in some embodiments the functionality of backend server [40] can be distributed among a plurality of physical or virtual servers and/or other suitable computing devices. In some embodiments, at least some of the functionality of the backend server [40] can be implemented on the same device that is used to administer the TMT to the user.

The backend server [40] can include at least one processor [46] and memory [47], with the memory having stored thereon program code comprising machine-readable instructions that when executed by the at least one processor [46], causes the backend server [40] to carry out at least some steps of the method described hereinbelow. For example, the at least one processor can be configured to generate a plurality of digital TMT canvases, transmit the generated digital TMT canvases to user device [20], receive a plurality of metrics from the user device [20] characterizing a user’s completion of a digital TMT, compare the plurality of metrics with historical metrics to detect possible changes in cognitive function, generate an indication or alert when at least one of the plurality of metrics differs from the historical metrics by a predetermined amount, and/or store the plurality of metrics to update the historical metrics.

The backend server [40] can include a communication module [58] in operative communication with to the at least one processor [46], to allow communication between the backend server [40] and a communication module [28] of one or more other devices, such as the user device [20], for example to transmit pre-generated TMT canvases [60], to receive metrics [80] associated with completed TMTs, and/or to transmit test results and/or alerts [75],

The backend server [40] can further include persistent storage [48] in operative communication with the at least one processor [46], The persistent storage [48] can comprise non-volatile memory, for example in the form of a database, and be configured to allow long term storage of historical metrics [70] comprising past metrics characterizing previously completed digital TMTs. Such historical metrics can, for example, be accessed by the at least one processor [46] to make comparisons and detect possible changes in cognitive function, as will be further detailed hereinbelow.

The backend server [40] can include a plurality of modules for implementing the functions described above. For example, the backend server [40] can include: a layout generator module [50] for generating TMT canvases, a metrics collection module [52] for receiving metrics of completed TMTs, a comparator module [54] for comparing metrics of completed TMTs with historical metrics [70], and an alert generator module [56] for generating alerts when a possible change in cognitive function is detected, as will be further detailed below. As can be appreciated, such modules can be implemented by the at least one processor [46] executing corresponding instructions stored in memory [47],

The communication module [28] of the user device [20] and the communication module [58] of the backend server [40] can communicate with one another over a network [15], It is appreciated that the network [15] can correspond to any connection that allows for data communication between computing devices. For example, the network [15] can comprise a direct data link between the user device [20] and the backend server [40], and/or computing networks through which the user device [20] and the backend server [40] can communicate, such as a personal area network (PAN), local area network (LAN), wide area networks (WAN) (such as the internet), etc.

Referring to Figure 4, a multi-user environment is shown. In this environment, a plurality of users [5, 5’, 5”, 5”’] are each provided with their own user device [20, 20’, 20”, 20”’]. Each user device [20] is connected to the network [15] via the communication module [28] of the user device [20], The backend server [40] is also connected to the network [15] via the communication module [58] of the backend server [40], and can therefore receive information and data from each of the plurality of user devices [20], Each of the user device [20] comprises a TMT application [25], as will be further detailed hereinbelow. The TMT application allows each user to identify themselves by authenticating and logging in with a user-specific profile, and self-administer the TMT. It is appreciated that a user device [20] could also be shared among a plurality of users. In this case, the user device [20] is reconfigured for each user following the authenticating and logging in with their userspecific profile. In the present embodiment, each user device [20, 20’, 20”, 20’”] is the same device, such as an iPad® tablet having the same screen size (for example a 12.9” screen size that is substantially the same size as sheet of A4 paper, and therefore suitable for resembling a paper-based TMT). In this fashion, TMT canvases will be displayed in the exact same way on each device, allowing TMTs to be administered in a uniform manner among different users [5, 5’, 5”, 5’”], and allowing metrics to be compared more directly from one user to another while avoiding variances that may be introduced by hardware differences. It is appreciated, however, that other configurations are possible and that in some embodiments, different users can use different devices, such as Android® tablets and/or tablets having different screen sizes. In these instances, TMT performance deviations due to screen size will be controlled for statistically through tracking of the type of user device and normalization of metrics which are sensitive to screen size, such as distance drawn and test completion time.

In some embodiments, an administrator [7] can have access to the backend server [40] via a web-based portal [8], The administrator [7] can be assigned as a group administrator (e.g., having access only to information and data related to a specific group of users) or can be assigned as a general administrator (e.g., having full access to all information and data of all users and being able to modify the system and/or method as described herein). The administrators [7] can be professionals, such as researchers or clinicians. The webbased portal [8] can provide the ability to visualize the data at various user levels over different time periods and according to different metrics, or download data according to specific queries pertaining to individuals or groups of users who they are administrators for. Thus, any observed changes in cognitive function based on test performance can be linked to contextual factors (e.g., changes in scheduling, implementation of wellness programs, etc.). In some embodiments, the web-based portal [8] allows professional to see personal information related to a selected user and/or other data specific to their professional and/or organizational context. With reference to Figure 5, an exemplary method [100] for detecting changes in cognitive function overtime is shown. As can be appreciated, the method [100] can be implemented using a system such as the system [10] described above.

A first step [110] includes generating a digital TMT layout [66], In some embodiments, this step can be performed on the backend server [40], for example by the layout generator module [50], The layout generator module [50] can, for example, execute a process to pre-generate a plurality of semi-random layouts [66], including type A layouts (for TMT-A) and type B layouts (for TMT-B). Each pre-generated layout [66] comprises a different arrangement of the plurality of sequential targets [62] by respecting the principles of TMT design as detailed for example in the article “Computerizing Trail Making Test for longterm cognitive self-assessment” by Zhiwei Zeng et al., dated March 2017, the entirety of which is incorporated herein by reference. The layouts [66] can be generated in accordance with a known size of a screen of a user device [20] through which the TMT will be administered. For example, layouts [66] can be generated specifically for iPad® tablets having a given layout. In some embodiments, such as in a multi-user environment where users are provided with different types of user devices for administering TMTs, specific layouts [66] can be generated for each different user device.

Each pre-generated layout [66] can be associated with a unique identifier or serial number [68], Having pre-generated layouts [66] identified with a unique serial number [68] can facilitate managing and comparing different tests performed by the user [5] and/or by different users. For example, serial numbers of TMTs completed by a user can be tracked such that a new layout is presented to the user each time the TMT is administered, thereby avoiding repetition of canvases presented to a user, and preventing the test results from being impacted by potential memorization of the layout or practice effects by the user. The plurality of pre-generated layouts [66] generated by the layout generator module [50] can be stored in the persistent storage [48] of the backend server [40],

In some embodiments, the layout generator module [50] can transmit the plurality of pregenerated layouts [66] from the backend server [40] via the backend communication module [58] to the user device [20] via the user device communication module [28], In some embodiments, a specifically selected pre-generated layout can be transmitted to the user device [20] on demand and stored temporarily on the memory [27] of the user device [20] when the TMT is being administered, and deleted thereafter. In some embodiments, the plurality of pre-generated layouts [66] can be transmitted to the user device in batches, and stored in persistent memory of the user device [20], One of the plurality of pregenerated layouts [66] can then be selected and loaded from the persistent storage when the TMT is to be administered, thus allowing TMTs to be administered via user device [20] while in an “offline” mode (i.e. , while not in direct communication with the backend server [40]). In some embodiments, a batch of pre-generated layouts [66] can be transmitted to user device [20] on demand, for example when the user device [20] has exhausted, or is close to exhausting, all available unique layouts to present to the user. In some embodiments, batches of pre-generated layouts [66] can be transmitted to one user device [20], or to a plurality of user devices [20, 20’, 20”, 20”’], simultaneously, for example at regular intervals and to assure that all users can be presented unique TMT layouts from the same pool of layouts.

Subsequent steps of the method [100] can include administering a TMT to user [5] using the pre-generated TMT layouts. As can be appreciated, the user [5] can be identified prior to administering the TMT such that a new or specific TMT layout can be selected for the user and such that the results of the test and corresponding metrics can be associated with the user. The user [5] can thus be invited to authenticate in the system [10], Any suitable authentication means can be used, such as for instance the user entering a username and a password and/or providing biometrics such as fingerprints and/or an image of their face for comparison with corresponding stored data and/or a QR code, etc.

Once the user is authenticated, a second step [120] includes displaying a TMT canvas on the user device [20], Displaying the TMT canvas can include displaying the plurality of sequential targets [62] according to a layout [66] on touch-enabled display [22], In an embodiment, the plurality of sequential targets [62] can be displayed according to a layout [66] that differs from layouts of TMTs previously administered to and/or completed by the user [5], In some embodiments, displaying the TMT canvas on the user device can comprise selecting one of the plurality of pre-generated layouts [66], and displaying the plurality of sequential targets [62] arranged according to the selected pre-generated layout [66], In an embodiment, displaying the digital TMT canvas comprises selecting one of the pre-generated layouts [66] having a serial number [68] that differs from serial numbers of TMTs previously administered to and/or completed by the user [5], In some embodiments, the selected layout can correspond to a pre-generated layout having a serial number that sequentially follows the serial number of the last layout presented to the user. In some embodiments, the layout can be selected at random from a pool of available layouts not yet presented to the user. In some embodiments, a specific pre-generated layout can be selected and presented across a plurality of user devices to permit metrics comparison across the plurality of users based solely on the specific pre-generated layout.

A third step [130] can include receiving user input corresponding to the user performing the TMT. To perform the TMT, the user [5] can be directed to use a tactile pen or their finger to draw a path [64] on the displayed canvas via the touch-enabled display [22] to connect the plurality of sequential targets [62], While the user is drawing the path [64], the user device [20] can record coordinate data and temporal data indicative of where and when the user [5] touches the touch-enabled display [22], The recorded coordinate data and temporal data can be stored in the memory [27] of the user device [20] for subsequent processing.

A fourth step [140] includes calculating a plurality of metrics characterizing the user’s completion of the digital TMT. The plurality of metrics can be calculated using the recorded coordinate and temporal data. In the present embodiment, the plurality of metrics is calculated by at least one processor [26] of the user device [20] following completion of the TMT, but it is appreciated that other configurations are possible. For example, in some configurations, some metrics can be calculated while the user is performing the TMT. In other configurations, the coordinate and temporal data can be transmitted to backend server [40], and the backend server [40] can calculate the metrics. In some embodiments, the metrics can be calculated in relation to an expected or ideal solution. In such embodiments, calculating the metrics can include determining an expected or ideal solution for the TMT canvas presented to the user, and quantifying differences between the expected or ideal solution and the recorded coordinate and temporal data. As can be appreciated, the plurality of calculated metrics can be associated with a specific user [5], and/or with the completion of a specific TMT layout [66] such that they can be distinguished from metrics calculated for different users and/or for the completion of different TMT layouts.

The plurality of metrics can be calculated based on various characteristics or parameters of the TMT, such as spatial and/or temporal characteristics. As an example, some spatial characteristics will be described with reference to Figure 7, where a TMT canvas [60] partially completed by a user is shown. The layout [66] comprises a plurality of targets [62] numbered from 1 to 25. Each target [62] has a corresponding boundary defining an area occupied by the target. In the present embodiment, the boundary of a target is defined by a circle having a predefined diameter, but it is appreciated that other shapes are also possible. The boundary of each target can circumscribe a set of coordinates, and a user can be considered to be within or touching a target when touching an area of the touch- enabled display having a coordinate corresponding to one of the coordinates circumscribed by the boundaries of that target.

When drawing a path [64] to connect sequential target [62] from number 1 to number 25, the path drawn by a user can include a plurality of segments [63], with each segment [63] corresponding to a portion of the path [64] that connects two targets [62], and having a length or a distance. A correct segment [63a] can be defined as a segment of the path that connects two sequential targets in the right order (e.g. a segment that connects target number 1 to target number 2). Targets connection in this manner can be referred to as correct targets. An incorrect segment [63b] can be defined as a segment that connects two non-sequential targets (e.g. a segment that connects target number 2 to target number 4). The target that was connected out of order can be referred to as an incorrect target.

For every TMT layout, an ideal path [65] can defined. The ideal path [65] can comprise a plurality of segments, referred to as ideal segments, with each ideal segment corresponding to a straight line connecting a center of two sequential targets [62], In practice, paths drawn by users will be imperfect and non-linear, and will thus deviate from an ideal path. Imperfection in the path drawn by a user can be quantified in different ways, including by measuring the difference in the distance between the user path and the ideal path for any given segment. Path imperfection could also be quantified based on the number of directional changes in the user path (which is zero for an ideal segment path) or the area between a user path and the ideal path.

The plurality of metrics can also be calculated based on temporal characteristics of the TMT. For example, time spent performing different aspects of the test can be measured. As an example, a time to complete the TMT can be measured by timing the user [5] from the beginning of the test until the user reaches the last sequential target [62], As another example, a time per segment can also be measured. The time per segment can be measured by timing the user when drawing a segment between two sequential target [62], whether it is a correct segment or incorrect segment. As yet a further example, the time spent by a user dwelling or pausing within the boundary of a target can be measured and/or time spent correcting errors can be measured.

A standard metric entitled Total Test Time can be calculated as the time taken to complete the TMT from test onset until the last target was correctly reached. This metric corresponds to the primary performance metric used in traditional paper-based TMTs. However, it will be appreciated that enhanced metrics can be calculated by using the more granular coordinate and temporal data recorded by the user device [20] and based on the spatial and/or temporal characteristics of the digital TMT canvases as described above. At least some enhanced metrics that can be calculated are described below.

1 . Total Correct Segment Time’.

This metric can be calculated as the sum of measured time spent drawing correct segments, while excluding time spent within targets, and excluding time from the test start until the first target is touched by the user (i.e. , the metric Start2Target1 as described below).

2. Total Correct and Incorrect Segment Time'.

This metric can be calculated as the sum of measured time spent drawing segments regardless of whether they were correct or incorrect, while excluding time spent within targets, and excluding time from the test start until the first target is touched by the user (i.e., the metric Start2Target1 as described below).

3. Mean Time Across Correct Segments’.

This metric can be calculated as the mean across all correct segment times.

4. Mean Time Across Correct and Incorrect Segments’.

This metric can be calculated as the mean across all correct and incorrect segment times.

5. Normalized Deviation of Correct Segment Distance’.

This metric can be calculated as the mean across all correct segments of the difference between a length of a correct segment drawn by the user and a length of a corresponding ideal segment, normalized by dividing by the length of the corresponding ideal segment.

6. Normalized Deviation of Correct and Incorrect Segment Distance’.

This metric can be calculated as the mean across all correct and incorrect segments of the difference between a length of a segment drawn by the user and a length of a corresponding ideal segment, normalized by dividing by the length of the corresponding ideal segment. Mean Velocity of Correct Segments:

This metric can be calculated as the mean across all correct segments of the length of the correct segment drawn by the user divided by the time taken to draw the correct segment. Mean Velocity of Correct and Incorrect Segments:

This metric can be calculated as the mean across all correct and incorrect segments of the length of the segment drawn by the user divided by the time taken to draw the segment. Velocity Correct SD:

This metric can be calculated as the standard deviation of the Mean Velocity of Correct Segments. Velocity Correct and Incorrect SD:

This metric can be calculated as the standard deviation of Mean Velocity of Correct and Incorrect Segments. Mean Velocity Correct Destination:

This metric can be calculated as the mean of the lengths of ideal segments divided by the time taken by the user to draw corresponding correct segments. Mean Velocity Correct and Incorrect Destination:

This metric can be calculated as the mean of the lengths of ideal segments divided by the time taken by the user to draw corresponding correct and incorrect segments. Mean SD Instantaneous Velocity of Correct Segments:

This metric can be calculated as the mean of the standard deviation in the instantaneous velocity calculated for every coordinate data point across all correct segments. Mean SD Instantaneous Velocity of Correct and Incorrect Segments:

This metric can be calculated as the mean of the standard deviation in the instantaneous velocity calculated for every coordinate data point across all correct and incorrect segments. Start2Target1:

This metric can be measured as the time from when the test starts until the first target is touched by the user. 16. Total Correct Target Time:

This metric can be measured as the sum of time spent by the user within all correct targets.

17. Total Correct and Incorrect Target Time:

This metric can be measured as the sum of time spent by the user within all correct and incorrect targets.

Although a particular set of enhanced metrics has been described, it will be appreciated that other metrics can be calculated as well.

In some embodiments, the above-described metrics can be normalized to facilitate comparison. For example, a scale can be defined for each metric, and a score having a fixed range (such as a percentage) can be calculated based on where the calculated metric falls within the scale. The scale for each metric can, for example, be defined based on expected “average” or “deficient” values for each metric as confirmed through clinical trial.

In some embodiments, aggregated scores can be calculated based on a combination of a plurality of metrics. In particular, the metrics described above can be categorized according to one or more metric categories, such as time-based metrics, speed-based metrics, accuracy-based metrics, among others.

As an example, time-based metrics can include: Total Test Time, Total Correct Segment Time, Total Correct and Incorrect Segment Time, Mean Time Across Correct Segments, Mean Time Across Correct and Incorrect Segments, Start2Target1, Total Correct Target Time, and Total Correct and Incorrect Target Time speed-based metrics can include: Mean Velocity of Correct Segments, Mean Velocity of Correct and Incorrect Segments, Velocity Correct SD, Velocity Correct and Incorrect SD, Mean Velocity Correct Destination, Mean Velocity Correct and Incorrect Destination, Mean SD Instantaneous Velocity of Correct Segments, and Mean SD instantaneous Velocity of Correct and Incorrect Segments and accuracy-based metrics can include: Normalized Deviation of Correct Segment Distance and Normalized Deviation of Correct and Incorrect Segment Distance. Aggregated scores can be calculated by normalizing and averaging at least some metrics included in the difference categories. For example, in an embodiment, a time average score [82] can be calculated by averaging at least some normalized timebased metrics, and a speed average score [84] can be calculated by averaging at least some normalized speed-based metrics. It is appreciated that other types of aggregated scores can be calculated using different combinations of metrics.

Once the plurality of metrics are calculated, they can be stored in the memory [27] and/or in persistent storage of the user device [20], In some embodiments, at least some of the plurality of metrics can be displayed to the user following completion of a TMT via the GUI [24], In some embodiments, the metrics displayed to the user can be limited to basic metrics, such as the test completion time Total Test Time. In the present embodiment, the plurality of metrics calculated on the user device [20] can be transmitted to the backend server [40] for long-term storage and subsequent analysis. The user device [20] can transmit the metrics via communication module [28] following completion of a TMT. In some configurations, if the user completed a plurality of TMTs while a connection with the backend server [40] is not available (e.g., while in an “offline” mode), the user device [20] can transmit metrics of a plurality of completed tests as a batch once a connection is available. The metrics collection module [52] of the backend server [40] can receive the plurality of metrics [80] via the backend server communication module [58], The metrics collection module [52] can further store the plurality of metrics [80], and/or aggregated scores, such as the time average score [82] and the speed average score [84], in the persistent storage [48] of the backend server [40], to update a historical metrics [70] database.

As shown for example in Figure 6, the historical metrics [70] can comprise past metrics characterizing digital TMTs previously completed by the user [5], and past metrics characterizing digital TMTs previously completed by other users [5, 5’, 5”, 5ⁿ], and aggregate historical metrics of groups of users. The historical metrics [70] can comprise sets of metrics associated with completed TMTs, with each set of metrics being associated with a corresponding user identifier [5, 5’, 5”, 5ⁿ] or group identifier [6], date of completion (DOC) of the test, and the serial number [68] of the TMT canvas [60] administered.

In the example shown in Figure 6, User 1 has completed TMT with serial numbers SNxxxxl , SNxxxx3 and SNxxxx6 at three different dates, while User n has completed TMT with serial numbers SNxxxxl and SNxxxx6 the same day.

In some embodiments, the method can further include calculating statistical parameters characterizing historical metrics [70], such as moving averages and standard deviations of the historical metrics [70], for at least one of the metrics, and/or a combination of metrics. For example, a historical moving average range [72] and an historical standard deviation range [74] can be calculated for each user [5], The moving averages and standard deviations can be calculated using at least some of the metrics of a predefined number n of TMTs previously completed by the user. The value of n can correspond to a predetermined number of previously completed TMTs, or a predetermined period of time, or both. For example, the moving averages and standard deviations can be calculated for a fixed number of tests previously completed by the user (e.g., the previous 20 tests), for all tests completed by the user during the last month, and/or for a fixed number of tests (e.g., the last 20 tests) complete by the user during the last month.

In some embodiments, averages and standard deviations can be calculated separately for different periods. In particular, averages and standard deviations can be calculated for a first historical period comprising a plurality of first past metrics, and separate averages and standard deviations can be calculated for a second historical period comprising a plurality of second past metrics, with the first period being distinct from the second historical period. For example, averages and standard deviations can be calculated for a first month, and separate averages and standard deviations can be calculated for a second month. As another example, averages and standard deviations can be calculated for a first biannual or annual period, and separate averages and standard deviations can be calculated for a second biannual or annual period.

As can be appreciated, the moving average range [72] and the standard deviation range [74] can characterize a user’s normal range of cognitive performance. In some embodiments, the moving average range [72] and the standard deviation range [74] can be calculated for one or more groups of users to characterize the one or more groups’ normal range of cognitive performance. As an example, moving average ranges and standard deviation ranges can be calculated separately for different age groups.

The statistical parameters characterizing the historical metrics [70] can be calculated by the backend server [40], In some embodiments, the statistical parameters can be calculated and/or updated in response to receiving a plurality of metrics following completion of a TMT. In some embodiments, the statistical parameters can be calculated and/or updated at regular time intervals, and/or after receiving metrics from at least a predetermined number of completed TMTs.

Following completion of a TMT and calculating a corresponding plurality of metrics, a fifth step [150] of the method includes comparing the plurality of metrics with historical metrics to detect a possible change in cognitive function. The plurality of metrics can be compared, for example, with respect to the calculated statistical parameters characterizing the historical metrics [70], In some embodiments, this step can be performed on the backend server [40], for example by the comparator module [54], In some embodiments, this can be performed at least partially on the user device [20], For example, in such embodiments, the backend server [40] can transmit calculated statistical parameters characterizing the historical metrics [70] to the user device [20] (e.g., at regular intervals and/or upon request by the user device [20]), and the user device [20] can then compare at least one of a plurality of metrics of a completed TMT to the historical metrics [70] using the received statistical parameters.

In some embodiments, comparing the plurality of metrics with historical metrics can include determining whether at least one of the plurality of metrics of a completed TMT differs from corresponding historical metrics by a predetermined amount. For example, it can be determined whether at least one of the plurality of metrics differs from a moving average of corresponding historical metrics by at least two standard deviations or by a statistically significant amount.

In some embodiments, comparing the plurality of metrics with historical metrics can include determining whether metrics associated with a first historical period differ from metrics associated with a second historical period by a predetermined amount. For example, it can be determined whether there is a statistically significant shift (e.g., p < 0.05) in statistical parameters (e.g., moving average and standard deviation) between the first historical period and the second historical period. The plurality of metrics of a recently completed TMT can be included in the first historical period (e.g., a period corresponding to the current month and/or comprising metrics of a predetermined number n of TMTs completed during the current month). The first historical period can be distinct from the second historical period (e.g., a period corresponding to the previous month and/or comprising metrics of a predetermined number n of TMTs completed during the previous month).

In some embodiments, comparing the plurality of metrics with historical metrics can include comparing the plurality of metrics with historical metrics associated with the same user. In other words, upon a user completing a TMT and a plurality of metrics being calculated, the calculated metrics can be compared with corresponding metrics of TMTs previously completed by that user.

In some embodiments, comparing the plurality of metrics can include comparing the plurality of metrics with historical metrics associated with other users or groups of users. In other words, metrics characterizing a user’s completion of one or more TMTs can be compared with corresponding metrics characterizing completion of TMTs by other users and/or groups of users, for example to compare a user’s metrics with those of other user in the same age group as the user, or other defined groups. In some embodiments, comparing a user’s metrics with historical metrics associated with other users or groups of users can include comparing metrics characterizing a user’s completion of a TMT with a given serial number (and/or given range of serial numbers) with metrics characterizing other users’ completion of a TMT with the same serial number (and/or range of serial numbers).

In some embodiments, comparing the plurality of metrics can include comparing at least the standard metric Total Test Time described above. In some embodiments, comparing the plurality of metrics can include comparing at least one enhanced metric as described above.

A final step [170] can include generating and presenting an alert to signal a possible change in cognitive function. The alert can be generated when it is determined that the plurality of metrics differs from historical metrics based on the comparison in step [150], For example, if it is determined in step [160], that a metric differs from a moving average by at least two standard deviations or by a statistically significant amount, an alert can be generated to notify the user and/or the administrator [170], thus affording an opportunity to initiate an intervention against the detected change in cognitive state. Subsequently, the user can continue with the method for continued monitoring of cognitive function changes. If no significant difference has been identified in step [160], then an alert need not be generated, and the user continues with the method until a change is detected.

In some embodiments, the alert can be generated and presented on the user device [20], For example, upon completion of a TMT, the user device can compare metrics of the completed TMT with statistical parameters of historical metrics previously received from the backend server [40], The user device [20] can then immediately generate and present an alert following completion of the TMT, for example at the same time as presenting the test results (e.g., indicating the total test time and/or completion score). The alert can be presented, for example, by providing a visual, audible, and/or tactile output via the output module of the user device, for example in the form of a warning, suggestion, advisory, or pop-up message.

In some embodiments, the alert can be generated by the backend server [40] and presented by the user device [20], For example, upon comparing metrics across different periods (e.g., at the end of the month, to compare metrics of a current month with those of a previous month) and determining metrics associated with a first period differ from metrics associated with a second period, and alert [75] can be generated by alert generator module [56] of backend server [40] transmitted via the backend communication module [58] to the user device [20] via the user device communication module [28], The alert [75] can then be presented by the user device [20] by providing a corresponding output via output module of user device [20], The alert transmitted and present in the fashion can, for example, be implemented in the form of a push notification.

As can be appreciated, comparing the metrics with historical metrics step [150] and generating an alert for possible changes in cognitive function step [170] can be carried out using artificial intelligence (Al). In some embodiments, a machine learning (ML) model can be trained to determine whether at least some of the metrics [80] received from a completed TMT deviate from expected metrics. For example, the ML model can be trained on one or more datasets containing TMT metrics from different users, based on statistical parameters of historical metrics previously received from the backend server [40], and based on the characteristics of the user (such as age or medical history).

Further, the ML model can be trained using supervised training techniques to recognize deviations of interest in metrics of the completed TMT, based on statistical parameters of historical metrics previously received from the backend server [40], and to generate appropriate alert. The ML model can, for example, be trained using historical metrics and/or training data comprising scientifically validated metrics.

In some embodiments, the alert can be generated and presented by the backend server [40], For example, reports can be generated on the backend server [40] which can be accessed by administrators [7] via web-based portal [8], or automatically sent to administrators via email. In some embodiments, the reports can be automatically sent to a primary healthcare provider or integrated into a user’s patient database at a healthcare institution. The reports can present and/or summarize test results and metrics of TMTs completed by one or more users, and provide alerts in the form of visual indications when significant deviations are detected in historical metrics. It is appreciated that the backend server [40] will be organized such that alert records can be readily searched for, compiled, and analyzed.

Figures 8A and 8B show an example of a view from the web-based portal [8] in which reports of historical metrics of daily test results from a single user over a weekly time period are presented to administrator [7], In the example shown, the administrator [7] can view a graphical representation of the historical moving average range [72] and the historical standard deviation range [74] over the time. The graphical representation separately shows progress lines indicating a time average score [82] and a speed average score [84] (which are highly correlated) of the user over the weekly time period. Points of the progress line of the user for the time average score [82] (similarly for the speed average score [84]) that fall outside of an expected range (e.g., that differ from the historical moving average [72] by at least two standard deviations [74] at any given time) are presented as alerts [75, 75’]. The alert [75, 75’] can be presented in the form of a visual mark that distinguished from other points on the progress line (e.g., different color, highlighted, flashing, bigger, etc.). In some embodiments, the distinctiveness of the alert can be proportional to the magnitude of the deviation of the progress line from the expected range. For example, the alert [75] could be light red if the progress line is just above the historical range zone, or the alert [75] could be flashing red if the progress line is far from the expected range.

In some embodiments, the system and method as described above can be implemented on a TMT application [25] for portable device, such as an iOS or Android application. The TMT application [25] can further facilitate self-administration of the TMT by providing an easy-to-understand TMT tutorial and an integrated secure registration, login, and authentication system. The authenticated registration and login allow a secured connection to the backend server [40] and ensure proper privacy of data collected by the backend server [40], including private user information and/or historical metrics of the user.

The system and method as described herein allows remotely tracking cognitive function performance over time in individuals and groups of users. Through iterative use, the system and method can provide a capability to establish a cognitive function baseline in a resource and identify outliers where the resource is not in an optimal state to perform assigned tasks. In particular, early detection of changes in cognitive function can permit earlier consultation or commencement of an intervention. The system and method can also allow earlier feedback on the effectiveness of an intervention related to cognitive function. Moreover, as mentioned, users can have the ability to access and share test history with primary care providers to better guide supplemental assessments, diagnoses, interventions, and follow-up care. The long-term collection of data can allow assessing how cognitive function can be affected by wellness and other programs.

In some implementations, the system and method can be deployed to assess individuals in different workplaces, and results of a specific workplace can be combined with the results of other workplaces, allowing to identify workplace factors that affect employees’ or user’s cognitive function and/or work performance.

The system and method as described herein can be used as a cognitive function assessment tool in various application, including for example but not limited to:

- to monitor cognitive development of children,

- to monitor cognitive recovery following illness or injury, - to prevent accidents in the workplace due to detection of acute cognitive functional impairment of employees,

- to assess the effects of wellness programs and other events at work on employee’s cognitive function, thereby enabling optimization of employees’ performance,

- to monitor cognitive function changes due to training or education, or aging,

- to better understand human brain in various pathological states (e.g. ADHD).

While the method and system have been described in conjunction with the exemplary embodiments described above, many equivalent modifications and variations will be apparent to those skilled in the art when given this disclosure. Accordingly, the exemplary embodiments set forth above are considered to be illustrative and not limiting.

In particular, while an embodiment directed to a TMT-based cognitive assessment has been broadly described, it is understood that the system and method can be applied to any other digital neuropsychological test. For example, the system and method could be applied to detect changes in cognitive function over time using other neuropsychological tests and/or combinations of neuropsychological tests that involve different test procedures or instructions, including but not limited to tests that can be carried out using pre-generated and/or semi-randomized test canvases that comprise targets that are subject to user interaction. The scope of the claims should not be limited by the preferred embodiments set forth in this disclosure but should be given the broadest interpretation consistent with the description as a whole.

Claims

1. A computer-implemented method for detecting changes in cognitive function over time, the method comprising: displaying a digital trail making test (TMT) canvas on a user device comprising a touch-enabled display, the digital TMT canvas comprising a plurality of sequential targets; receiving user input via the touch-enabled display, the user input corresponding to a user completing the digital TMT by drawing a path on the touch-enabled display to connect the plurality of sequential targets; calculating a plurality of metrics characterizing the user’s completion of the digital TMT; comparing the plurality of metrics with historical metrics, the historical metrics comprising past metrics characterizing previously completed digital TMTs; and detecting a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined amount.

2. The method of claim 1 , comprising calculating moving averages and standard deviations of the historical metrics, and detecting the possible change in cognitive function when at least one of the plurality of metrics differs from its moving average by at least two standard deviations or by a statistically significant amount.

3. The method of claim 2, comprising displaying the historical metrics on a graphical user interface (GUI), and highlighting metrics that differ from their moving averages by the at least two standard deviations or by a statistically significant amount.

4. The method of any one of claims 1 to 3, comprising displaying an alert on the touch-enabled display of the user device when the possible change in cognitive function is detected.

5. The method according to any one of claims 1 to 4, wherein the historical metrics comprise past metrics characterizing digital TMTs previously completed by the user.

6. The method of any one of claims 1 to 5, wherein the historical metrics comprise past metrics characterizing digital TMTs previously completed by other users or a group of users.

7. The method of any one of claims 1 to 6, comprising comparing the plurality of metrics with historical metrics from a predetermined number of previously completed TMTs.

8. The method according to any one of claims 1 to 7, comprising comparing the plurality of metrics with historical metrics from TMTs completed during a predetermined previous period.

9. The method according to any one of claims 1 to 8, wherein the plurality of metrics is associated with a first historical period comprising a plurality of first past metrics, the first historical period being distinct from a second historical period comprising a plurality of second past metrics; further wherein comparing the plurality of metrics comprises comparing the plurality of first past metrics with the plurality of second past metrics.

10. The method according to claim 9, comprising calculating averages and standard deviations of the first past metrics, calculating averages and standard deviations of the second past metrics, and detecting the possible change in cognitive function when an average or standard deviation of a metric during the first historical period differs from an average or standard deviation of the metrics during the second historical period by a predetermined threshold, including a statistical significance threshold.

11. The method of any one of claims 1 to 10, wherein the digital TMT canvas comprises the plurality of sequential targets arranged according to a layout, further wherein displaying the digital TMT canvas comprises displaying the plurality of sequential targets according to a layout that differs from layouts of the previous TMTs completed by the user.

12. The method of any one of claims 1 to 11 , comprising pre-generating a plurality of layouts, wherein displaying the digital TMT canvas comprises selecting one of the pregenerated layouts, and displaying the plurality of sequential targets arranged according to the selected pre-generated layout.

13. The method of claim 12, comprising: pre-generating the plurality of layouts on a backend server; and transmitting the plurality of pre-generated layouts from the backend server to the user device.

14. The method of claim 13, wherein the plurality of pre-generated layouts are transmitted from the backend server to a plurality of user devices for completion by a plurality of users, the method further comprising comparing the plurality of metrics characterizing the user’s completion of the digital TMT with sequential targets displayed according to one of the plurality of pre-generated layouts with historical metrics comprising past metrics characterizing other users’ completion of digital TMTs with sequential targets displayed according to the same one of the plurality of pre-generated layouts.

15. The method of claim 13, wherein pre-generating the plurality of layouts comprises associating a unique serial number with each layout; and wherein displaying the digital TMT canvas comprises selecting one of the pre-generated layouts having a serial number that differs from serials numbers of previous TMTs completed by the user, and/or selecting a specific pre-generated layout across a plurality of user devices to permit metrics comparison across a plurality of users based solely on the specific pre-generated layout.

16. The method of any one of claims 1 to 15, wherein the plurality of metrics are recorded from coordinate data and temporal data indicative of where and when the user touched the touch-enabled display to draw the path to connect the plurality of sequential targets.

17. The method of any one of claims 1 to 16, wherein the plurality of metrics comprises at least one of: a time spent to draw each correct segments, excluding a time spent in the sequential targets, and excluding a time from test start until the first sequential target is touched; a time spent to draw correct and incorrect segments, excluding the time spent in the sequential targets, and excluding the time from test start until the first sequential target is touched; a mean across all correct segments of the time spent to draw each correct segment, excluding a time spent in the sequential targets, and excluding a time from test start until the first sequential target is touched; a mean across all correct and incorrect segments of the time spent to draw correct and incorrect segments, excluding the time spent in the sequential targets, and excluding the time from test start until the first sequential target is touched; a mean across all correct segments of the difference between the path drawn and an ideal segment, normalized by the length of the ideal segment; a mean across all correct and incorrect segments of the difference between the path drawn and the ideal segment, normalized by the length of the ideal segment; a mean of distance drawn by the user over time to complete all the correct segments, normalized by the sum total distance of all ideal segment paths; a mean of distance drawn by the user over time to complete all the correct and incorrect segments, normalized by the sum total distance of all ideal segment paths; a standard deviation of the mean of distance drawn by the user over time to complete all the correct segments, normalized by the length of the ideal segment; a standard deviation of the mean of distance drawn by the user over time to complete all the correct and incorrect segments, normalized by the length of the ideal segment; a mean of ideal segments distance over time to complete all correct segments; a mean of ideal segments distance over time to complete all correct and incorrect segments; a mean of a standard deviation in an instantaneous velocity calculated for every coordinate data pair across all correct segments; a mean of a standard deviation in an instantaneous velocity calculated for every coordinate data pair across all correct and incorrect segments; a time from test start until the first sequential target is touched by the user; a sum of the time spent by the user in the sequential targets over all consecutive sequential targets; a sum of the time spent by the user in the sequential targets over all consecutive and non-consecutive sequential targets; and a time spent by the user to complete the TMT from test onset until a last sequential target is correctly reached; wherein the correct segment corresponds to the path drawn by the user between two consecutive sequential targets, the incorrect segment corresponds to the path drawn by the user between two non-consecutive sequential targets, and the ideal segment corresponds to a straight line connecting a center of two consecutive sequential targets.

18. The method of any one of claims 1 to 17, wherein the plurality of metrics further comprises at least one of: a time average score; and a speed average score.

19. The method of any one of claims 1 to 18, wherein detecting the possible change in cognitive function comprises determining whether at least some of the plurality of metrics deviate from expected metrics using a machine learning model.

20. A system for detecting changes in cognitive function over time, the system comprising: a user device comprising: at least one processor; a communications module; a touch-enabled display; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the user device to: display a digital TMT canvas on the touch-enabled display, the digital TMT canvas comprising a plurality of sequential targets; receive user input via the touch-enabled display, the user input corresponding to a user completing the digital TMT by drawing a path on the touch-enabled display to connect the plurality of sequential targets; record a plurality of metrics characterizing the user’s completion of the digital TMT; and transmit the plurality of metrics via the communications module; a backend server comprising: at least one processor; a communications module; persistent storage storing historical metrics comprising past metrics characterizing previously completed digital TMTs; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the backend server to: receive the plurality of metric from the user device via the communications module; compare the plurality of metrics with the historical metrics, and generate an indication of a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined or statistically significant amount; and store the plurality of metrics in the persistent storage to update the historical metrics; and a user interface that obtains results associated with a given TMT for a given user, wherein the results are transmitted to the backend server; wherein the backend server further comprises a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising: retrieving test results associated with the given user; generating metrics; and transmitting to the user interface the metrics of the given user.

21. A non-transitory computer-readable storage medium storing instructions that, when executed by a computing system having one or more processors, cause the computing system to perform a method for detecting changes in cognitive function over time, comprising: displaying a digital trail making test (TMT) canvas on a user device comprising a touch-enabled display, the digital TMT canvas comprising a plurality of sequential targets; receiving user input via the touch-enabled display, the user input corresponding to a user completing the digital TMT by drawing a path on the touch-enabled display to connect the plurality of sequential targets; calculating a plurality of metrics characterizing the user’s completion of the digital TMT; comparing the plurality of metrics with historical metrics, the historical metrics comprising past metrics characterizing previously completed digital TMTs; and detecting a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined amount.

22. A computer-implemented method for detecting changes in cognitive function over time, the method comprising: displaying a neuropsychological test canvas on a user device comprising a touch- enabled display, the neuropsychological test canvas comprising a plurality of targets; receiving user input via the touch-enabled display, the user input corresponding to a user completing the digital neuropsychological test by interacting with targets on the touch-enabled display in compliance with neuropsychological test instructions; calculating a plurality of metrics characterizing the user’s completion of the neuropsychological test; comparing the plurality of metrics with historical metrics, the historical metrics comprising past metrics characterizing previously completed neuropsychological tests; and detecting a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined amount.

23. A system for detecting changes in cognitive function over time, the system comprising: a user device comprising: at least one processor; a communications module; a touch-enabled display; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the user device to: display a neuropsychological test canvas on the touch-enabled display, the neuropsychological test canvas comprising a plurality of targets; receive user input via the touch-enabled display, the user input corresponding to a user completing the neuropsychological test by interacting with targets on the touch-enabled display in compliance with neuropsychological test instructions; record a plurality of metrics characterizing the user’s completion of the neuropsychological test; and transmit the plurality of metrics via the communications module; a backend server comprising: at least one processor; a communications module; persistent storage storing historical metrics comprising past metrics characterizing previously completed neuropsychological tests; and a memory having instructions stored thereon which, when executed by the at least one processor, cause the backend server to: receive the plurality of metric from the user device via the communications module; compare the plurality of metrics with the historical metrics, and generate an indication of a possible change in cognitive function when at least one of the plurality of metrics differs from the historical metrics by a predetermined or statistically significant amount; and store the plurality of metrics in the persistent storage to update the historical metrics; and a user interface that obtains results associated with a given neuropsychological test for a given user, wherein the results are transmitted to the backend server; wherein the backend server further comprises a memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising: retrieving test results associated with the given user; generating metrics; and transmitting to the user interface the metrics of the given user.