[go: up one dir, main page]

WO2023034347A1 - Technologie portable multi-sensorielle d'assistance, et procédé de fourniture d'un soulagement sensoriel à l'aide de celle-ci - Google Patents

Technologie portable multi-sensorielle d'assistance, et procédé de fourniture d'un soulagement sensoriel à l'aide de celle-ci Download PDF

Info

Publication number
WO2023034347A1
WO2023034347A1 PCT/US2022/042100 US2022042100W WO2023034347A1 WO 2023034347 A1 WO2023034347 A1 WO 2023034347A1 US 2022042100 W US2022042100 W US 2022042100W WO 2023034347 A1 WO2023034347 A1 WO 2023034347A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
sensory
mediation
data
autistic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2022/042100
Other languages
English (en)
Other versions
WO2023034347A9 (fr
Inventor
David RUTTENBERG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Phoeb X Inc
Original Assignee
Phoeb X Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US17/882,517 external-priority patent/US11779275B2/en
Application filed by Phoeb X Inc filed Critical Phoeb X Inc
Priority to CA3230610A priority Critical patent/CA3230610A1/fr
Priority to EP22865462.0A priority patent/EP4396842A4/fr
Priority to JP2024556032A priority patent/JP2025500073A/ja
Publication of WO2023034347A1 publication Critical patent/WO2023034347A1/fr
Anticipated expiration legal-status Critical
Publication of WO2023034347A9 publication Critical patent/WO2023034347A9/fr
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/168—Evaluating attention deficit, hyperactivity
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/486—Biofeedback
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A61B5/6803—Head-worn items, e.g. helmets, masks, headphones or goggles
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00—ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/60—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records
    • G16H10/65—ICT specially adapted for the handling or processing of patient-related medical or healthcare data for patient-specific data, e.g. for electronic patient records stored on portable record carriers, e.g. on smartcards, RFID tags or CD
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment

Definitions

  • This application addresses the above-described challenges, by providing a wearable technology that offers ground-breaking opportunities to: (i) monitor environments and adjust user-experiences; (ii) lessen sensory-load and enable greater participation; and (iii) improve mental health with efficacious interventions.
  • the wearable technology described herein increases attentional focus, reduces sensory distraction, and improves quality-of- life/lessens anxiety and fatigue.
  • One embodiment of the application is directed to a system, comprising: a wearable device comprising one or more sensors; one or more processors; and one or more non-transitory computer-readable media having executable instructions stored thereon that, when executed by the one or more processors, cause the system to perform operations comprising: obtaining user sensory sensitivity data corresponding to user input indicating whether a user of the wearable device is visually sensitive, sonically sensitive, or interoceptively sensitive; determining, using at least the user sensory sensitivity data, one or more sensory thresholds specific to the user and mediation data corresponding to one or more mediations specific to the user, the one or more sensory threshold selected from auditory, visual, or physiological sensory thresholds; storing the one or more sensory thresholds and the mediation data; recording, using the one or more sensors, a sensory input stimulus to the user; comparing the sensory input stimulus with the one or more sensory thresholds specific to the user; in response to comparing the sensory input stimulus with the one or more sensory thresholds, determining, based at least on the mediation data,
  • the operations further comprise: storing a first identifier that indicates whether the user is neurodiverse or neurotypical; and determining the one or more sensory thresholds specific to the user and the mediation data corresponding to one or more mediations specific to the user, comprises: determining, using at least the first identifier and the user sensory sensitivity data, the one or more sensory thresholds and the mediation data.
  • the operations further comprise: receiving user demographic data corresponding to user input indicating an age, education level, or gender of the user; and determining the one or more sensory thresholds specific to the user and the mediation data corresponding to one or more mediations specific to the user, comprises: determining, using at least the first identifier, the user sensory sensitivity data, and the user demographic data, the one or more sensory thresholds and the mediation data.
  • the first identifier indicates whether or not the user is autistic.
  • the first identifier indicates that the user is autistic.
  • the mediation is configured to provide the user relief from fatigue; the mediation comprises the filter mediation; and the filter mediation comprises filtering, in real-time, an audio signal presented to the user or an optical signal presented to the user.
  • the mediation is configured to provide the user relief from a distraction by increasing a response time of the user to the distraction.
  • obtaining the user sensory sensitivity data comprises receiving, at a graphical user interface, one or more first responses by the user to one or more first prompts indicating whether the user is visually sensitive, sonically sensitive, or interoceptively sensitive; and the operations further comprise deriving the first identifier indicating that the user is autistic by: receiving, at the graphical user interface, one or more second responses by the user to one or more second prompts indicating an anxiety level of the user; deriving, based on the sensory sensitivity data, one or more sensory sensitivity scores comprising a visual sensitivity score, a sonic sensitivity score, or an interoceptive sensitivity score; deriving, based on the one or more second responses, an anxiety score; and predicting, using a model that predicts a probability of autism based at least on an anxiety level and one or more sensory sensitivity levels, based at least on the anxiety score and the one or more sensory sensitivity scores, that the user is autistic.
  • obtaining the user sensory sensitivity data comprises receiving, at a graphical user interface, one or more first responses by the user to one or more first prompts indicating whether the user is visually sensitive, sonically sensitive, or interoceptively sensitive; and the operations further comprise deriving the first identifier indicating that the user is autistic by: receiving, at the graphical user interface, one or more second responses by the user to one or more second prompts indicating a fatigue level of the user; deriving, based on the sensory sensitivity data, one or more sensory sensitivity scores comprising a visual sensitivity score, a sonic sensitivity score, or an interoceptive sensitivity score; deriving, based on the one or more second responses, a fatigue score; and predicting, using a model that predicts a probability of autism based at least on a fatigue level and one or more sensory sensitivity levels, based at least on the fatigue score and the one or more sensory sensitivity scores, that the user is autistic.
  • obtaining the user sensory sensitivity data further comprises: recording, using at least the one or more sensors, a response by the user to a visual stimulus, a sonic stimulus, or a physiological stimulus.
  • the mediation comprises a combination mediation of at least two mediations selected from the alert mediation, the guidance mediation, and the filter mediation.
  • the combination mediation comprises the alert mediation followed by the filter mediation.
  • the alert mediation comprises alerting the user about a distraction that is visual or auditory; and the filter mediation comprises: comprising filtering, in real-time, an audio or optical signal presented to the user, the audio or optical signal associated with the distraction.
  • the system further comprises one or more fog nodes configured to locally store sensor data collected by the one or more sensors, the sensor data including first sensor data associated with the sensory input stimulus.
  • storing the one or more sensory thresholds and the mediation data comprises: locally storing, using the one or more fog nodes, the one or more sensory thresholds and the mediation data; and comparing the sensory input stimulus with the one or more sensory thresholds, comprises: comparing, using the one or more fog nodes, the sensory input stimulus with the one or more sensory thresholds.
  • the system further comprises one or more edge nodes configured to communicatively couple to the one or more fog nodes and a cloud server remotely located from the wearable device.
  • the one or more edge nodes are configured to: encrypt the first sensor data associated with the sensory input stimulus to obtain encrypted data; transmit the encrypted data to the cloud server; and receive a response from the cloud server.
  • the one or more fog nodes and the one or more edge nodes reside on a local area network (LAN) containing the wearable device, an ad-hoc network containing the wearable device, a LAN of a mobile device directly coupled to the wearable device, or an ad-hoc network of the mobile device.
  • LAN local area network
  • the sensor data comprises second sensor data that does not trigger a mediation; and the system is configured such that the second sensor data that does not trigger a mediation is not made available to any cloud server remotely located from the wearable device.
  • the mediation comprises the filter mediation that comprises filtering, in real-time, an optical signal presented to the user;
  • the first sensor data associated with the sensory input stimulus comprises first image data;
  • the one or more edge nodes or the one or more fog nodes are configured to determine whether the first image data is sufficiently similar to second image data stored at the cloud server; and determining the mediation to be provided to the user comprises in response to determining that the first image data is sufficiently similar to the second image data, determining the filter mediation.
  • the mediation comprises the filter mediation that comprises filtering, in real-time, an audio signal presented to the user; the first sensor data associated with the sensory input stimulus comprises first audio data; the one or more edge nodes or the one or more fog nodes are configured to determine whether the first audio data is sufficiently similar to second audio data stored at the cloud server; and determining the mediation to be provided to the user comprises in response to determining that the first audio data is sufficiently similar to the second audio data, determining the filter mediation.
  • the operations further comprise: presenting to the user, on a graphical user interface, one or more access controls for controlling user data that is made available to one or more other users, the user data comprising sensor data collected by the one or more sensors, the one or more sensory thresholds, the mediation data, or a record of mediations presented to the user; and receiving data corresponding to user input selecting the one or more access controls.
  • the one or more access controls may be configured such that only sensor data that triggered a mediation is accessible to one or more other users (e.g., a general practitioner, a therapist, a family member, etc.).
  • the one or more access controls can be configured such that certain types of sensor data (e.g., image or audio data of the environment) are not made available to other users.
  • the one or more access controls can be configured such that there are different hierarchies of data access, where some users have more access to certain types of data than other users.
  • the operations further comprise: presenting to the user, on a graphical user interface, one or more access controls that grant or deny access to one or more other users to influence mediations that are presented to the user; and receiving data corresponding to user input actuating the one or more access controls.
  • a wearer user can grant a therapist user access to modify the user’s preferences to optimize the mediation that is presented to the user.
  • certain types of mediations can be disabled or enabled.
  • the operations further comprise: presenting to the user, on a graphical user interface, a graphical summary of progress of the user from using the wearable device, the graphical summary including a moving average or change of time between mediations.
  • the graphical summary of progress of the user can indicate a change in the sensory thresholds and/or mediations over time, a change/moving average of the user’s average response time to distracting stimuli, a change / moving average of the number of mediations required in some time frame (e.g., during the day) and/or some event (e.g., while in the workplace or classroom), etc.
  • the one or more sensors comprise multiple sensors of different types, the multiple sensors comprising: an auditory sensor, a galvanic skin sensor, a pupillary sensor, a body temperature sensor, a head sway sensor, or an inertial movement unit; recording the sensory input stimulus to the user comprises obtaining first sensory data corresponding to a first sensory input stimulus from a first sensor of the multiple sensors, and second sensory data corresponding to a second sensory input stimulus from a second sensor of the multiple sensors; and determining the mediation to be provided to the user, comprises: inputting at least the first sensory data and the second sensory data into a fusion-based deep leaming (FBDL) model that outputs an identification of the mediation to be provided to the user.
  • FBDL fusion-based deep leaming
  • determining the mediation to be provided to the user comprises: inputting at least the first sensory data, the second sensory data, and the mediation data into the FBDL model that outputs the identification of the mediation to be provided to the user.
  • One embodiment of the application is directed to a method, comprising: obtaining, at a wearable device system, user sensory sensitivity data corresponding to user input indicating whether a user of a wearable device of the wearable device system is visually sensitive, sonically sensitive, or interoceptively sensitive; determining, at the wearable device system, using at least the user sensory sensitivity data, one or more sensory thresholds specific to the user and mediation data corresponding to one or more mediations specific to the user, the one or more sensory threshold selected from auditory, visual, or physiological sensory thresholds; storing, at a storage of the wearable device system, the one or more sensory thresholds and the mediation data; recording, using one or more sensors of the wearable device system, a sensory input stimulus to the user; comparing, at the wearable device system, the sensory input stimulus with the one or more sensory thresholds specific to the user; in response to comparing the sensory input stimulus with the one or more sensory thresholds, determining, based at least on the mediation data, a mediation to be provided to the user, the
  • One embodiment of the application is directed to a system, comprising: a wearable device comprising one or more sensors; one or more processors; and one or more non-transitory computer-readable media having executable instructions stored thereon that, when executed by the one or more processors, cause the system to perform operations comprising: connecting to a datastore that stores one or more sensory thresholds specific to a user of the wearable device, the one or more sensory thresholds selected from auditory, visual or physiological sensory thresholds; recording, using the one or more sensors, a sensory input stimulus to the user; comparing the sensory input stimulus with the one or more sensory thresholds specific to the user to determine an intervention to be provided to the user, the intervention configured to provide the user relief from distractibility, inattention, anxiety, fatigue, or sensory issues; and providing the intervention to the user, the intervention comprising filtering, in real-time, an audio signal presented to the user or an optical signal presented to the user.
  • the physiological sensory thresholds can be physiological/psychophysiological sensory thresholds.
  • the operations further comprise: communicatively coupling the system to an Internet of Things (loT) device, the sensory input stimulus generated at least in part due to sound emitted by a speaker of the loT device or light emitted by a light emitting device of the loT device; and providing the intervention to the user, comprises: controlling the loT device to filter, in real-time, the audio signal or the optical signal.
  • LoT Internet of Things
  • the loT device comprises the light emitting device; controlling the loT device to filter, in real-time, the audio signal or the optical signal, comprises controlling the loT device to filter, in real-time, the optical signal; and filtering the optical signal adjusts a brightness or color of light output by the lighting device.
  • the loT device comprises the speaker; controlling the loT device to filter, in real-time, the audio signal or the optical signal, comprises controlling the loT device to filter, in real-time, the audio signal; and filtering the audio signal adjusts a frequency of sound output by the speaker.
  • the wearable device further comprises a bone conduction transducer or a hearing device; and providing the intervention to the user comprises: filtering, at the wearable device, in real-time, the audio signal in a frequency domain; and after filtering the audio signal, presenting the audio signal to the user by outputting, using the bone conduction transducer or the hearing device, a vibration or sound wave corresponding to the audio signal.
  • the wearable device further comprises a head mounted display (HMD) that presents the optical signal to the user, the HMD worn by the user; and providing the intervention to the user further comprises filtering, in real-time, the optical signal by modifying a real-time image of the real-world environment presented to the user via the HMD.
  • HMD head mounted display
  • comparing the sensory input stimulus with the one or more sensory thresholds specific to the user to determine the intervention to be provided to the user comprises: determining, based on the same sensor data recorded by the one or more sensors, to filter the audio signal and to filter the optical signal.
  • the wearable device further comprises a HMD that presents the optical signal to the user, the HMD worn by the user; and providing the intervention to the user includes filtering, in real-time, the optical signal by modifying a realtime image of the real-world environment presented to the user via the HMD.
  • modifying the real-time image comprises inserting a virtual object into the real-time image or modifying the appearance of an object of the real- world environment in the real-time image.
  • comparing the sensory input stimulus with the one or more sensory thresholds specific to the user to determine the intervention to be provided to the user comprises: inputting the sensory input stimulus and the one or more user-specific sensory thresholds into a trained model to automatically determine, based on an output of the trained model, a visual intervention to be provided to the user.
  • the one or more sensors comprise multiple sensors of different types, the multiple sensors comprising: an auditory sensor, a galvanic skin sensor, a pupillary sensor, a body temperature sensor, a head sway sensor, or an inertial movement unit; recording the sensory input stimulus to the user comprises recording a first sensory input stimulus from a first sensor of the multiple sensors, and a second sensory input stimulus from a second sensor of the multiple sensors; and inputting the sensory input stimulus into the trained model comprises inputting the first sensory input stimulus and the second sensory input stimulus into the trained model.
  • the visual intervention comprises: presenting an alert to the user of a visually distracting object; and after it is determined that the user does not sufficiently respond to the alert within a period of time, filtering, in real-time, the optical signal presented to the user.
  • the visual intervention comprises: filtering, in realtime, the optical signal to hide a visually distracting object without providing a prior alert to the user that the visually distracting object is present.
  • the operations further comprise determining the one or more sensory thresholds specific to the user and one or more interventions specific to the user by: presenting multiple selectable templates to the user, each of the templates providing an indication of whether the user is visually sensitive, sonically sensitive, or interoceptively sensitive, and each of the templates associated with corresponding one or more sensory thresholds and one or more interventions; and receiving data corresponding to input by the user selecting one of the templates.
  • determining the one or more sensory thresholds specific to the user and the one or more interventions specific to the user further comprises: receiving additional data corresponding to additional user input selecting preferences, the preferences comprising audio preferences, visual preferences, physiological preferences, alert preferences, guidance preferences, or intervention preferences; and in response to receiving the additional data, modifying the one or more thresholds and the one or more interventions of the selected template to derive the one or more sensory thresholds specific to the user and the one or more interventions specific to the user.
  • the physiological preferences are psychophysiological preferences.
  • comparing the sensory input stimulus with the one or more sensory thresholds specific to the user to determine the intervention to be provided to the user comprises: inputting the sensory input stimulus and the one or more user-specific sensory thresholds into a trained model to automatically determine, based on an output of the trained model, the intervention to be provided to the user.
  • the user is neurodiverse. In some implementations, the user can be autistic.
  • the intervention further comprises an alert intervention; and with the alert intervention, a response time for the user increases by at least 3% and accuracy increases by at least about 26% from baseline for errors of commission, the errors of commission being a measure of a failure of the user to inhibit a response when prompted by a feedback device.
  • the intervention further comprises a guidance intervention; and with the guidance intervention, a response time for the user increases by at least about 20% and accuracy increases by at least about 10% from baseline for errors of commission, the errors of commission being a measure of a failure of the user to inhibit a response when prompted by a feedback device.
  • the intervention further comprises a guidance intervention; and with the guidance intervention, a response time for the user increases by at least about 2% and accuracy increases by at least about 30% from baseline for errors of omission, the errors of omission being a measure of a failure of the user to take appropriate action when a prompt is not received from a feedback device.
  • a response time for the user increases by at least about 10% from baseline for errors of omission, the errors of omission being a measure of a failure of the user to take appropriate action when a prompt is not received from a feedback device.
  • a response time for the user is at least about 15% faster than would be a response time for a neurotypical user using the system for errors of omission, the errors of omission being a measure of a failure of the user to take appropriate action when a prompt is not received from a feedback device.
  • the intervention further comprises a guidance intervention; and with the guidance intervention, a response time for the user is at least about 20% faster and accuracy is about 8% higher than would be a response time and accuracy of a neurotypical user using the system for errors of commission, the errors of commission being a measure of a failure of the user to inhibit a response when prompted by a feedback device.
  • the intervention further comprises an alert intervention; and with the alert intervention, accuracy for the user is at least about 25% higher than would be an accuracy of a neurotypical user using the system for errors of commission, the errors of commission being a measure of a failure of the user to inhibit a response when prompted by a feedback device.
  • One embodiment of the application is directed to a method, comprising: connecting a wearable device system to a datastore that stores one or more sensory thresholds specific to a user of a wearable device of the wearable device system, the one or more sensory thresholds selected from auditory, visual or physiological sensory thresholds; recording, using one or more sensors of the wearable device, a sensory input stimulus to the user; comparing, using the wearable device system, the sensory input stimulus with the one or more sensory thresholds specific to the user to determine an intervention to be provided to the user, the intervention configured to provide the user relief from distractibility, inattention, anxiety, fatigue, or sensory issues; and providing, using the wearable device system, the intervention to the user, the intervention comprising filtering, in real-time, an audio signal presented to the user or an optical signal presented to the user.
  • the physiological preferences are psychophysiological preferences.
  • the method further comprises communicatively coupling the wearable device system to an loT device; providing the intervention to the user comprises controlling the loT device to filter, in real-time, the audio signal or the optical signal; and the sensory input stimulus generated at least in part due to sound emitted by a speaker of the loT device or light emitted by a light emitting device of the loT device.
  • the loT device comprises the light emitting device; controlling the loT device to filter, in real-time, the audio signal or the optical signal, comprises controlling the loT device to filter, in real-time, the optical signal; and filtering the optical signal adjusts a brightness or color of light output by the lighting device.
  • the loT device comprises the speaker; controlling the loT device to filter, in real-time, the audio signal or the optical signal, comprises controlling the loT device to filter, in real-time, the audio signal; and filtering the audio signal adjusts a frequency of sound output by the speaker.
  • the wearable device further comprises a bone conduction transducer or a hearing device; and providing the intervention to the user comprises: filtering, at the wearable device, in real-time, the audio signal in a frequency domain; and after filtering the audio signal, presenting the audio signal to the user by outputting, using the bone conduction transducer or the hearing device, a vibration or sound wave corresponding to the audio signal.
  • One embodiment of this application is directed to a system for providing sensory relief from distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, to a user in need thereof, the system comprising: (i.) a wearable device; (ii) a database of one or more user-specific sensory thresholds selected from auditory, visual, and physiological sensory thresholds, one or more user-specific sensory resolutions selected from auditory, visual and physiological sensory resolutions, or combinations thereof; (iii) an activation means for connecting the wearable device and the database; (iv) one or more sensors for recording a sensory input stimulus to the user; (v) a comparing means for comparing the sensory input stimulus recorded by the one or more sensors with the database of one or more user-specific sensory thresholds to obtain a sensory resolution for the user; (vi) one or more feedback devices for transmitting the sensory resolution to the user; and (vii) a user-specific intervention means for providing relief to the user from the distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof.
  • the user-specific intervention means is selected from an alert intervention, a filter intervention, a guidance intervention, or a combination thereof, and the user can be a neurodiverse user or a neurotypical user.
  • the neurodiverse user can be an autistic user.
  • the physiological sensory thresholds are psychophysiological sensory thresholds
  • the physiological sensory resolutions are psychophysiological sensory resolutions.
  • the wearable device is an eyeglass frame comprising the one or more sensors and the one or more feedback devices.
  • the one or more sensors are selected from one or more infrared sensors, one or more auditory sensors, one or more galvanic skin sensors, one or more inertial movement units, or combinations thereof.
  • the one or more feedback devices are selected from one or more haptic drivers, one or more bone conduction transducers, or combinations thereof.
  • the system further comprises a wireless or wired hearing device.
  • the sensory input stimulus is selected from an ecological auditory input, an ecological visual input, a egocentric physiological/psychophysiological input, or combinations thereof.
  • the sensory input stimulus is measured by evaluating one or more parameters selected from eye tracking, pupillometry, auditory cues, interoceptive awareness, physical movement, variations in body temperature or ambient temperatures, pulse rate, respiration, or combinations thereof.
  • the sensory resolution is provided by one or more alerts selected from a visual alert, an auditory alert, a physiological/psychophysiological alert, a verbal alert, or combinations thereof.
  • the activation means is a power switch located on the wearable device.
  • the power switch is located at a left side of the wearable device.
  • the power switch is located at a right side of the wearable device.
  • the power switch is a recessed power switch.
  • the database is stored in a storage device.
  • the storage device is selected from a fixed or movable computer system, a portable wireless device, a smartphone, a tablet, or combinations thereof.
  • a response time for autistic users increases by at least about 3% and accuracy increases by at least about 26% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a response time for neurotypical users increases by at least about 18% and accuracy increases by at least about 2.0% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a response time for autistic users increases by at least about 20% and accuracy increases by at least about 10% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a response time for autistic users increases by at least about 2% and accuracy increases by at least about 30% from baseline for errors of omission, wherein the errors of omission is a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a response time for autistic users increases by at least about 10% from baseline for errors of omission, wherein the errors of omission is a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a response time for autistic users is at least about 15% faster than neurotypical users for errors of omission, wherein the errors of omission are a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a response time for autistic users is at least about 20% faster and accuracy is about 8% higher than neurotypical users for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • accuracy for autistic users is at least about 25% higher than neurotypical users for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a method of providing sensory relief from distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, to a user in need thereof comprises: creating a database of one or more user-specific sensory thresholds selected from auditory, visual and physiological/psychophysiological sensory thresholds, one or more user-specific sensory resolutions selected from auditory, visual and physiological/psychophysiological sensory resolution, or combinations thereof; attaching a wearable device to the user, wherein the wearable device comprises one or more sensors and one or more feedback devices; activating and connecting the wearable device to the database; recording a sensory input stimulus to the user via the one or more sensors; comparing the sensory input stimulus with the database of one or more user-specific sensory thresholds; selecting an appropriate user-specific sensory resolution from the database; delivering the user-specific sensory resolution to the user via the one or more feedback devices; and providing a user-specific intervention (a/k/a digital mediation) to provide relief to the user from the distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, wherein
  • the one or more feedback devices is selected from one or more haptic drivers, one or more bone conduction transducers, or combinations thereof.
  • the sensory input stimulus is selected from an auditory input, a visual input, a physiological/psychophysiological input or combinations thereof.
  • the sensory input stimulus is measured by one or more parameters selected from eye tracking, pupillometry, auditory cues, interoceptive awareness, physical movement, variations in body or ambient temperatures, pulse rate, respiration, or combinations thereof.
  • the user-specific sensory resolution is provided by one or more alerts selected from a visual alert, an auditory alert, a physiological/psychophysiological alert, a verbal alert or combinations thereof.
  • the activation and connection of the wearable device to the database is through a power switch located on the wearable device.
  • the power switch is located at a left side of the wearable device or a right side of the wearable device
  • the power switch is a recessed power switch.
  • the wearable device is an eyeglass frame.
  • the database is stored in a storage device.
  • the storage device is selected from a fixed or movable computer system, a portable wireless device, a smartphone, a tablet, or combinations thereof.
  • a response time for autistic users increases by at least about 3% and accuracy increases by at least about 26% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a response time for neurotypical users increases by at least about 18% and accuracy increases by at least about 2.0% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a response time for autistic users increases by at least about 20% and accuracy increases by at least about 10% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a response time for autistic users increases by at least about 2% and accuracy increases by at least about 30% from baseline for errors of omission, wherein the errors of omission are a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a response time for autistic users increases by at least about 10% from baseline for errors of omission, wherein the errors of omission are a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a response time for autistic users is at least about 15% faster than neurotypical users for errors of omission, wherein the errors of omission are a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a response time for autistic users is at least about 20% faster and accuracy is about 8% higher than neurotypical users for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • accuracy for autistic users is at least about 25% higher than neurotypical users for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a wearable device comprises one or more sensors and one or more feedback devices, wherein a combination of the one or more sensors and the one or more feedback devices provides sensory relief from distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, to a user/wearer in need thereof.
  • the wearable device is an eyeglass frame.
  • the one or more sensors are connected to the eyeglass frame.
  • the one or more feedback devices are connected to the eyeglass frame.
  • the eyeglass frame comprises a rim, two earpieces and hinges connecting the earpieces to the rim.
  • the one or more sensors are selected from the group consisting of one or more infrared sensors, one or more auditory transducers, one or more galvanic skin sensors, one or more inertial movement units, or combinations thereof.
  • the infrared sensor is surface-mounted on an inner side of the wearable device.
  • the infrared sensor is arranged to be incident on a right eye, a left eye or both eyes of a user.
  • the auditory transducer is a subminiature microphone.
  • the subminiature microphone is surface-mounted on an outer side of the wearable device.
  • the wearable device comprises at least two auditory transducers, wherein a first auditory transducer is arranged at an angle of about 110° to a second auditory transducer.
  • the galvanic skin sensor is surface-mounted on an inner side of the wearable device, and wherein the galvanic skin sensor is in direct contact with skin of a user.
  • the inertial movement unit is internally -mounted on an inner-side of the wearable device.
  • the one or more feedback devices are selected from one or more haptic drivers, one or more bone conduction transducers, or combinations thereof.
  • the haptic drive is internally mounted on an inner side of the wearable device. [0118] In some implementations, the haptic drive is internally mounted on an inner side of the wearable device and behind the inertial movement unit.
  • the haptic drive provides a vibration pattern in response to a sensory input stimulus selected from eye tracking, pupillometry, auditory cues, interoceptive awareness, physical movement, variations in body or ambient temperature, pulse rate, respiration, or combinations thereof.
  • the stereophonic bone conduction transducer is surface-mounted on an inner side of the wearable device, and the stereophonic bone conduction transducer is in direct contact with a user’s skull.
  • the stereophonic bone conduction transducer provides an auditory tone, a pre-recorded auditory guidance, real-time filtering, or combinations thereof, in response to a sensory input stimulus selected from eye tracking, pupillometry, auditory cues, interoceptive awareness, physical movement, variations in body or ambient temperature, pulse rate, respiration, or combinations thereof.
  • the wearable device further comprises an optional wireless or wired hearing device.
  • the wearable device further comprises an intervention means to providing relief to a user from the distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, the intervention means selected from an alert intervention, a filter intervention, a guidance intervention, or a combination thereof.
  • the wearable device further comprises a power switch.
  • the power switch can be located at a left side of the wearable device or a right side of the wearable device.
  • the power switch can be a recessed power switch.
  • a non-transitory computer-readable medium has executable instructions stored thereon that, when executed by a processor, cause a wearable device to perform operations comprising: connecting the wearable device to a datastore that stores one or more sensory thresholds and one or more sensory resolutions specific to a user, the one or more sensory thresholds selected from auditory, visual or physiological/psychophysiological sensory thresholds, and the one or more sensory resolutions selected from auditory, visual, or physiological/psychophysiological sensory resolutions; recording, via one or more sensors, a sensory input stimulus to the user; comparing the sensory input stimulus recorded by the one or more sensors with one or more sensory thresholds to obtain a sensory resolution for the user; and transmitting the sensory resolution to the user.
  • the operations further comprise: communicatively coupling to an loT device providing the sensory input stimulus to the user; and transmitting the sensory resolution to the user, comprises: after communicatively coupling to the loT device, controlling the loT device to transmit the sensory resolution.
  • the loT device comprises a networked lighting device; and controlling the loT device to transmit the sensory resolution, comprises: controlling a brightness or color output of the networked lighting device.
  • the loT device comprises a networked speaker; and controlling the loT device to transmit the sensory resolution, comprises: controlling a volume, an equalization setting, or a channel balance of the networked speaker.
  • comparing the sensory input stimulus recorded by the one or more sensors with the one or more user-specific sensory thresholds to obtain the sensory resolution for the user comprises: inputting the sensory input stimulus and the one or more user-specific sensory thresholds into a trained model to automatically determine the sensory resolution for the user.
  • the operations further comprise determining the one or more sensory thresholds and the one or more sensory resolutions by: presenting multiple selectable templates to the user, each of the templates providing an indication of whether the user is visually sensitive, sonically sensitive, or interoceptively sensitive, and each of the templates associated with corresponding one or more thresholds and one or more sensory resolutions; and receiving data corresponding to input by the user selecting one of the templates.
  • determining the one or more user-specific sensory thresholds and the one or more user-specific sensory resolutions further comprises: receiving additional data corresponding to additional user input selecting preferences, the preferences comprising audio preferences, visual preferences, physiological/psychophysiological preferences, alert preferences, guidance preferences, or intervention preferences; and in response to receiving the additional data, modifying the one or more thresholds and one or more sensory resolutions of the selected template to derive the one or more user-specific sensory thresholds and the one or more user-specific sensory resolutions.
  • FIG. 1 is a schematic representation of a wearable device, in accordance with some implementations of the disclosure.
  • FIG. 2 is a graphical representation of sensitivities across three modalities — visual, aural and anxiety — as observed in Pre-Trial Battery Examination (PTBE), as described herein.
  • PTBE Pre-Trial Battery Examination
  • FIG. 3 is a graphical representation of interest in a wearable device among autism spectrum condition (ASC) participants in PTBE.
  • ASC autism spectrum condition
  • FIG. 4 is a flowchart of a standard study protocol of Sustained Attention to Response Task (SART) testing.
  • FIG. 5 is a flowchart of a standard Wizard of Oz (Wizard of Oz) study protocol.
  • FIG. 6 is a flowchart of the SART/WoZ study protocol, in accordance with some implementations of the disclosure.
  • FIG. 7 is a graphical representation of recruitment scores of study participants for the wearable device studies.
  • FIGs. 8A to 8C are graphical representations of the Errors of Commission (EOC) of the full cohort of participants in the SART/WoZ study described herein.
  • FIG. 8A shows the EOC from baseline to baseline.
  • FIG. 8B shows the EOC intervention effect.
  • FIG. 8C shows the lasting effect of EOC.
  • FIGs. 9A to 9C are graphical representations of EOC as it relates to Response Time (RT) of the full cohort of participants in the SART/WoZ study described herein.
  • FIG. 9A shows the EOC vs RT from starting baseline to final baseline.
  • FIG. 9B shows the EOC vs RT intervention effect.
  • FIG. 9C shows the lasting effect of EOC vs RT.
  • FIGs. lOA to 10C are graphical representations of EOC grouped by study participants.
  • FIGs. HA to 11C are graphical representations of EOC vs RT grouped by study participants.
  • FIGs. 12A to 12C are graphical representations of the Errors of Omission (EOO) of the full cohort of participants in the SART/WoZ study described herein.
  • FIG. 12A shows the EOO from starting baseline to final baseline.
  • FIG. 12B shows the EOO intervention effect.
  • FIG. 12C shows the lasting effect of EOO.
  • FIGs. 13A to 13C are graphical representations of EOO as it relates to RT of the full cohort of participants in the SART/WoZ study described herein.
  • FIG. 13A shows the EOO vs RT from starting baseline to final baseline.
  • FIG. 13B shows the EOO vs RT intervention effect.
  • FIG. 13C shows the lasting effect of EOO vs RT.
  • FIGs. 14A to 14C are graphical representations of EOO grouped by study participants.
  • FIGs. 15A to 15C are graphical representations of EOO vs RT grouped by study participants.
  • FIG. 16 is a block diagram of components of a wearable device, in accordance with some implementations of the disclosure.
  • FIG. 17 is a block diagram of additional microprocessor details (ARM processor) of a wearable device, in accordance with some implementations of the disclosure.
  • ARM processor microprocessor details
  • FIG. 18 is a flowchart of the various components of the study variables, in accordance with some implementations of the disclosure.
  • FIG. 19 depicts a wearable device system including a wearable device in communication with a mobile device and a datastore, in accordance with some implementations of the disclosure.
  • FIG. 20 shows an operational flow diagram depicting an example method for initializing and iteratively updating one or more sensory thresholds and one or more interventions associated with a specific user, in accordance with some implementations of the disclosure.
  • FIG. 21 depicts a wearable device system including a wearable device in communication with a mobile device that controls an loT device with a speaker, in accordance with some implementations of the disclosure.
  • FIG. 22 depicts a wearable device system including a wearable device in communication with a mobile device that controls an loT device with a light emitting device, in accordance with some implementations of the disclosure.
  • FIG. 23 depicts an example wearable device that can be utilized to provide visual interventions, in accordance with some implementations of the disclosure.
  • FIG. 24A depicts interventions that can be delivered using a real-time optical enhancement algorithm, the interventions including haptic alerts, tone alerts guidance, and an eraser effect, in accordance with some implementations of the disclosure.
  • FIG. 24B depicts interventions that can be delivered using a real-time optical enhancement algorithm, the interventions including a text alert, a blur effect, and a cover-up effect, in accordance with some implementations of the disclosure.
  • FIG. 24C depicts interventions that can be delivered using a real-time optical enhancement algorithm, the interventions including color balance, a contrast effect, and an enhancement effect, in accordance with some implementations of the disclosure.
  • FIG. 25 depicts one particular example of a workflow that uses a real-time optical enhancement algorithm to provide interventions, in real-time, in a scenario where there is a distracting visual source, in accordance with some implementations of the disclosure.
  • FIG. 26 depicts a sensitivity mental health distractibility model, in accordance with some implementations of the disclosure.
  • FIG. 27 is a flowchart depicting a design/method of the PPI study described herein.
  • FIG. 28 depicts a word cloud derived from alternative, autistic-voiced expressions during the PPI study described herein.
  • FIG. 29 depicts the mean distribution of anxiety and distractibility scores for diagnostic groups across demographic variable for the PPI study described herein.
  • FIG. 30 depicts non-autistic mediation models, in accordance with some implementations of the disclosure.
  • FIG. 31 depicts autistic mediation models, in accordance with some implementations of the disclosure.
  • FIG. 32 depicts an autistic mediation model predicting distractibility from auditory via fatigue, in accordance with some implementations of the disclosure.
  • FIG. 33 depicts an autistic mediation model predicting distractibility from physiology via fatigue, in accordance with some implementations of the disclosure.
  • FIG. 34A shows summary results of the PPI study described herein.
  • FIG. 34B shows summary results of the PPI study described herein.
  • FIG. 35A shows summary results of the SART/WOz clinical study described herein.
  • FIG. 35B shows summary results of the SART/WOz clinical study described herein.
  • FIG. 36A is an operational flow diagram illustrating an example method for initializing and iteratively updating one or more sensory thresholds and one or more mediations associated with a specific user, in accordance with some implementations of the disclosure.
  • FIG. 36B is an operational flow diagram illustrating an example method for predicting whether a user is neurodiverse (e.g., autistic) or neurotypical, in accordance with some implementations of the disclosure.
  • neurodiverse e.g., autistic
  • FIG. 36C is an operational flow diagram illustrating an example method for predicting whether a user is neurodiverse (e.g., autistic) or neurotypical, in accordance with some implementations of the disclosure.
  • neurodiverse e.g., autistic
  • FIG. 37 illustrates an example system architecture/topology for implementing fog data processing, in accordance with some implementations of the disclosure.
  • FIG. 38A depicts a particular example of a wearable system architecture, including data flows, that leverages fog and edge computing, in accordance with some implementations of the disclosure.
  • FIG. 38B is a flow diagram illustrating operations that are performed by the system of FIG. 38 A, in accordance with some implementations of the disclosure.
  • FIG. 39 is a high-level flowchart of an Open Learner Model (OLM) framework, in accordance with some implementations of the disclosure.
  • OLM Open Learner Model
  • FIG. 40 depicts a table of the OLM described herein, the table describing what is available.
  • FIG. 41 depicts a flowchart of the OLM described herein, the flowchart depicting what is available.
  • FIG. 42 depicts a table of the OLM described herein, the table describing how the model is presented to stakeholders.
  • FIG. 43 depicts a flowchart of the OLM described herein, the flowchart depicting how the model is presented to stakeholders.
  • FIG. 44 depicts a table of the OLM described herein, the table describing who controls access over others.
  • FIG. 45 depicts a flowchart of the OLM described herein, the flowchart depicting who controls access over others.
  • FIG. 46 depicts a system that implements an augmented reality-based multimodal learning analytic framework, in accordance with some implementations of the disclosure.
  • FIG. 47 illustrates one example of a fusion-based, deep learning model, in accordance with some implementations of the disclosure.
  • FIG. 48 is a high level flow diagram conceptually illustrating the operation of a multi-sensory assistive wearable system, in accordance with some implementations of the disclosure.
  • any apparatus, device or product described herein is intended to encompass apparatus, device or products which consist of, consist essentially of, as well as comprise, the various constituents/components identified herein, unless explicitly indicated to the contrary.
  • variable can be equal to any value(s) within that range, as well as any and all sub-ranges encompassed by the broader range.
  • the variable can be equal to any integer value or values within the numerical range, including the end-points of the range.
  • a variable which is described as having values between 0 and 10 can be 0, 4, 2-6, 2.75, 3.19 - 4.47, etc.
  • the term “alert intervention” can include: in the event of an ecological and/or physiological (e.g., psychophysiological) threshold’s activation that corresponds to a wearer’s preferences, a signal is delivered to: (i) a haptic driver that provides a gentle, tactile vibration pattern to convey information to the wearer that focus, anxiety, fatigue or related characteristics require their attention; and/or (ii) a bone conduction transducer that delivers an auditory/sonic message (e.g., pre-recorded text-to-speech, beep tone, etc.) reinforcing the haptic with an aural intervention and set of instructions.
  • an auditory/sonic message e.g., pre-recorded text-to-speech, beep tone, etc.
  • the term “filter intervention” can include: in the event of an ecological and/or physiological (e.g., psychophysiological) threshold’s activation that corresponds to a wearer’s preferences and requires auditory or optical filtering, performing audio signal processing or optical signal processing.
  • Digital audio signal processing can deliver real-time and low-latency audio signals that include corrected amplitude (compression, expansion), frequency (dynamic, shelving, low/hi-cut, and parametric equalization), spatial realignment (reposition, stereo to mono) and/or phase correction (time delay, comb filtering, linear phase alignment).
  • the filter invention can be delivered to a bone conduction transducer.
  • the filter invention can be delivered to optional wireless or wired hearing devices, including but not limited to earbuds, earphones, headphones, and the like.
  • the term “guidance intervention” can include an intervention similar to an alert intervention, where the guidance can be provided by way of step-by-step instructions for re-alignment of focus, head sway, pupillary activity, pulse, temperature, respiration, anxiety, and fatigue coaching.
  • step-by-step instructions for re-alignment of focus, head sway, pupillary activity, pulse, temperature, respiration, anxiety, and fatigue coaching.
  • the term “combination intervention” can include as follows: an intervention that can be selected by the wearer, which can be a combination of alert, filter and guidance interventions, and which are provided depending upon the triggering mechanism. For example, only sonic disturbances can be addressed through filter intervention, while all other issues (attentional-focus, anxiety, fatigue, and the like) can be intervened through haptic, text-to-speech alerts, long-form step-by-step guidance, and the like.
  • errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device
  • errors of omission are a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device
  • response time is intended to include the time taken by a participant to respond to a sensory cue and/or an alert, filter and/or guidance intervention.
  • Response Time may also be interchangeably referred to as Reaction Time and is defined as the amount of time between when a participant perceives a sensory cue and when the participant responds to said sensory cue.
  • Response Time or Reaction Time is the ability to detect, process, and respond to a stimulus.
  • processing such as, for example, any processing that can include filtering of an audio signal and/or an optical signal that is presented to a user
  • real-time is intended to refer to processing and/or filtering the signal with a minimal latency after the original audio signal and/or optical signal occurs.
  • the latency can be a non-zero value of about 500 milliseconds(ms) or less, about 250 ms or less, about 200 ms or less, about 150 ms or less, about 100 ms or less, about 90 ms or less, about 80 ms or less, about 70 ms or less, about 60 ms or less, about 50 ms or less, about 40 ms or less, about 30 ms or less, about 20 ms or less, or about 10 ms or less, ranges and/or combinations thereof and the like.
  • the minimum latency can be subject to system and hardware and software limitations, including communication protocol latency, digital signal processing latency, electrical signal processing latency, combinations thereof and the like. In some instances, real-time filtering of an audio signal and/or an optical signal can be perceived by a user as being immediate, instantaneous or nearly immediate and/or instantaneous.
  • ASC Autism Spectrum Condition
  • RRBIs include hyper-, seeking- and/or hypo-reactivity to sensory input along with attainably unusual interests in sensory aspects of the environment and physiological/psychophysiological responses to visuals, textures, smells, touch, and sounds.
  • ASC populations increase exponentially over time, an ever-expanding social policy chasm proliferates, whereby an autistic individual's smooth transition into the fabric of daily life is often compromised.
  • Experts identify this as a gap stemming from either: (i) stunted public/govemment support for neurodiverse individuals; ii) tensions between the autism community and society; and iii) limited support for later-life educational/vocational pathways.
  • the negative effects of policy- related factors are a consequence resulting in societal costs that have a potential to become still more significant and possibly irremediable.
  • This application provides various interventions to alter, redirect and/or attenuate disruptive stimuli.
  • described herein are systems, devices and methods to determine whether distractions exist, which can be exacerbated at school and at work, and provide interventions to compensate for such distractions, thereby lessening anxiety for neurotypical and neurodiverse individuals, and providing sensory relief
  • This application aspires to help individuals leam, adapt, and internalize how best to respond to encroaching ecological stimuli and resulting physiological/psychophysiological responses.
  • Wearables as described herein, may, through repetitive processes observed and experienced by users, pave the way for a call and response process that may eventually transfer directly from a machine or system to the person, thus embedding guidance for similarly reoccurring/future scenarios.
  • An autistic individual for example, might watch, experience, and leam precisely how an Artificial Intelligence/Cognitive Enhancement system detects, filters and coaches herself when confronted with an undesirable sensory stimulus.
  • One embodiment is directed to a system for providing sensory relief from distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, to a user in need thereof, the system comprising: (i.) a wearable device; (ii) a database of one or more user-specific sensory thresholds selected from auditory, visual and physiological/psychophysiological sensory thresholds, one or more user-specific sensory resolutions selected from auditory, visual and physiological/psychophysiological sensory resolutions, or combinations thereof; (iii) an activation means for connecting the wearable device and the database; (iv) one or more sensors for recording a sensory input stimulus to the user; (v) a comparing means for comparing the sensory input stimulus recorded by the one or more sensors with the database of one or more user-specific sensory thresholds to obtain a sensory resolution for the user; (vi) one or more feedback devices for transmitting the sensory resolution to the user; and (vii) a user-specific intervention means for providing relief to the user from the distractibility, inattention, anxiety, fatigue, sensory issues, or
  • Another embodiment is directed to a method of providing sensory relief from distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, to a user in need thereof, the method comprising: (i) creating a database of one or more user-specific sensory thresholds selected from auditory, visual and physiological/psychophysiological sensory thresholds, one or more user-specific sensory resolutions selected from auditory, visual and physiological/psychophysiological sensory resolution, or combinations thereof; (ii) attaching a wearable device to the user, wherein the wearable device comprises one or more sensors and one or more feedback devices; (iii) activating and connecting the wearable device to the database; (iv) recording a sensory input stimulus to the user via the one or more sensors; (v) comparing the sensory input stimulus with the database of one or more userspecific sensory thresholds; (vi) selecting an appropriate user-specific sensory resolution from the database; (vi) delivering the user-specific sensory resolution to the user via the one or more feedback devices; and (vii) providing a user-specific intervention to provide relief to the
  • the wearable device can be an eyeglass frame.
  • One or more sensors and/or one or more feedback devices can be connected to the eyeglass frame.
  • the eyeglass frame may comprise a rim, two earpieces and hinges connecting the earpieces to the rim.
  • the wearable device may include jewelry, smart clothing, and accessories, including but not limited to rings, sensor woven fabrics, wristbands, watches, pins, hearing aid, assistive devices, medical devices, virtual, augmented, and mixed reality (VR/AR/MR) headsets, and the like.
  • the wearable device may have the ability to coordinate with mobile and/or network devices for alert, filter, and guidance interventions, and may include sensors and feedback devices in various combinations.
  • the one or more sensors can be selected from one or more infrared sensors, one or more auditory sensors, one or more galvanic skin sensors, one or more inertial movement units, or combinations thereof.
  • the infrared sensor can be surface-mounted on an inner side of the wearable device.
  • the infrared sensor can be arranged to be incident on a right eye, a left eye or both eyes of a user.
  • the one or more feedback devices can be selected from one or more haptic drivers, one or more bone conduction transducers, or combinations thereof.
  • the wearable device may further comprise a wireless or wired hearing device.
  • the sensory input stimulus can be selected from an auditory input, a visual input, a physiological/psychophysiological input or combinations thereof.
  • the sensory input stimulus can be measured by evaluating one or more parameters selected from eye tracking, pupillometry, auditory cues, interoceptive awareness, physical movement, variations in body or ambient temperature, pulse rate, respiration, or combinations thereof.
  • the sensory resolution can be provided by one or more alerts selected from a visual alert, an auditory alert, a physiological/psychophysiological alert, a verbal alert or combinations thereof.
  • the activation means can be a power switch located on the wearable device.
  • the power switch can be located at a left side of the wearable device and/or at a right side of the wearable device.
  • the power switch can be a recessed power switch.
  • power may be supplied when in stand-by mode from a user interface component, including but not limited to mobile phones, laptops, tablets, desktop computers, and the like, and any user interface known in the field can be used without limitation.
  • the activation means may include a power switch or power source that can be activated remotely (i. e. , when not in proximity of a user).
  • the activation means may be triggered by the wearable’s accelerometer, pupillary and head sway sensors, and the like.
  • the accelerometer senses when the wearer (and wearable) is idle.
  • the unit can be in a low-power or power-off mode, and when the wearable is engaged (e.g., the wearable is lifted from a surface, move or agitated), such engagement is recognized by the accelerometer, which switches the wearable into a power-on mode.
  • the power management system includes the ability to place the unit into a battery conservation mode (e.g., low-power mode). If, for example, a wearer was to shut their eyes whilst resting with a wearable “in place”, the sensors would react to a novel movement and immediately return the system into a powered-on state when/if the user was to eventually arise from a period of rest, and the like.
  • a battery conservation mode e.g., low-power mode
  • an activation means may include a power-on activity programmed from a biopotential analogue front end (AFE), which includes galvanic skin sensor response applications including perspiration, heart rate, blood pressure, temperature, and the like, all of which can trigger an activation of the wearable device.
  • AFE biopotential analogue front end
  • the wearable device’s activation can be fully accessed by any type of network device / protocol because of its loT connectivity, which enables communication, activation, and the like, of the wearable device.
  • a database can be stored in a storage device.
  • the storage device can be selected from a fixed or movable computer system, a portable wireless device, a smartphone, a tablet, or combinations thereof.
  • the database can be stored locally on or in the wearable device.
  • the databased can be stored remotely, including but not limited to cloud-based systems, secured datacenters behind DMZ, and the like, and the database can be in encrypted and decrypted communication with the secured wearable device and its data.
  • a response time for autistic users increases by at least about 0.5% to about 5%, about 1% to about 4.5%, about 1.5% to about 4%, about 2% to about 3.5%, and preferably about 3% after alert intervention and accuracy increases by at least about 10% to about 50%, about 15% to about 40%, about 20% to about 30%, and preferably about 26% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • a response time for neurotypical users increases by at least about 0.5% to about 50%, about 5% to about 40%, about 10% to about 30%, about 15% to about 20%, and preferably about 18% after alert intervention and accuracy increases by at least about 0.01% to about 5%, about 0.05% to about 4%, about 1% to about 3%, and preferably about 2.0% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • a response time for autistic users increases by at least about 0.5% to about 50%, about 1% to about 40%, about 10% to about 30%, and preferably about 20% after guidance intervention and accuracy increases by at least about 0.5% to about 30%, about 1.0% to about 20%, about 5% to about 15%, and preferably about 10% from baseline for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • a response time for autistic users increases by at least about 0.01% to about 5%, about 0.05% to about 4%, about 1% to about 3%, and preferably about 2% after guidance intervention and accuracy increases by at least about 10% to about 50%, about 15% to about 45%, about 20% to about 40%, and preferably about 30% from baseline for errors of omission, wherein the errors of omission is a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • a response time for autistic users increases by at least about 0.5% to about 30%, about 1.0% to about 20%, about 5% to about 15%, and preferably about 10% from baseline after filter intervention for errors of omission, wherein the errors of omission is a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • a response time for autistic users is at least about 0.5% to about 30%, about 1.0% to about 25%, about 5% to about 20%, and preferably about 15% faster than neurotypical users after filter intervention for errors of omission, wherein the errors of omission are a measure of the user’s failure to take appropriate action when a prompt is not received from the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • a response time for autistic users is at least about 0.5% to about 50%, about 1% to about 40%, about 10% to about 30%, and preferably about 20% faster after guidance intervention and accuracy is at least about 0.5% to about 30%, about 1.0% to about 20%, about 5% to about 15%, and preferably about 8% higher than neurotypical users for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • accuracy for autistic users is at least about 0.5% to about 50%, about 1% to about 4%, about 10% to about 30%, and preferably about 25% higher than neurotypical users after alert intervention for errors of commission, wherein the errors of commission are a measure of the user’s failure to inhibit a response when prompted by the feedback device.
  • a numerical value within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges.
  • an auditory transducer can be a subminiature microphone.
  • the subminiature microphone may preferably be surface-mounted on an outer side of the wearable device.
  • a wearable device may include at least two auditory transducers, and the arrangement of the first and second auditory transducers can be one that is known in the art, including but not limited to the first and second auditory transducers being arranged at an angle ranging from about 45° to about 135°, about 55° to about 130°, about 65° to about 125°, about 75° to about 120°, about 85° to about 120°, about 95° to about 115°, about 100°, about 110°, and the like.
  • the numerical value of any specific angle within these ranges can be equal to any integer value or values within any of these ranges, including the end-points of these ranges can be.
  • a galvanic skin sensor can be surfacemounted on an inner side of the wearable device, and the galvanic skin sensor can be in direct contact with skin of a user.
  • the inner side of the wearable device can be a side facing the skin or substantially facing the skin.
  • an inertial movement unit may preferably be internally -mounted on an inner-side of the wearable device.
  • the one or more feedback devices can be selected from one or more haptic drivers, one or more bone conduction transducers, or combinations thereof.
  • the haptic drive can be internally mounted on an inner side of the wearable device.
  • the haptic drive can be internally mounted on an inner side of the wearable device and behind the inertial movement unit.
  • the haptic drive provides a vibration pattern in response to a sensory input stimulus selected from eye tracking, pupillometry, auditory cues, interoceptive awareness, physical movement, variations in body or ambient temperature, pulse rate, respiration, or combinations thereof.
  • the feedback device may also include a heads-up visual component, or other feedback devices that provide pupillary projection, distracting visual blurring, removal, squelching, recoloring, or combinations thereof.
  • the stereophonic bone conduction transducer can be surface-mounted on an inner side of the wearable device, and the stereophonic bone conduction transducer can be in direct contact with a user’s skull.
  • the stereophonic bone conduction transducer provides an auditory tone, a pre-recorded auditory guidance, real-time filtering, or combinations thereof, in response to a sensory input stimulus selected from eye tracking, pupillometry, auditory cues, interoceptive awareness, physical movement, variations in body and ambient temperature, pulse rate, respiration, or combinations thereof.
  • the wearable device may further include an intervention means to providing relief to a user from the distractibility, inattention, anxiety, fatigue, sensory issues, or combinations thereof, wherein the intervention means is selected from an alert intervention, a filter intervention, a guidance intervention, or a combination thereof.
  • the intervention means and the feedback means can be the same or different.
  • FIG. 16 Various possible intervention means available to the user and delivered by the wearable device are illustrated in the block diagram of FIG. 16. As illustrated in FIG. 16, following sensor(s) data stream delivery and microprocessor 312 comparison between ecological/environmental and physiological/psychophysiological thresholds to real-time data, those events deemed subject for interventional processing can be delivered to one of two discrete (or simultaneous) components: a haptic driver 313 or a bone conduction transducer 314. Pending a wearer’s previously defined preferences (stored in the microprocessor), one of four interventional strategies can be invoked: alert, filter, guidance, or combination.
  • Alert intervention In the event of an ecological and/or physiological/psychophysiological threshold’s activation that corresponds to a wearer’s preferences, a signal is delivered to: (i) the haptic driver that provides a gentle, tactile vibration pattern to convey information to the wearer that focus, anxiety, fatigue or related characteristics require their attention; and/or (ii) the bone conduction transducer(s) that deliver an auditory/sonic message (e.g., pre-recorded text-to-speech, beep tone, etc.) reinforcing the haptic with an aural intervention and set of instructions.
  • an auditory/sonic message e.g., pre-recorded text-to-speech, beep tone, etc.
  • Filter intervention In the event of an ecological and/or physiological/psychophysiological threshold’s activation that corresponds to a wearer’s preferences and requires auditory filtering, digital audio signal processing delivers real-time and low-latency audio signals that include corrected amplitude (compression, expansion), frequency (dynamic, shelving, low/hi-cut, and parametric equalisation), spatial realignment (reposition, stereo to mono) and/or phase correction (time delay, comb filtering, linear phase alignment). Though typically delivered to bone conduction transducers, these can be delivered to optional wireless or wired hearing devices, including but not limited to earbuds, earphones, headphones, and the like.
  • Guidance intervention Similar to alert intervention, the guidance by way of step-by-step instructions for re-alignment in focus, head sway, pupillary activity, anxiety, and fatigue coaching is provided to a wearer. These pre-recorded, text-to-speech audio streams are delivered to the bone conduction systems, which provide step-by-step instructional intervention both privately and unobtrusively.
  • Combination intervention Selectable by the wearer, a combination of alert, filter and guidance interventions are provided depending upon the triggering mechanism. For example, only sonic disturbances are addressed through filter intervention, while all other issues (attentional-focus, anxiety, etc.) can be intervened through haptic, text-to-speech alerts and long-form step-by-step guidance.
  • the wearable device may further include a power switch.
  • the power switch can be located at a left side of the wearable device and/or a right side of the wearable device.
  • the power switch can be a recessed power switch.
  • the wearable device may have a structure illustrated in FIG. 1.
  • the wearable device 10 can be in the form of an eyeglass frame including a rim 109, left and right earpieces, each having a temple portion 106 and temple tip 108 and screws 103 and hinges 104 connecting the earpieces to the rim 109.
  • the frame may further include lenses 101, a nose pad 102, end pieces 107, and a bridge 105 connecting left- and right-sides of the frame.
  • the wearable device may have one or more sensors connected to the frame, including infrared pupillometry sensors 204, galvanic skin sensors 205, inertial movement units 206, wireless transceiver and A/D multiplexers 208, microphones 201, and the like.
  • the wearable device 10 may also include one or more feedback devices connected to the frame, including haptic drivers 203, bone conduction transducers 202, and the like.
  • the wearable device 10 may further include an optional wireless or wired hearing device 209, and a power switch (not shown) and/or a rechargeable power source 207.
  • the wearable device is depicted as an eyeglass frame in FIG. 1, it should be appreciated that the wearable device can be implemented using a different type of head mount such as a visor or helmet.
  • Other exemplary embodiments of the wearable device can include, but are not limited to, wrist worn devices, bone conduction devices, and the like, and any wearable device known in the field and adaptable to the method described herein can be used, and any of which may work in conjunction with a user interface described herein.
  • the wearable device can be implemented as a combination of devices (e.g., wearable eyeglasses, ring, wrist-worn, clothing/textile, and watch).
  • the wearable device can be communicatively coupled to a mobile device (e.g., smartphone and/or other smart device) that controls operations of, works in concert with, and/or provides a user interface for change settings of the wearable device.
  • a mobile device e.g., smartphone and/or other smart device
  • FIG. 19 depicts a wearable device system including a wearable device 10 in communication with a mobile device 20, and a datastore 30.
  • the wearable device 10 communicates with mobile device 20 over a wireless communication network.
  • the wireless communication network can be any suitable network that enables communications between the devices.
  • the wireless communication network can be an ad-hoc network such as a WiFi network, a Bluetooth network, and/or a network using some other communication protocol.
  • the wearable device 10 can be tethered to mobile device 20.
  • the mobile device 20 processes sensor data collected by one or more sensors of wearable device 10.
  • the mobile device 20 can determine, based on the processed sensor data, one or more interventions to be applied using the wearable device 10 and/or some other device. The determination can be based on one or more sensory thresholds 31 specific to a user wearing the wearable device 10.
  • the interventions that are applied can be based on one or more user-specific sensory resolutions 32 specific to the user.
  • the datastore storing thresholds 31 and resolutions 32 is illustrated in this example as being separate from wearable device 10 and mobile device 20, in other implementations the datastore 30 can be incorporated within wearable device 10 and/or mobile device 20.
  • the user can initiate personalization of the wearable device by identifying individual sound, visual and physiological/psychophysiological thresholds using software integrated in the wearable device.
  • Personalization can identify unique sensory, attentional-focus and anxiety/fatigue producing cues that a user finds distracting particularly in educational, employment, social, and typical daily activities, and can be derived from the Participant Public Information (PPI) study described herein.
  • PPI Participant Public Information
  • the user-specific thresholds are used to customize subsequent alerts, filters, and guidance experienced by the user when wearing the wearable device.
  • the thresholds are transmitted to the wearable device.
  • the personalization thresholds may be updated over time (e.g., periodically or dynamically) as the user adapts to stimuli or is presented with new stimuli.
  • the device may be configured via a mobile application (app), web-based application or other web-based interface (e.g., website).
  • apps a mobile application
  • web-based application or other web-based interface e.g., website
  • the user can be presented with a graphical user interface or other user interface via the wearable device or via a smartphone or other device communicatively coupled to the wearable device.
  • the wearable device or other device can include a processor that executes instructions that cause the device to present (e.g., display) selectable controls or choices to the user that are used to refine a set of thresholds, alerts, filters, and/or guidance in discrete or combined formats.
  • the personalization process can be conducted by the wearer of the device, a healthcare provider, or a caretaker of the user.
  • the personalization process can be conducted by running an application instance on the wearable device or other device and receiving data corresponding to input from the wearer, health provider, or caretaker making selections (e.g., telemetry /biotelemetry).
  • selections e.g., telemetry /biotelemetry.
  • different user interfaces and options can be presented depending on whether personalization is conducted by the wearer, healthcare provider, or caretaker.
  • a datastore associated with the wearable device may pre-store initialization templates that correspond to a particular set of thresholds (e.g., sound, visual, and/or physiological/psychophysiological) and/or alerts, filters, and/or guidance.
  • thresholds e.g., sound, visual, and/or physiological/psychophysiological
  • alerts, filters, and/or guidance For example, templates corresponding to predominantly sonically sensitive wearers, predominantly visually sensitive wearers, predominantly interoceptive sensitive wears, combination wearers, and the like can be preconfigured and stored by the system.
  • the wearer can select one of the templates (e.g., the user is predominantly visually sensitive), and the configured parameters (e.g., thresholds, alerts, filters, and/or guidance) for the selected template can be further customized in response to additional user input.
  • the additional user input can include responses to questions, or a selection of preferences as further discussed below.
  • each of the templates can be associated with a trained model that given a set of inputs (e.g., sensor readings from the wearable device, user thresholds, etc.) generates one or more outputs (e.g., alerts, filters, guidance, sonic feedback, visual feedback, haptic feedback) experienced by the user.
  • the model can be trained and tested with anonymized historical data associated with users to predict appropriate outputs given sensory inputs and thresholds. Supervised learning, semi-supervised learning, or unsupervised learning can be utilized to build the model.
  • parameters of the model e.g., weights of input variables
  • the user can be presented with selectable preferences and/or answers to questions.
  • the personalization process may present the user with selectable choices relating to demographics (e.g., gender, age, education level, handedness, etc.) and sensitivities (e.g., audio preferences, visual preferences, physiological/psychophysiological preferences, alert preferences, guidance preferences, intervention ranking preferences, and the like).
  • demographics e.g., gender, age, education level, handedness, etc.
  • sensitivities e.g., audio preferences, visual preferences, physiological/psychophysiological preferences, alert preferences, guidance preferences, intervention ranking preferences, and the like.
  • a particular set of thresholds e.g., sound, visual, and/or physiological/psychophysiological
  • alerts, filters, and/or guidance may be customized for the user and stored.
  • the system can be configured to perform digital signal processing of audio signals before audio is played to the user to adjust the energy of different frequency ranges (e.g., bass, mid-range, treble, etc.) within the audible frequency band (e.g., 20Hz to 20,000 Hz), the audio channels that emit sound, or other characteristics of audio.
  • the user can specify a preference for filtering (e.g., enhancing, removing, or otherwise altering) low-range sounds, mid-range sounds, high-range sounds, soft sounds, loud sounds, reverberant sounds, surround sounds, etc.
  • a user can specify a preference for receiving alerts of sounds having particular sonic characteristics (e.g., alert for loud, echoing, and/or surround sounds before they occur).
  • a user can prefer that guided sounds have particular characteristics (e.g., soft-spoken words, gentle sounds) when the user becomes anxious, unfocused, or sensitive.
  • the user’s selected preferences and/or answers during personalization can be used to build a model that given a set of inputs (e.g., sensor readings from the wearable device, user thresholds, etc.) generates one or more outputs (e.g., alerts, filters, guidance, sonic feedback, visual feedback, haptic feedback) experienced by the user.
  • a website or mobile app configurator accessed via a user login can generate one or more tolerance scores based on the user’s answers to questions pertaining to visual, auditory, or physiological/psychophysiological stimuli.
  • the one or more tolerance scores can be used to initialize the model.
  • the model can also be initialized, modified and/or monitored by a specialist, healthcare provider, or caretaker.
  • a user can rank the types of interventions. For example, a user can rank and/or specify a preferred type of alert (e.g., beep, haptic, voice, or some combination thereof), a preferred audio filter (e.g., volume (compression, limiting), equalization (tone, EQ), noise reduction, imaging (panning, phase), reverberation (echo), imaging (panning, phase), or some combination thereof), a preferred type of guidance (e.g., encouragement), and the like.
  • a preferred type of alert e.g., beep, haptic, voice, or some combination thereof
  • a preferred audio filter e.g., volume (compression, limiting), equalization (tone, EQ), noise reduction, imaging (panning, phase), reverberation (echo), imaging (panning, phase), or some combination thereof
  • a preferred type of guidance e.g., encouragement
  • one or more sensors of the wearable device can be calibrated during initialization of the device.
  • a user can be presented with an interface for calibrating sensors and/or adjusting sensor parameters. For example, the user can specify whether all or only some sensors are active and/or gather data, adjust sensor sensitivity, or adjust a sensor threshold (e.g., brightness for an optical sensor, loudness for an audio sensor) and what order of implementation they are desired (e.g., alerts first, followed by guidance, followed by filters, etc.).
  • a sensor threshold e.g., brightness for an optical sensor, loudness for an audio sensor
  • validation of the configuration can be conducted by presenting the user with external stimuli, and providing alerts, filters, and/or guidance in accordance with the user-configured thresholds. Depending on the user’s response, additional configuration can be conducted. This validation process can also adjust sensor settings such as sensor sensitivity.
  • the model can be retrained over time based on collected environmental and/or physiological/psychophysiological data. To save on computational resources and/or device battery life, retraining can be performed at night and/or when the system is not in use.
  • FIG. 20 shows an operational flow diagram depicting an example method 400 for initializing and iteratively updating one or more sensory thresholds and one or more interventions associated with a specific user.
  • method 400 can be implemented by one or more processors (e.g., one or more processors of wearable device 10 and/or mobile device 20) of a wearable device system executing instructions stored in one or more computer readable media (e.g., one or more computer readable media of wearable device 10 and/or mobile device 20).
  • Operation 401 includes presenting multiple selectable templates to the user, the multiple templates corresponding to one or more sensory thresholds and one or more interventions.
  • the multiple selectable templates can be presented via a GUI (e.g., using wearable device 10 and/or mobile device 20).
  • Operation 402 includes receiving data corresponding to input by the user selecting one of the templates.
  • the one or more sensory thresholds and one or more interventions associated with the template can be associated with the user.
  • the one or more sensory thresholds and one or more interventions can be stored in a datastore 30 including an identification and/or user profile corresponding to the user.
  • operations 401-402 can be performed during and/or after an initialization process.
  • Operation 403 includes receiving data corresponding to user input selecting preferences.
  • the preferences can comprise audio preferences, visual preferences, physiological/psychophysiological preferences, alert preferences, guidance preferences, and/or intervention preferences.
  • Operation 404 includes in response to receiving additional data corresponding to additional user input selecting preferences, modifying the one or more sensory thresholds and the one or more interventions associated with the user. For example, the datastore thresholds and interventions can be updated. As depicted, operations 403-404 can iterate over time as the user desires to further define the thresholds/interventions and/or as the user develops new preferences.
  • Operation 405 includes collecting sensor data and environmental data while the user wears the wearable device.
  • Operation 406 includes in response to collecting the sensor data and/or environmental data while the user wears the wearable device, modifying the one or more sensory thresholds and the one or more interventions associated with the user.
  • operations 405-406 can iterate over time as the user utilizes the wearable device system to provide sensory relief
  • the frequency with which the one or more sensory thresholds and the one or more interventions are updated in response to newly-collected data can be configurable, system-defined, and/or user-defined. For example, updates can depend on the amount of data that is collected and/or the amount of time that has passed. In some implementations, operations 405-406 can be skipped.
  • the wearable device can be configured to communicate with and/or control loT devices that present stimuli. For example, based on configured thresholds for a user, the wearable device can control the operation of smart devices such as networked hubs, networked lighting devices, networked outlets, alarm systems, networked thermostats, networked sound systems, networked display systems, networked appliances, and other networked devices associated with the user.
  • smart devices such as networked hubs, networked lighting devices, networked outlets, alarm systems, networked thermostats, networked sound systems, networked display systems, networked appliances, and other networked devices associated with the user.
  • the audio output (e.g., loudness and balance) of a networked sound system and/or display output (e.g., brightness, contrast, and color balance) of networked display system can be altered to meet individual sound or visual thresholds.
  • the devices can be linked to an account of the user, which can be configured via an application running on a smartphone (e.g., native home control application) or other device (e.g., mobile device 20).
  • behavior or one or more scenes for an loT device can be preconfigured based on the thresholds associated with the user. The behavior or scenes can be activated when the wearable device detects that it is in the presence (e.g., same room) of the loT device.
  • FIG. 21 depicts a wearable device system including a wearable device 10 in communication with a mobile device 20 that controls an loT device 40 with a speaker 41.
  • FIG. 22 depicts a wearable device system including a wearable device 10 in communication with a mobile device 20 that controls an loT device 50 with a light emitting device 51.
  • wearable device 10 can use one or more sensors to collect a sensory input stimulus. This sensory input stimulus can be transmitted to a mobile device 20 that compares the sensory input stimulus with one or more sensory thresholds specific to the user (e.g., thresholds 31) to determine an intervention to be provided to the user, to provide the user relief from distractibility, inattention, anxiety, fatigue, and/or sensory issues.
  • one or more sensory thresholds specific to the user e.g., thresholds 31
  • the sensory input stimulus can be generated at least in part due to sound emitted by the speaker 41 of the loT device 40.
  • the user can generate a physiological/psychophysiological response to music and/or other sounds being played at a certain frequency and/or range of frequencies by speaker 41.
  • the intervention can include the mobile device 20 controlling loT device 40 to filter, in the frequency domain, an audio signal such that sound output by speaker 41 plays in a frequency that does not induce the same physiological/psychophysiological response in the user.
  • the sensory input stimulus can be generated at least in part due to light emitted by the light emitting device 51 of the loT device 50.
  • the user can experience discomfort when the output light is too bright or too cool (e.g., >4000K) in color temperature. This discomfort can be measured using the sensory input stimulus collected by the one or more sensors of the wearable device 10.
  • the intervention can include the mobile device 20 controlling loT device 50 to filter an optical signal of light device 51 to lower a brightness and/or color temperature of light output by the lighting device 51.
  • the wearable device can include various user interface components, including but not limited to mobile phones, laptops, tablets, desktop computers, and the like, and any user interface known in the field can be used.
  • the wearable device can be synchronized with a smartphone.
  • the wearable device can be configured to accept calls, adjust call volume, present notification sounds or vibrations, present ringtones, etc.
  • the wearable device can be granted access to user contacts, text messages or other instant messages, etc.
  • the intensity of sounds or vibrations, or the pattern of sounds or vibrations, presented via mobile integration can depend on configured thresholds of the user.
  • the initial configuration and personalization of the wearable device can be conducted via an application installed on a smartphone or other device.
  • the wearable device can include one or more network interfaces (e.g., WiFi, Bluetooth, cellular, etc.) for communicating with other networked devices and/or connecting to the Internet.
  • a WiFi interface can enable the wearable device to select and communicatively couple to a local network, which can permit communication with loT devices.
  • Bluetooth can enable pairing between the wearable device and a smartphone or other device.
  • the wearable device can include or communicatively couple to one or more datastores (e.g., memories or other storage device) that are accessed during its operation.
  • Storage can be local, over a network, and/or over the cloud. Storage can maintain a record of user preferences, user performance, trained models, and other data or instructions required to operate the device.
  • a wearable device is operated as described herein.
  • the wearable device can remain in passive mode, i.e., non-operating mode, before it is worn by a user. This can optimize battery life.
  • the wearable device detects and responds to one or more sensory cues selected from a myriad of sensory cues received and detected by one or more sensors located on the wearable device.
  • Such sensory cues can include environmental and physiological/psychophysiological signals, and the like.
  • the wearable device also provides additional and appropriate resolution in response to the sensory cues via alerts, filters, and guidance to the user whenever personalized thresholds for the use are exceeded. Thresholds and interventions can be iteratively set, adjusted, muted, and otherwise cancelled at any time and throughout the use of the wearable device by the user by returning to the computer/application.
  • Various types of sensory cues can be received and detected by the wearable device, including visual, auditory, and physiological/psychophysiological cues, but are not limited thereto.
  • visual distractions can be detected via eye tracking and pupillometry monitored by in infrared sensor that can be surface mounted on an inner side of the wearable device, for example, at an intersection of frame rim/right hinge temple and aimed at an eye of the user, for e.g., the right eye or the left eye or both.
  • auditory distractions and audiometric thresholds can be monitored by subminiature and wired electret microphones that can be surface mounted on an outer side of the end pieces, near the intersection of the frame front and temples.
  • physiological/psychophysiological distractions, interoceptive thresholds and user head sway can be monitored by a galvanic skin sensor that is surface-mounted on an inner side of the left earpiece and in direct contact with the skin just above the user’s neckline and/or an inertial movement unit that is internally mounted on an inner side of the wearable device, and can be located behind an ear piece.
  • a galvanic skin sensor that is surface-mounted on an inner side of the left earpiece and in direct contact with the skin just above the user’s neckline and/or an inertial movement unit that is internally mounted on an inner side of the wearable device, and can be located behind an ear piece.
  • Various types of resolutions can be provided to the user in response to the sensory cues received by the wearable device.
  • the resolutions may include visual, auditory and physiological/psychophysiological resolutions, but are not limited thereto.
  • the visual resolutions can be delivered through a haptic driver that can be internally mounted on an inner side of an ear piece and behind an inertial movement unit intersection of frame rim/right hinge temple.
  • Visual resolutions can be provided via unique vibrations associated with optical distractions when a pupillary or inertial/head sway threshold is detected.
  • the visual alerts can be delivered by a stereophonic bone conduction that can be surface mounted on an inner side of wearable device, for example, at both temples midway between the hinges and temple tips and coming into direct contact with the user’s left and right skull in front of each ear and provides either a beep tone and/or pre-recorded spoken guidance in the event a pupillary or inertial/head sway threshold is detected.
  • a stereophonic bone conduction can be surface mounted on an inner side of wearable device, for example, at both temples midway between the hinges and temple tips and coming into direct contact with the user’s left and right skull in front of each ear and provides either a beep tone and/or pre-recorded spoken guidance in the event a pupillary or inertial/head sway threshold is detected.
  • auditory resolutions can be delivered to the user through a single, haptic driver by providing uniquely coded vibrational alerts in the event a sonic threshold is detected and/or through a bone conduction transducer that provides both beep tone, prerecorded spoken guidance and/or real-time filtering using digital signal processing (DSP) for those distracting, environmental audiometric events (e.g., compression, equalization, noise reduction, spatial panning, limiting, phase adjustment, and gating) when a sonic threshold(s) is/are detected and can be processed according to user’s personalization settings.
  • DSP digital signal processing
  • real-time digital audio streams recorded by microphones connected to the wearable device provide the microprocessor with audio data that undergo system manipulation to achieve a predetermined goal.
  • the DSP produces feedback in the form of altered audio signals (the filtered intervention) that ameliorates volume (amplitude, compression, noise reduction), tonal (equalization), directional (spatial, etc.).
  • guidance may include one or more tonal alerts retrieved from a datastore.
  • the device can be configured to boost certain audible frequencies depending on the user’s age or hearing. For example, the device can boost low, mid, and/or high frequencies depending on the user’s age and/or hearing profile. In some cases, the device can execute instructions to provide a hearing test to generate the hearing profile. A control can be provided to enable or disable sound boosting.
  • physiological/psychophysiological resolutions can be delivered to the user through a haptic driver mentioned above and by providing uniquely coded vibrational alerts in the event a physiological/psychophysiological, anxiety, fatigue or other interoceptive thresholds are detected and/or through a bone conduction transducer, which provides both beep tone, and/or pre-recorded spoken guidance for similar threshold alert and guidance.
  • a haptic driver mentioned above and by providing uniquely coded vibrational alerts in the event a physiological/psychophysiological, anxiety, fatigue or other interoceptive thresholds are detected and/or through a bone conduction transducer, which provides both beep tone, and/or pre-recorded spoken guidance for similar threshold alert and guidance.
  • the various resolution components described herein are merely exemplary, and any suitable components can be used.
  • the wearable device may also include an internally mounted central processing unit, that may further include subminiature printed circuit boards combined with a self-contained connected and rechargeable power source, wireless transceiver and analog/digital multiplexers reside within both earpieces and provide evenly weighted distribution to wearer.
  • the comparing means compares the sensory input stimulus recorded by the one or more sensors with the database of one or more user-specific sensory thresholds to obtain a sensory resolution for a user.
  • the comparing means performs the afore-mentioned functions as follows.
  • the user-specific thresholds can be obtained by having the user complete a decision-tree styled survey (similar in scope to the survey described in the Sustained Attention to Response Test (SART) protocol described herein), and then a microprocessor measures the user-specific thresholds against ecological and physiological/psychophysiological data streams to deliver appropriate intervention assistance.
  • the user-specific thresholds can dynamically change using a machine learning capability.
  • FIG. 16 An exemplary embodiment of how input stimulus is compared to stored data to generate user-specific interventions is illustrated via a block diagram in FIG. 16, but this application is not limited thereto.
  • six components make up the wearable device’s input section and include: an optical module 301; an inertial measurement unit (IMU) 304; an audio sensor 305; a galvanic module 306; a temperature sensor 309; and a biopotential analogue front end (AFE) 310.
  • IMU inertial measurement unit
  • AFE biopotential analogue front end
  • these components deliver both ecological (environmental) and physiological/psychophysiological data to a sensor hub 311 (multiplexer), and the data is processed (typically through wireless, bi-directional communication, though it can be directly connected) with the system’s microprocessor (e.g., ARM Cortex) for rapid analysis and comparison to existing thresholds, characteristics, and user-preferences.
  • the microprocessor 312 e.g., ARM microprocessor
  • the microprocessor 312 delivers the appropriate commands for interventional activities to be processed by those related system components as described herein.
  • the six components are further described as follows.
  • the optical module 301 includes: (i) an inward facing pair of infrared sensors 302 that monitor pupillary response, portending to a user’s focus and attentional lability; and (ii) a single outward facing sensor to determine ecological/environmental cues of a visual nature.
  • a tuned optical AFE 303 provides the appropriate pupillary data stream for processing and simultaneously provides an environmental data stream for image recognition allowing the microprocessor 312 to determine visual environmental cues for which the user is responding.
  • image recognition (whether pupillary response, saccades, computer screen, books, automobile roadways, office/academic surroundings, etc.) rely on a computer vision technique that allows the microprocessor 312 to interpret and categorize what is seen in the visual data stream.
  • This type of image classification (or labelling) is a core task and foundational component in comparing real-time visual input to a library/catalogue of pre-labelled images that are interpreted and then serve as the basis for an intervention, provided that the user’s thresholds are exceeded (or unsurpassed).
  • the IMU 304 measures and reports a body's specific force (in this case, the user’s head/face). It also provides angular rate and orientation using a combination of accelerometers, gyroscopes, and magnetometers to deliver a data stream relating to the user’s head sway and attentional focus, when compared and contrasted to the optical AFE 303 and processed similarly against pre-labelled and classified data.
  • the IMU can include a 3-axis gyroscope/accelerometer.
  • the audio sensor(s) 305 provide environmental data streams of a sonic nature which can be compared to known aural signatures that have been labelled and available for computer micro processing.
  • the aural signatures that reach frequency, amplitude, spatial, time-delay/phase and similar user-selected thresholds could then be delivered for interventional processing.
  • Both the galvanic module 306 and temperature sensor 309 provide physiological/psychophysiological and ambient/physiological data streams that measures the wearer’s electrodermal activity (EDA), galvanic skin response (GSR), body and ambient temperature. These are utilized in combination with the biopotential AFE 310 resulting in real-time and continuous monitoring of the wearer’s electrical skin properties, heart rate, respiratory rate, and blood pressure detection. Like the previous sensors, all are timestamped/synchronized for microprocessor processing, analysis, labelling/comparison and interventional activation.
  • EDA electrodermal activity
  • GSR galvanic skin response
  • the biopotential AFE 310 provides electrocardiogram (ECG) waveforms, heart rate and respiration, which in turn, feeds forward to the microprocessor 312 to assist with a user’s physiological/psychophysiological state, processing, and attend onal focus/anxiety/fatigue intervention(s).
  • ECG electrocardiogram
  • FIG. 17 An additional block diagram providing additional microprocessor details (ARM processor) is illustrated in FIG. 17.
  • a catalogue of user-specific cues and resolutions can be stored in a database in communication with the software stored in and executed from the wearable device and the control program/app, and available for machine learning purposes providing the application and hardware with ever-increasing understanding of user environments and physiology cues, alerts, filters, and guidance.
  • An artificial intelligence (Al) algorithm continuously processes user personalization, input cues and uniquely crafted resolutions to further narrow and accurately predict and respond to physiological/psychophysiological input and responses. This machine learning and Al algorithms increase user training and promote greater autonomy, comfort, alertness, focus and mental health.
  • the catalogue is available for user and professional analyses, data streams and progress reports are available for clinical study, medical practitioner/telemedicine, evaluation, and further review.
  • the wearable device preferences can be modified by the user to optimize device battery life.
  • the device can be configured to operate in a power saving mode that conserves battery life by making the sensor(s) less sensitive, limits power for less used operations, or otherwise operates in a manner to maximize battery life.
  • the user can have the option of selecting an enhanced processing mode that emphasizes processing (e.g., makes the sensor(s) more sensitive) but uses more battery per unit of time.
  • the wearable device can be associated with an application that provides diagnostic data relating to the user, system, or for a caretaker/healthcare professional.
  • user diagnostic data can include user preferences, user responsivity, and generated issues and warnings.
  • System diagnostic data can include environment and device responsivity, and issues and warnings.
  • Caretaker/healthcare professional diagnostic data can include user efficacy performance (e.g., sonic, visual, or interoceptive), and any areas of concern such as wearer guidance or device guidance.
  • real-time filtering of audio signals can be implemented in response to collecting sensory data from one or more sensors of the wearable device. As contrasted with adjusting time or overall amplitude of the signal experienced by the listener, this filtering can take place in the frequency domain and affect at least a center frequency (Hz), a cut or boost (dB), and/or a width (Q). For example, all low frequency hum associated with a real-time detection of machinery and/or light ballasts in an environment can be eliminated and/or otherwise reduced, minimized and/or mitigated.
  • Hz center frequency
  • dB cut or boost
  • Q width
  • audio filtering can also apply to additional domains, including time, amplitude, and spatial positioning (e.g., to filter distracting sounds that modulate from a given direction).
  • While some implementations have been primarily described in the context of modifying and/or filtering distracting sounds (i.e., audio interventions), the technology described herein can implement a similar set of interventions related to visual stimuli, either separately or in combination with other types of stimuli.
  • interventions such as alerts, guidance, and/or combinations without filtering mediations can be implemented.
  • visual interventions can be based upon pupillary response, accelerometers, IMU, GSR detection, and/or video of the wearer’s environment.
  • identified visual interventions can work in concert with audio modifications.
  • FIG. 23 depicts an example wearable device 500 that can be utilized to provide visual interventions, in accordance with some implementations of the disclosure.
  • wearable device 500 can include the sensors and/or transducers of wearable device 10.
  • wearable device 500 also includes a camera 550 and display 551.
  • wearable device 500 is implemented as a wearable HMD.
  • a glasses form factor is shown, the HMD can be implemented in a variety of other form factors such as, for example, a headset, goggles, a visor, combinations thereof and the like.
  • the wearable device can be implemented as a monocular HMD.
  • Display 551 can be implemented as an optical see-through display such as a transparent LED and/or OLED screen that uses a waveguide to display virtual objects overlaid over the real -world environment.
  • display 551 can be implemented as a video see-through display supplementing video of the user’s real world environment with overlaid virtual objects. For example, it can overlay virtual objects on video captured by camera 550 that is aligned with the field of view of the HMD.
  • the integrated camera 550 can capture video of the environment from the point of view of the wearer/user of wearable device 500. As such, as further discussed below, the live video/image feed of the camera can be used as one input to detect visual objects that the user is potentially visually sensitive to, and trigger a visual intervention.
  • real-time overlay interventions can be implemented whereby visual objects and/or optical interruptions are muted, squelched, minimized, mitigated and/or otherwise removed from a wearer’s field of vision.
  • the system’s transducer components e.g., microphones and/or outward facing optics
  • the system’s transducer components can be used in concert with on-board biological sensors and/or projection techniques that train/detect, analyze/match/predict, and/or modify optical cues and/or visible items that correlate to a wearer’s visual sensitivity, attention, fatigue and/or anxiety thresholds.
  • disrupting visual, optical, and/or related scenery can be filtered in real-time such that a wearer does not notice that which is distracting.
  • one or two types of real-time optical enhancement (REOPEN) algorithms can be implemented to detect, predict, and/or modify visual inputs that decrease distraction/mental health issues and increase attention, calmness, and/or focus.
  • the algorithms can provide real-time (i) live-editing of visual scenes, imagery and/or object and advanced notification for distracting optics that match a user’s visual profile; and/or, (ii) live- modification of visual distractions without advanced notification. Interventions that can be delivered in real time using a REOPEN algorithm are illustrated by FIGs. 24A-24C, further discussed below.
  • a Realtime Optical Enhancement and Visual Apriori Intervention Algorithm can be implemented to train/detect, analyze/match/predict, and/or modify visual items that a user deems distracting (e.g., based upon a previously-described and/or created personalized preferences profile) and compares these to prior and/or current physiological/psychophysiological responses to the environment.
  • REOPEN-VAIL Realtime Optical Enhancement and Visual Apriori Intervention Algorithm
  • REOPEN-VAIL Upon detecting a threshold crossing and/or match between interoceptive reactivity (egocentric) and visual cue (exocentric video) detection, REOPEN-VAIL can provide iterative analysis, training, enhancement, contextual modification, and/or advanced warning of optical distractions prior to the wearer’s ability to sense these visual and/or related physiological/psychophysiological cues.
  • VASILI Visual A Posteriori Intervention Algorithm
  • VASILI can be implemented to use multimodal learning methods to train/detect, analyze/match and/or modify visual items that previously a user deems distracting (e.g., based upon previously described or created preferences, prior and/or current physiological/psychophysiological responses, etc.).
  • the optics provide contextual modifications without advanced warning of distractibility in the form of interventions that are delivered following the system’s identification of either ecological and/or the wearer’s physiological/psychophysiological cue(s), in real-time after the user has been exposed to the visual distraction, and as part of an iterative process that can serve as a basis for future training, sensing, and/or apriori algorithms.
  • Various interventions can potentially be delivered in real time using a REOPEN algorithm.
  • a certain visual object e.g., a distracting visual object and/or visual anomaly
  • haptic alerts, tone alerts, guidance alerts, combinations thereof and the like can be delivered to notify the wearer.
  • the guidance alerts can provide user-selectable verbal instructions of anticipated visual distraction and/or coaching to intervene with continued focus, calmness, and/or attention.
  • the aforementioned guidance alerts can be implemented as visual and/or text based guidance that is viewable to the wearer, via a displayed (e.g., using display 551) user selectable system visible to one and/or both eyes and/or sightlines to intervene with continued focus, calmness, attention, combinations thereof and the like, (e.g., FIG. 24B).
  • a displayed e.g., using display 551
  • user selectable system visible to one and/or both eyes and/or sightlines to intervene with continued focus, calmness, attention, combinations thereof and the like, (e.g., FIG. 24B).
  • visual distractions e.g., certain objects, faces, etc.
  • This blurring effect can blur the identified, distracting, and/or otherwise offending image as a user-selectable and/or predefined intervention affecting what the wearer sees, (e.g., FIG. 24B).
  • a visual scene can be rendered with a modified background, eliminating the visual distraction, and/or identified image.
  • the system can interpolate nearby images to the distracting object and/or replicate the background by overlaying a “stitched” series of images that naturally conceal and/or suppress the sensory effects of the offending optics, all of which are user-selectable, (e.g., FIG. 24A).
  • a predetermined, user-selectable emoticon and/or place-holder image can be rendered that camouflages the distracting optic and/or visual disruption, (e.g., FIG. 24B).
  • a visual distraction can be rendered with a modified color palette and/or related pigmentation can be modified to user-preference to reduce the effects of distraction, sensitivity, focus, anxiety, and/or fatigue, and/or combinations thereof and the like (e.g., FIG. 24C).
  • a visual distraction can be rendered with edited brightness and/or sharpening of images that are user-selectable as either muted and/or modified visuals (e.g., FIG. 24C).
  • a visual distraction can be rendered with an edited size such that images are augmented and/or modified such they become more prominent, larger, and/or highly visible (e.g., FIG. 24C).
  • FIG. 25 depicts one particular example of a workflow that uses a REOPEN algorithm to provide interventions, in real-time, in a scenario where there is a singular distracting visual source (e.g., birds flying across the sky and causing the individual to become unfocused from work).
  • the algorithm is implemented using a convolutional neural network (CNN).
  • CNN convolutional neural network
  • distracting stimuli can be visualized by the individual wearing a wearable device (e.g., wearable device 500) that captures cues and then processes and trains on that data (e.g., environmental and/or psychophysiology).
  • Convolution layers of the CNN are formed and/or iteratively examined, for example, visual data that triggers the distraction — or physiology such as pupillary movement, including edges, shapes, and/or directional movement.
  • Connected layers digitize the prior convolutional layers. This can be repeated for multimodal data types such that when layers correlate to pre-defined “eyes on target”, a learned state of focused activity can be recorded.
  • a separate learned event can be memorialized and/or tagged as unfocused activity, resulting in delivery of a digital mediation until a “focused” condition is observed.
  • an apriori intervention could be one that has already been trained on the flow depicted in FIG. 25, and then sensed by the system pursuant to a similar external cue and/or an early reflection of pupillary unfocused prediction. This could generate an alert prior to the actual long-term individual state in an attempt to mediate prior to distractibility.
  • the posteriori flow could repeat, this time offering mediations consisting of alerts, guidance, and potentially filtering to mute, eliminate, mitigate, minimize and/or otherwise modify the offending distraction.
  • a PPI study was conducted to identify dependent/thematic variables and dependent/ demographic factors related to the utilization of the wearable device.
  • the PPI participants included verbally able, autistic and neurotypical adolescents and adults aged 15- 84. All participants had intelligence in the normal or above average range and the majority were living independent lives, i.e., study participants did not fall into the general learning disabilities range. Participants provided health/medical conditions and disability information relevant to their opinions about distractibility, focus and anxiety at both school and work. Before the study, participants provided informed consent along with a verifiable ASC diagnosis, where applicable. All participants were invited to take part in a Focus Group, User Survey Group or both. [0307] The Focus Group included 15 participants, ages 17-43, and participants studied distractibility and attentional focus. The main task of the Focus Group was to comment on sensory issues and provide input into the design of a user survey to ensure relevance to autism and adherence to an autism-friendly format.
  • the User Survey Group included 187 participants, ages 18-49, and provided first-person perspectives on distractibility and focus while gathering views and opinions of which aspects of technological aid/support would be most welcomed and have the biggest impact on sensory, attentional, and quality of life issues.
  • LEA 2 Se Lived Experience Attention Anxiety Sensory Survey
  • the variables are mutually exclusive (i. e. , no question was included in the computation of more than one variable).
  • inter-item correlations for each variable were investigated, all of which had a Cronbach’s alpha greater than .80, which demonstrates high internal validity.
  • VDQ Visual Difficulty Quotient
  • SDQ Sound Difficulty Quotient
  • PDQ Physiological Difficulty Quotient
  • a SART study was conducted subsequent to the PPI study and PTBE.
  • the study included online testing designed to test sensory issues affecting participants diagnosed or identifying with ASC.
  • this study examined a subset of components within a wearable prototype to answer two questions: (i) is it possible to classify and predict autistic reactivity/responsiveness to auditory (ecological) disturbances and physiological/psychophysiological distractors when autistic individuals are assisted through alerts, filters and guidance; and (ii) can the exploration of Multimodal Learning Analytics (MMLA) combined with supervised artificial intelligence/machine learning contribute toward understanding autism’s heterogeneity with high accuracy thereby increasing attentional focus whilst decreasing distractibility and anxiety.
  • MMLA Multimodal Learning Analytics
  • baseline testing and related scores are derived both procedurally on pre- and post-subtests to create putative, cognitive conflicts during subtests that may result in a hypothesized and measurable uptick in both distractibility and anxiety. Simultaneously, this upsurge will likely pool with diminished focus and conical attentional performance. Finally, and during the latter subtests, a “confederate” (human wizard) will present a collection of hand-crafted alerts, filters and guidance. These will emulate the operation of the wearable intervention by offsetting and counterbalancing distracting aural stimuli. To reduce fatigue effect, these interventions will either exist in counterbalanced, randomized and possibly multiple sessions.
  • This study tests a sub-system mock-up using multimodal, artificial intelligence-driven (MM/ Al) sensors designed to provide personalized alerts, filters, and guidance to help lessen distractibility and anxiety whilst increasing focus and attention by enhancing cognitive load related to unexpected ecological and physiological/psychophysiological stimuli.
  • the study uses a series of online experiments in which the wearable’s operation is simulated by a confederate, human operator.
  • This study proposes within-subjects, two-condition SART employing multimodal sensors during which a user’s performance is measured (Robertson, I. H., Manly, T., Andrade, J., Baddeley, B. T., & Yiend, J. (1997). 'Oops!
  • SART tasks are performed, and data is collected, with and without the effects of distracting sonic stimuli.
  • This modality serves as both the singular and irrelevant foil, when accompanied by various subtest combinations of advanced alerts, audio filtering and retum-to-task guidance models. These combinations serve as the intervention(s).
  • the study subtests exploit visual search of targets against competing and irrelevant foils (e.g., alpha-numeric). Supplementing these textual targets with additional contesting modalities (e.g., sonic foils and interventions) makes this SART study novel compared to previously-conducted studies. SART requires participants to “actively inhibit competing distractors and selective activation of the target representation.
  • the first PPI study and PTBE facilitated a deeper understanding of the lived experiences of autistic individuals’ and their focus, distractibility and anxiety concerns with a particular focus on later-life, educational and workplace experiences.
  • the PPI study and PTBE also provided information regarding a potential decrease in both anxiety and sensitivity as autistic people age, and that these trends differ within specific modalities. Stability is achieved across various ages for a sonic variable but varies for both visual and physiological/psychophysiological variables. Further, anxiety and sensitivity may not relate across gender. And while there are downward aging trends in both technology tolerance and distractibility, there is variation in ages 30-39 perhaps due to the massive size of this particular sample.
  • the study design is rooted in a S ART/W oZ design and includes online experiments whereby system operations were simulated by a human operator armed with prior, hand-crafted interventions and scripts that support participants’ testing (Bemsen, N. O., Dybkjser, H., & Dybkjser, L. (1994). Wizard of oz prototyping: How and when. Proc. CCI Working Papers Cognit. Sci./HCI, Roskilde, Denmark). The Wizard of Oz (WoZ) study design provides economical and rapid implementation and evaluation, and has gained academic acceptance and popularity for decades.
  • WoZ proposes a within- subjects, two-condition SART employing multimodal sensors during which a user’s errors of commission, errors of omission, reaction time, state-anxiety, and fatigue levels are computed.
  • a statistically significant test result indicates that the test hypothesis is false or should be rejected, and a p-value greater than 0.05 means that no effect was observed.
  • the statistical power of a significance test depends on: (i) the sample size (N), such that when N increases, the power increases; (ii) the significance level (a), such that when a increases, the power increases; and (iii) the effect size, such that when the effect size increases, the power increases.
  • Half the sample included neurotypical participants and half identified as or possessed an ASC diagnoses. All participants utilized pre/post WoZ manipulations. Baseline testing and related scores were derived both procedurally on pre- and post-subtests. Putative, cognitive conflicts during subtests that may result in a hypothesized and measurable uptick in both distractibility and anxiety were created. Simultaneously, this upsurge likely pooled with diminished focus and conical attentional performance.
  • STAFI state-trait anxiety and state-trait fatigue inventory
  • Participants selected from five state anxiety items including illustrations and text that depicted how they were feeling at the moment of query, including: “1 — Extremely anxious”, “2 — Slightly anxious”, “3 — Neither anxious nor calm”, “4 — Slightly calm” or “5 — Extremely calm”; “I am concerned”; “I feel calm”; I feel secure.” Lower scores indicated greater anxiety.
  • Performance on the SART clearly requires the ability to inhibit or withhold a response. This is made more difficult when distractors are introduced into the testing paradigm. Specifically, hand-crafted sonics of varying amplitude, frequency, time/length, distortion, localization, and phase were introduced to mimic those sounds that might occur in office, workplace, education, and scholastic settings.
  • a total of twenty-eight (28) sound sources were played over a duration of five- minutes and included office industrial, fire alarms, telephone ringing, busy signals and dial tones, classroom lectures, photocopier and telefacsimile operations, footsteps, sneezes, coughs, pencil scribbling, and the like.
  • the human wizard is predominantly a conductor/ evaluator whose functions and monitoring of programmatic materials are unidentified to the participant. Users make selections through a “dumb” control panel, provisioning their customized alerts, filters and guidance. Importantly, the mechanism advances autonomy by providing specific functionalities for participant evaluation, whilst ostensibly eliminating evaluator influence. In selecting these components, the following questions are reviewed: What requirements should the evaluator meet before conducting a study? How does the evaluator follow the plan, and what measurements will reflect test and sub-test flow? How should control panel component be designed, and how would this affect its operation? How does the evaluator’s personal behavior affect system operation?
  • the overarching study was divided into four components including: (i) the PPI study and PTBE described earlier; (ii) the evaluator (including tasks, self-reports and controls); (iii) the system prototype (a non-wearable sub-system); and (iv) the participants (who were recorded). Study variables are listed in Table 3A, and illustrated in FIG. 18:
  • FIG. 6 is a flowchart illustrating the S ART/W oZ Protocol used in this study, and includes four higher-order classes that include study aims, variables, assessments, and outcome measures. Study questions, independent and dependent variable, potential assessments/activities and expected results are also depicted. Based upon this SART/WoZ Protocol design, the corresponding class descriptions are listed in Table 3B:
  • Each participant took part in a single experimental session after first completing consent and demographic forms.
  • the session commenced with a short (1-2 minute) tutorial to ensure that the participant was comfortable with the proper operation of the testing software, and to introduce the participant to the importance of staying within range of the web camera and pointing devices for proper monitoring of the environment and their physiology.
  • participants were advised that the evaluator was available throughout the session to help monitor the system and to answer any questions between tests. Participants were not advised of the evaluator’s contribution to the testing (WoZ), that any alerting, filtering or guidance programming was pre-defined prior to the experiment, or that their control of the system preferences was of a placebo nature.
  • the WoZ testing (from baseline through multiple interventions and then a return to baseline) included three phases. Phase I commenced with Baseline I cognitive testing; that is, there were neither distracting cues nor interventions. Phase II introduced accompanying filters, alerts and guidance applied in concert with randomized sonic distractions and testing. Phase III reintroduced a return to baseline to ensure that participants’ recovery and responses were not memorized and that randomization effects were properly sustained. [0353] Alerts, filters and guidance structure:
  • Protocol testing measures [0358] Protocol testing measures:
  • Participants were instructed to remain in close proximity to their computer’s web camera and in direct contact with at least one of their pointing devices (e.g., mouse, trackpad, keyboard) at all times during the experiment. Participants were also informed that: measures of engagement, focus, comfort, productivity, and autonomy would be tested; environmental and physiological/psychophysiological monitoring (e.g., ecology and interoceptive) would occur during testing; and participant head sway, pupillary responsivity, GSR, environmental sound and vision would be collected.
  • pointing devices e.g., mouse, trackpad, keyboard
  • participant received combinations of support by way of alerts prior to distraction and/or filtered audio cues (e.g., distractions that are muted, spatially centered, etc.).
  • participants also received post-stimuli guidance to help them return to tasks/activities/tests.
  • This study utilized three data capturing methods — direct computer input/s coring, video analysis and self-reporting. The first is integrated in the Gorilla application, the second aims to record and make possible observations of subjects’ system interactions, and the third may reflect the participant’s and evaluator’s operation experiences (Goldman, N., Lin, I.-F., Weinstein, M. and Lin, Y.-H. 2003. Evaluating the quality of selfreports of hypertension and diabetes. Journal of Clinical Epidemiology 56, 148-154).
  • LEA 2 Se Lived Experience Attention Anxiety Sensory Survey
  • the POC/T confirmed adequate systems operation, and translation from user interfaces to data collection devices and downstream to analysis applications.
  • Each participant was given four discrete tests including the matrix reasoning item bank (MaRs-IB): a novel, openaccess abstract reasoning items for adolescents and adults; the Autism-Spectrum Quotient (AQ): a 50-item self-report questionnaire for measuring the degree to which an adult with normal intelligence has the traits associated with the autistic spectrum; and the Adult ADHD Self-Report Scale (ASRS A and ASRS B) Symptom Checklist: a self-reported questionnaire used to assist in the diagnosis of adult Attention Deficit Hyperactivity Disorder (ADHD) and specifically daily issues relating to cognitive, academic, occupational, social and economic situations.
  • ADHD Attention Deficit Hyperactivity Disorder
  • Stepwise Regression [0371] Dummy variables were created for both age and gender (i.e., the only demographic factors that were not correlated), and were combined with sensitivity, anxiety and distractibility variables (SI, AP and DQ) embedded within a stepwise regression analysis to predict scores in sound, visual and physiological/psychophysiological/interoceptive modalities. The model(s) with the highest R2/significance are reported in Table 7:
  • EOC-RT Errors of Commission Response Times
  • autistic response times were shorter (faster) than neurotypical controls. This can be due to various factors differentiating neurodiverse responsivity — including, but not limited to, greater neural processing, differences in genetic makeup affecting sensory reactivity, and superior activity in the visual cortex (Schallmo, M.-P., & Murray, S. (2016). People with Autism May See Motion Faster. 19).
  • autistic participants experienced a RT increase of 19.39% (i.e., a desired slowing from onset of distraction to guidance intervention) while neurotypical counterparts produced an undesirable decreased in RT (speeding up) of nearly one percent (-0.74%) for the same period.
  • FIGS. 9A to 9C are graphical representations of EOC as it relates to Response Time (RT) of the full cohort of participants in the SART/WoZ study described herein.
  • FIG. 9A shows the EOC vs RT from starting baseline to final baseline
  • FIG. 9B shows the EOC vs RT intervention effect
  • FIG. 9C shows the lasting effect of EOC vs RT.
  • a slowing of reaction time portends to greater mindfulness, which can be defined as a participant’s awareness of their internal feelings and a subsequent ability to maintain awareness without evaluation or judgement (e.g., defined as an outcome).
  • the wearable device described herein cultivates mindfulness vis a vis bespoke intervention (assistive technology). This helps to shift and shape a participant’s wandering mind and their awareness. Essentially, the participants in this study become more aware, productive, and comfortable through alerts, filters, and guidance when exposed to sensory interruptions during a Sustained Attention to Response Task (SART). Over time, participants become more attentive, less sensitive, less anxious, and less fatigued.
  • RTs increase (slow down) 17.72 ms for both ASC (i. e. , from distraction onset to guidance intervention) and for NT (i.e., by 19.06 ms for distraction onset to alert interventions). These represent the maximum increases in RT for both groups and are non-contrasting (i.e., again, both slow down). Equally significant is RTs lasting effect; that is, neither autistic nor non-autistic participants benefit from a slowing RT once the intervention is removed. Both ASC and NT groups speed up their responses by 19.10 ms and 2.91 ms, respectively (even though there is positive lasting performance by way of fewer errors). These results are shown in FIGS. 10A to 10C.
  • autistic response times are typically faster than neurotypical participants for the same tasks and interventions.
  • reduced errors improved performance
  • autistic participants exhibit greater variability in improvement, while neurotypical participants produce fewer errors overall.
  • combined interventions e.g., alerts, filters, and guidance
  • both NT and ASC are equivalent a lessoning to 7.4 errors each.
  • EOO response times resembled EOC for autistic participants; in that, both were faster than neurotypical controls, due in part to previously mentioned neuronal processing and responsivity.
  • autistic participants experienced an RT increase of 9.28% (i.e., a desired slowing from onset of distraction to filters intervention), while neurotypical counterparts produced an undesirable decrease in RT (speeding up) of nearly one percent (4.44%) for the identical intervention.
  • RTs also effect Errors of Omission, when comparing autistic and non-autistic groups. There is a lessening of EOO (though these still produce inaccuracies) among neurodiverse participants (-15.12%). Similarly, an increase in accuracy (less EOOs) are exhibited among neurotypical participants. Unsurprisingly, faster RT (4.44% in the case of NT participants from distraction to filter) did, in fact, create more errors (15.09%). As would also be expected, slower RTs among autistic participants (9.28%) resulted in fewer EOOs (15.12%). Curiously, both groups responded oppositely to similar intervention (by way of RTs), and by equal and opposite magnitudes in accuracy with NTs (not ASC participants) experiencing greater errors.
  • the PPI study examined issues and connections among three variables: sensory (sensitivity), mental health (anxiety and fatigue), and distractibility (attention).
  • the PPI study was used to develop a sensitivity mental health distractibility model, depicted by FIG. 26, designating how anxiety and fatigue can mediate sensory sensitivity and distractibility, within both autistic and non-autistic diagnostic groups.
  • the model of FIG. 26 designating how anxiety and fatigue can mediate sensory sensitivity and distractibility, within both autistic and non-autistic diagnostic groups.
  • the model links sensory cues (e.g., labeled #1 that includes an individual’s hyper, hypo- and sensory-seeking characteristics) to mental health mediators (e.g., labeled #2 that describes an individual’s anxiety and/or fatigue) to distractibility (e.g., labeled #3 that explains an individual’s capacity to focus/maintain attention). From an ordering standpoint, the model extends sensory cues through mental health characteristics that can further modulate an individual’s attentional reactivity, versus a straight line leading from cue to distractibility alone. While sensitivity has been previously hypothesized to disrupt top-down and bottom-up attention, the model of FIG. 26 embodies a new and lateral relationship more fully depicting an autistic individual’s sensitivity and attention processing.
  • sensory cues e.g., labeled #1 that includes an individual’s hyper, hypo- and sensory-seeking characteristics
  • mental health mediators e.g., labeled #2 that describes an individual’s anxiety and/or fatigue
  • distractibility
  • FIG. 27 is a flowchart depicting the design/method of the PPI study discussed above.
  • the PPI study was implemented in two parts.
  • the first phase consisted of five autistic- only (14 adults, 18-54-y ear-old) online focus groups to better understand daily experiences relating to sensory sensitivity (FIG. 27, Item #2).
  • the focus groups examined how sensory sensitivity impacted both attention and mental health across three themes: (i) lived experience in adult contexts of higher education institutions, employment, and social venues; (ii) technology tolerance, and digital mediations that can help autistic individuals in adverse sensory environments; and (iii) language that was relevant, easy to understand, and autismfriendly for the Phase 2 questionnaire with a larger group of participants.
  • the questionnaire design provided for diversity in lived experiences by allowing opportunities for open-ended responses, in addition to multiple choice questions.
  • Table 12 shows the demographics of the participants in the first phase of the PPI study.
  • Theme 1 Sensitivity/Impact
  • Table 16 shows the demographics, including gender, age, and education level, of the participants in the second phase of the PPI study.
  • the online questionnaire was designed to expand the initial focus group inquiry beyond word usage and their alternations to data collection that described sensory patterns, technology experiences, and desire for accommodations - all described using genuine, clear, and relatable language by a larger sample of respondents who might share experiences identified by the smaller group of original participants.
  • a majority of phase two questions were derived from various sources including the UCL Student Mental Health Survey (SENSE; McCloud et al., 2019), the Stanislaus State Concentration Questionnaire (CQ, 2019), the CogniFit Online Cognitive Assessment Battery for Concentration (CAB-AT, 2018), the Cognifit Cognitive Assessment for ADHD Research (CAB-ADHD, 2018), and those from semi-structured interviews developed by Ashbumer and colleagues (2013). These questions were supplemented and/or fine-tuned by the researchers.
  • VSV Visual Sensitivity Variable
  • CQ Concentration Questionnaire
  • CAB-AT Cognitive Assessment Battery for Concentration
  • Sample queries included “I am easily distracted or sensitive to certain environmental sights/visions ” and “I avoid visually stimulating environments ”.
  • ASV Auditory Sensitivity Variable
  • Seven questions derived from the Stanislaus State Concentration Questionnaire (CQ, 2019), the Cognitive Assessment Battery for ADHD Research (CAB-ADHD; Cognifit, 2018), and semi-structured interviews (Ashbumer et al., 2013) — described loud, startling, and other cues unfavorably affecting individuals. Examples questions included “I am easily distracted or sensitive to certain environmental sounds (for example, noises, loudness, pitches, conversations) ” and “I would describe sounds like humming of lights or refrigerators, fans, heaters or clocks ticking as distracting” . Four questions were reverse scored reflecting higher scores that indicating greater sensitivity.
  • PSV Physiological Sensitivity Variable
  • CQ Stanislaus State Concentration Questionnaire
  • CAB-ADHD Cognitive Assessment Battery for ADHD Research
  • Sample queries include “I do not like being touched” and “I am easily distracted or sensitive to certain physiological feelings of thoughts (for example anxiousness, racing thoughts, ringing in my ears)
  • Anxiety Variable 16 questions — derived from UCL’s Student Mental Health Survey (SENSE; McCloud et al., 2019), interviews (Ashbumer et al., 2013), a Cognitive Assessment Battery for Concentration (CAB-AT; Cognifit 2018) — described how sounds, sights, and sensations affect anxiety. Questions included “Certain sounds, sights, or stimuli make me feel nervous, anxious, or on edge” and “Nothing really distracts me or makes me anxious”. Three questions were reverse scored reflecting higher scores that indicated greater anxiety.
  • Autism Spectrum Quotient 10 questions — derived from the Autism Spectrum Quotient (Allison et al., 2012) — examined only non-autistic individuals who may have undiagnosed and/or milder levels of autistic symptomatology (Baron-Cohen et al., 2001). These measures were used to ensure in-group participant matching and determine if subclinical traits existed in non-diagnosed individuals.
  • the AQ-10 is a subset of the larger 50-item survey and uses a four-point Likert Scale ranging from Nonetheless Agree to Hence Disagree across 10 questions. A single point maximum is scored for each answer and totals scores of six or greater indicate concentrations of autistic traits.
  • Diagnostic mediation analyses tested the hypothesis connecting anxiety as an intermediary between sensory and distractibility (See FIG. 26).
  • a statistical package was used to carry out analysis evaluating measurement errors, adjusting with bootstrapping techniques (e.g., random sampling with replacement bias-corrected and accelerated 95% confidence intervals; 1000 resamples).
  • FIG. 29 shows graphs of mean anxiety and distractibility/attention scores.
  • a series of one-way between subjects ANOVA tests were conducted to compare differences in anxiety and distractibility scores across various demographic features.
  • Table 17 shows sensory sensitivity variables and mental health scores and statistics for diagnostic groups, including means scores and between group differences.
  • Tables 18 and 19 show sensitivity and outcome variable correlations for non- autistic and autistic participants.
  • N 39. Minimum possible values were 1 and maximum possible values were 5. Distraction Fatigue, Intervention Fatigue Alert and Intervention Fatigue Combination were each missing 1 data point.
  • N 39. RT measured in milliseconds (ms).
  • N 38; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • a multinominal logistic regression was performed to model the relationship between diagnostic group, age, education, and gender with the best performance intervention. As shown in Table 32, this regression was repeated such that the reference category was varied between the intervention types (alert, filter, guidance, combination).
  • N 39; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • N 39; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • N 39; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • ASD M (SD) NT: M (SD) t(df)
  • N 349; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • Table 38 shows Bonferroni adjusted Spearman rank correlations between the variables for both the non-autistic and autistic groups.
  • Application of the Kolmogorov- Smirnov test revealed that most variables, except for visual and anxiety in the non-autistic group, were non-normal.
  • For the non-autistic group significant associations were found between each of the sensory sensitivity variables and the outcome variables (anxiety, fatigue, and distractibility), as well as between all the sensory sensitivity variables.
  • N 162; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • N 187; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • Table 41 shows a regression predicting distractibility from the sensory variables as well as the demographic variables for the non-autistic group.
  • N 162; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • N 187; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • N 162; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • N 187; *p ⁇ 0.05 (two-tailed); **p ⁇ 0.01 (two-tailed).
  • the Sobel test was then conducted to test the indirect effect for statistical significance for each model.
  • FIGs. 34A-34B show summary results of the PPI study described herein, which provided a basis for conducting and refining the SART/WOz clinical study described herein.
  • different associations between individual demographics e.g., age sex, and education
  • anxiety, fatigue, and/or focus were found in the autistic group versus the neurotypical group. Additionally, it was found that the autistic group tended to be more sensitive to visual and physiological stimuli, and less fatigued.
  • FIG. 34B it was found that in some cases anxiety and fatigue mediated sensory sensitivity in a different manner for the autistic group versus the non-autistic group. For example, whereas for autistic individuals it was found that fatigue significantly mediated the indirect and direct relationship between auditory sensitivity and distractibility, no such relationship was found for non-autistic individuals.
  • FIGs. 35A-35B show summary results of the SART/WOz clinical study described herein. As described above, the study involved multiple trials that tested different mediations/interventions and different sensory cues. As depicted by FIG. 35A, all mediations/interv entions (e.g., alert, filter, guidance, or combination hereof) were found to improve anxiety in both autistic and non-autistic individuals. By contrast, only some mediations (e.g., filters, or combination) were found to improve fatigue whereas others (e.g., guidance) were found to potentially be detrimental. In either case, mediations that were customized to particular individuals for each group were found to be the most effective.
  • all mediations/interv entions e.g., alert, filter, guidance, or combination hereof
  • FIG. 36A is an operational flow diagram illustrating an example method 3600 for initializing and iteratively updating one or more sensory thresholds and one or more mediations associated with a specific user.
  • method 3600 can be implemented by one or more processors (e.g., one or more processors of wearable device 10 and/or mobile device 20) of a wearable device system executing instructions stored in one or more computer readable media (e.g., one or more computer readable media of wearable device 10 and/or mobile device 20).
  • processors e.g., one or more processors of wearable device 10 and/or mobile device 20
  • computer readable media e.g., one or more computer readable media of wearable device 10 and/or mobile device 20.
  • Operation 3601 includes obtaining demographic data of the user of the wearable device. This can include receiving user input at a user interface indicating an age, education level, gender, or other demographic data of the user.
  • Operation 3602 includes obtaining user sensory sensitivity data indicating whether the user is visually sensitive, sonically sensitive, or interoceptively sensitive. This can include receiving user input at the user interface indicating whether the user of the wearable device is visually sensitive, sonically sensitive, or interoceptively sensitive.
  • the user input can includes input at a GUI including one or more responses by the user to one or more prompts that are indicative of whether the user is visually sensitive, sonically sensitive, and/or interoceptively sensitive.
  • These responses can indicate user preferences to certain sensory inputs such as stimuli that the user prefers, stimuli that make the user uncomfortable, the user’s perceived and/or measured sensitivity to different stimuli, and the like.
  • the response can include responses to questions as described with reference to the studies discussed above.
  • Operation 3603 includes obtaining neurodiversity data indicating whether the user is neurodiverse or neurotypical.
  • the neurodiversity data can indicate whether the user is autistic or non-autistic.
  • the system can store a first identifier that indicates whether the user is neurodiverse or neurotypical.
  • the neurodiversity data can be obtained by user input at a user interface indicating whether the user has been diagnosed as neurotypical.
  • the wearable device system can be configured to perform a method for providing a diagnostic prediction of whether the user is neurodiverse or neurotypical.
  • Operation 3604 includes initializing and storing the one or more sensory thresholds and one or more mediations associated with the user. The thresholds and mediations associated with the user can be based on the user sensory sensitivity data, the demographic data, and/or the neurodiversity data. In some cases, the demographic data can be ignored.
  • Operation 3605 includes collecting sensor data and environmental data while the user wears the wearable device.
  • Operation 3606 includes in response to collecting the sensor data and/or environmental data while the user wears the wearable device, modifying the one or more sensory thresholds and the one or more mediations associated with the user.
  • operations 3605-3606 can iterate over time as the user utilizes the wearable device system to provide sensory relief.
  • the frequency with which the one or more sensory thresholds and the one or more mediations are updated in response to newly-collected data can be configurable, system-defined, and/or user-defined. For example, updates can depend on the amount of data that is collected and/or the amount of time that has passed.
  • operations 3605-3606 can be skipped. For example, the user can disable updating the thresholds and/or mediations based on actual use of the wearable device.
  • the features found to be correlated with autistic versus non-autistic users can provide a basis for training a model that given, specific features corresponding to a user (e.g., sensory sensitivity features, anxiety features, fatigue features, demographic features, etc.) outputs a prediction (e.g., as a likelihood/probability) that a user is autistic or not autistic.
  • a prediction e.g., as a likelihood/probability
  • method 3610 or method 3620 are operational flow diagrams illustrating example methods 3610, 3620 for predicting whether a user is neurodiverse (e.g., autistic) or neurotypical.
  • method 3610 or method 3620 can be implemented by one or more processors (e.g., one or more processors of wearable device 10 and/or mobile device 20) of a wearable device system executing instructions stored in one or more computer readable media (e.g., one or more computer readable media of wearable device 10 and/or mobile device 20).
  • Operation 3611 includes deriving, based on the sensory sensitivity data, one or more sensory sensitivity scores including a visual sensitivity score, a sonic sensitivity score, and/or an interoceptive sensitivity score. For example, based on the user’s response to the prompts, one or more scores (e.g., normalized on a scale such as 0-100) can be derived.
  • Operation 3612 includes obtaining anxiety data measuring a general anxiety level of the user. For example, this can include receiving at a GUI one or more responses by the user to one or more prompts indicating an anxiety level of the user in different contexts.
  • Operation 3613 includes deriving, based on the anxiety data, an anxiety score.
  • Operation 3614 includes predicting, using a trained model, based on the one or more sensory sensitivity scores and the anxiety score, a likelihood that the user is neurodiverse.
  • the model can be configured/trained to predict a probability of autism based at least on features includes an anxiety level/score and one or more sensory sensitivity levels/scores of a given user. Each of the features can be weighted differently. It should be noted that the model can also be trained to consider other features (e.g., demographic data) when making the prediction.
  • Operation 3615 includes making a determination that user is neurodiverse or neurotypical, and storing an associated identifier. For example, if the prediction output by the model meets a threshold (e.g., > 80% probability), a prediction that a user is autistic can be made.
  • the system can validate the prediction by measuring the user’s performance in response to certain tasks when mediations are present and not present. This performance can be measured using the wearable device and/or mobile device by administering SARTs as discussed above. The level of improvement in the user’s performance, given a particular mediation, can further validate whether the predicted diagnosis is correct or incorrect.
  • Operation 3621 includes obtaining fatigue data measuring a general fatigue level of the user 3621. For example, this can include receiving at a GUI one or more responses by the user to one or more prompts indicating a fatigue level of the user in different contexts.
  • Operation 3622 includes deriving, based on the fatigue data, a fatigue score. For example, based on the user’s response to the prompts, a score (e.g., normalized on a scale such as 0-100) can be derived.
  • Operation 3623 includes predicting, using a trained model, based on the one or more sensory sensitivity scores and the fatigue score, a likelihood that the user is neurodiverse 3623.
  • the model can be configured/trained to predict a probability of autism based at least on features includes a fatigue level/score and one or more sensory sensitivity levels/scores of a given user. Each of the features can be weighted differently. It should be noted that the model can also be trained to consider other features (e.g., demographic data) when making the prediction. In some implementations, both anxiety and fatigue features can be considered in the trained model.
  • the multi-sensory, assistive wearable technology described herein can be implemented using a network topology that ensures user data privacy and facilitates ethical relationships among device layers, systems, and stakeholders such as the user/wearer, the user’s family, the user’s therapist, and/or the user’s general practitioner.
  • edge and fog computing can be implemented using devices localized at the system’s perimeter to facilitate and secure any cloud connectivity using devices localized at the system’s perimeter. These devices can be independent and connect to both sensors and applications while serving as data transceivers between components, software, and — only when required — the cloud. This can provide desirable and reliable constraints for data computation, particularly as sensitive data can be substantial, often disorganized, and subject to exploitation. Owing to the cloud’s limitations for exposure, fog computing can provide additional layers of efficiency and security.
  • FIG. 37 illustrates an example system architecture/topology for implementing fog data processing in accordance with some implementations of the disclosure.
  • the system architecture includes loT sensors 3710, edge layer 3720 including edge nodes 3721, fog layer 3730 including fog nodes 3731, and cloud layer 3740 including one or more cloud computing devices 3741.
  • FIG. 37 will be primarily described in context of a system architecture as applied to a single user/wearer, it should be appreciated that this system architecture can be extended to multiple independent users.
  • loT sensors 3710 can be sensors implemented as part of a wearable device (e.g., wearable device 10 or wearable device 500.
  • the sensors can include a pupillometry sensor 204, a galvanic skin sensor 205, an inertial movement unit 206, a temperature sensor 309, an audio sensor 309, an image sensor (e.g., as part of camera 550), etc.
  • the loT sensors 3710 can also include sensors that are in the same environment as the wearer but implemented in a different device.
  • the sensors can include sensors implemented in a mobile device 20 (e.g., GPS or motion sensors), ambient temperature sensors, image sensors of external loT devices, audio sensors of external loT devices, etc.
  • Edge nodes 3721 and fog nodes 3731 can be implemented in hardware including, but not limited to, client-side wearable devices (e.g., wearable device 10 or 500), a mobile device 20, and/or locally (i.e. , pre-cloud) operated servers or database devices that can be provided by the provider of the wearable device system.
  • the fog layer 3730 resides between the edge layer 3720 and cloud layer 3740.
  • the edge nodes can reside between the cloud nodes and fog nodes.
  • some edge nodes reside between cloud nodes and fog nodes, and some fog nodes reside between edge nodes and cloud nodes.
  • fog node(s) 3731 receive data from edge node(s) 3721 they can filter the data by deterministically passing only appropriate data to the cloud computing devices 3741 for processing, storage, networking, etc.
  • edge and fog computations can be implemented where an ecological parameter (e.g., temperature) or physiological parameter (e.g., heart rate) is regularly sensed and collected as data (e.g., every second of operation) to align user fatigue and anxiety with other ecological/physiological measures.
  • an ecological parameter e.g., temperature
  • physiological parameter e.g., heart rate
  • every sensor measurement could potentially be transmitted to a cloud application to accommodate the user/ wearer and downstream monitoring for therapists, general practitioners, and/or family members.
  • a rules-based fog layer 3730 could prevent this excessive data transfer from congesting the network and/or compromising the user’s privacy/security.
  • a fog node 3731 can be configured to pass only critical data as it occurs (e.g., excessive temperature spikes), or only data collected by certain sensors (e.g., no image or sound data is made available to the cloud).
  • the fog node(s) 3731 or edge nodes 3721 can also encrypt any data prior to making it available to a cloud computing device 3741 such that information can remain pseudonymized, thereby protecting the user’s privacy. During operation, all encryption, decryption, and purging of data can take place locally at the user level and not using cloud software or hardware.
  • the edge nodes 3721 can be responsible for maintaining a middleware position that manages data flow, encryption/ decry ption, and ultimately expunging data once it is no longer needed.
  • the same device can function as both a fog node and an edge node.
  • data protection and privacy rules can be controlled and managed by the user, allowing configuration of what can and cannot be collected, transmitted, and/or stored.
  • localization can occur within a LAN and/or ad-hoc network of the wearable device (e.g., 10) and/or mobile device (e.g., 20) coupled to the wearable device.
  • a cloud layer 3740 can include a data lake (DL) repository that stores machine learning (ML) data that is not personalized, including images, audio, and/or video. Some or most of this data can be public domain.
  • the fog layer 3730 can compare private and distracting conditions (e.g., as determined from data collection by the wearable device) to the data stored in the cloud layer 3740.
  • the edge layer 3720 can coordinate data flows to the cloud layer 3740, only allowing the most limited flow to the cloud, while the fog layer 3740 can be used to detect distractions by the user based at least in part by the repository of data stored on the cloud layer 3740. As such, the system can operate without the cloud layer personally identifying a user.
  • all personalized user data including thresholds, sensory resolutions, mediations, demographic data, diagnostic data, etc. can be stored at the local level.
  • Deep learning and machine learning data e.g., auditory, visual, etc.
  • distractibility data can be encrypted and stored globally, while real-time comparative reactivity to ecological and physiological data can be momentarily stored locally.
  • FIG. 38A depicts a particular example of a wearable system architecture, including data flows, that leverages fog and edge computing, in accordance with some implementations of the disclosure.
  • FIG. 38B is a flow diagram illustrating operations that are performed by the system of FIG. 38A, in accordance with some implementations.
  • the system of FIG. 38A includes a wearable device 10, one or more edge services 3810, fog services 3820, gateway 3830 that can mediate communication between edge server 3810 and fog services 3820, and one or more cloud computing devices 3840.
  • the functionalities of edge server 3810 can be implemented in wearable device 10 or a mobile device 20 communicatively coupled to wearable device 10.
  • Operation 3901 includes wearable device 10 collecting sensor data.
  • one or more sensors of the wearable device 10 can be used to record a sensory input stimulus to the user. This can include sensing ecological and physiological/psychophysiological data as described above.
  • other devices besides wearable device 10, but in the same environment as wearable device 10 e.g., a mobile device 20
  • collect sensor data e.g.
  • Operation 3902 includes one or more fog nodes of fog services 3820 processing, storing, and/or managing the sensor data that was collected.
  • the one or more fog nodes include a datastore that stores and/or manages the sensor data.
  • the one or more fog nodes include a datastore that stores one or more sensory thresholds specific to a user of the wearable device 10 (e.g., one or more sensory thresholds selected from auditory, visual, or physiological sensory thresholds).
  • the one or more fog nodes compare the sensory input stimulus with the one or more sensory thresholds specific to the user to determine that an intervention could be required.
  • Operation 3904 includes the edge server(s) 3810 encrypting and uploading data to the one or more cloud computing devices 3840. For example, if the fog services 3820 determined, after reviewing a subset of sensor data, that a threshold has been met, this subset of sensor data that trigged the determination can be encrypted by edge server 3810 and uploaded to the cloud.
  • Operation 3905 includes applying a data processing and machine learning pipeline/process.
  • the pipeline can be performed using at least one or more cloud computing devices.
  • Operation 3907 includes presenting an intervention/mediation to the user.
  • a user can be visually distracted, which triggers changes in pupillary measurements.
  • the updated pupillary measurements can result in a threshold being met that causes a mediation/interv ention (e.g., alert to the user to refocus) to be presented to the user.
  • the mediation can be triggered as follows.
  • an outward facing camera e.g., as incorporated in a wearable device captures an image of an object causing the distraction (e.g., the camera captures an image in the direction of the pupillary gaze). If the image matches or is sufficiently similar to (e.g., as determined by calculating a similarity score based on image features) a publicly stored image on the cloud of the same/similar object that was previously tagged as a personalized trigger for the user as a distracting cue, the mediation can be triggered.
  • the machine learning pipeline can be used to match the captured image to the cloud’s data store, and the image can be confirmed up and downstream as a distracting image. To make the comparison can be processed based on different parameters, including color, shape, edge detection, etc.
  • an FBDL model can be used to generate customized mediations given data from one or more sensors as inputs.
  • Deep learning is a machine learning category that uses neural network algorithms that memorialize data for analysis and prediction. Neural networks use hidden layers to obtain features by connecting one another for replicable outcomes (output layers). FBDL confines connections between input and hidden layers so that every veiled unit attaches to a sub-section of its corresponding input. Hence, lower dimensioned characteristics can be derived by arbitrarily sampling big data.
  • FIG. 47 illustrates one example of a FBDL model, in accordance with some implementations of the disclosure.
  • inputs can be one or more different types of sensor data, including audio data, pupillary data, IMU data, GSR data, optical data (e.g., image data), temperature data, etc.
  • the FBDL can be trained to recognize, based on the input data, a particular/personalized mediation type depending on recognition, where the mediation can be an alert, filter, guidance, or combination thereof.
  • OLM graphicalally represent
  • users of multi- sensory, assistive wearable technology e.g., neurodiverse individuals
  • the users or other interested party e.g., therapist
  • OLM components can be incorporated into the systems and methods described herein to improve individual’s hyper-, hypo-, and sensory-seeking challenges, which may affect task accuracy (i.e., performance), and mental health (i.e., calmness and alertness), particularly when distracted by eco — or psychophysiological cues.
  • Mediations that are fully transparent can provision results better than those that limit user’s data access, straightforward system control, confidence, and trust in technologies.
  • FIGs. 39-45 depict an OLM framework in accordance with some implementations of the disclosure.
  • the depicted OLM framework includes three tables (FIGs. 40, 42, and 44) and four flowcharts (FIGs. 39, 41, 43, and 45).
  • the OLM framework depicts custom labeled characteristics pertinent to securing data by the individual user/wearer and their support (e.g., therapist, family, etc.)
  • a wearer may select/actuate controls on a GUI to determine how little or how much data can be sensed, collected, processed, and/or shared on a feature-by -feature basis.
  • characteristics or features can be divided into what elements are important and to be sensed, mediated, and/or stored, how this is accomplished, and access privileges for reviewing and administering these functions.
  • FIG. 39 is a high-level flowchart of the OLM framework.
  • This example OLM framework defines eleven elements (i.e. , model accessibility, presentation, access method, accessibility control, etc.) within three categories, their corresponding properties (i.e., complete, partial, current, future, etc.), and their description (i.e., a textual explanation of each purpose element) used in defining the specific OLM (See tables of FIGs. 40, 42, 44, left to right).
  • These properties signify levels of accessibility purpose elements across eleven aspect columns (i.e.,, from left to right including right to access, control through trust, assessment, etc.).
  • One of the OLM maps describes “what is available” (FIGs. 40-41) by addressing the extent of model accessibility, underlying representations, access to uncertainty, role of time, access to source issues, and access to model personalization.
  • the model’s extent of accessibility (Item #1) is predominantly open “Completely” across the board with critical availability to nearly all stakeholders.
  • One of the OLM maps describes “how the model is presented” to stakeholders, including friends and acquaintances (FIGs. 42-43). Included are presentation details (i.e., word cloud, skill meters, radar plots, etc.), access methods (i.e., inspectable, editable, user versus system persuasion, etc.), and access flexibility. Compared to the “what is available” table, this table include elements tagged with critical and especially critical rankings.
  • FIGs. 44-45 discloses two purpose elements that map focal points (i.e., whom accessibility is derived from) and dominant access (i.e., who controls access over others).
  • the multi-sensory assistive wearable technology described herein can leverage an AR-supported framework of development, analysis, and assessment criteria.
  • the AR support can refer to the use of AR to sonically or visually replace certain auditory or visual information presented to the user, such as, for example, blurring, squelching, or erasing an offending image, or performing digital signal processing of an audio signal to make it less distracting.
  • the framework can provide a mechanism for implementing improved OLM, quantified self (QS) frameworks, and /or multimodal learning analytic (MMLA) frameworks.
  • FIG. 46 depicts a system that implements an AR-based MMLA framework, in accordance with some implementations of the disclosure.
  • the system is configured to implement at least three functions for the user and/or other stakeholders of the multi-sensory assistive wearable technology described herein: battery, diagnoses, and personalization; objectives, aims, and iterative outcomes; and mediative strategy and digital accommodations via technology (e.g., using multimodal sensors and implementing intervention strategies).
  • stakeholders including users can maintain accessibility throughout the framework
  • an ISIP can refer to a data profile provisioned by one or more mobile application(s) and used daily by a user to customize their reactivity and provide behavior modification (e.g., using the Distraction Intervention Desire Questionnaire).
  • An ISIP can help personalize user-specific sensory thresholds and/or sensory resolutions, described herein, that can affect alert, filter, and guidance interventions provided for a given user.
  • ISIPs can utilize state-based anxiety and fatigue monitoring (SAFE) and randomized, regular feedback (FADE) to ensure ethical compliance, efficacy, and user satisfaction.
  • Data can be stored on wearable devices (e.g., wearable devices 10) mobile devices (e.g. mobile device 20), and/or other devices within a LAN or ad-hoc network of the wearable device/mobile device, and available for OLM application parsing or stakeholder review.
  • Owing to the data’s sensitive, contextual and personalized nature the majority of information can localized, and processed — wherever possible — only using edge and fog transmission as described above with reference to FIGs. 37 and 38A-38B.
  • Cloud processing, transmission, and storage can be minimized or avoided entirely to preserve privacy/security and ensure ethical robustness. Further security can be enabled through encryption/ decrypt! on policies that provide additional safeguarding layers whenever stakeholders review or process sensitive data.
  • FIG. 48 is a high level flow diagram conceptually illustrating the operation of a multi-sensory assistive wearable system, in accordance with some implementations of the disclosure.
  • An individual/wearer reacts to the environment.
  • MMLA sensors e.g., as incorporated in the wearable device and/or some other device in the user’s environment
  • collect data corresponding to ecological cues e.g., temperature data, image data, etc.
  • psychophysiological cues e.g., pupillary data, GSR data, heart rate data, etc.
  • the user’s sensitivity profile which can include thresholds and intervention/mediation preferences, is used to determine an intervention/media that is an alert, filter, guidance, or combination thereof.
  • a feedback loop can enable a constant and consistent pathway to unfold, whereby the individual’s responses are weighed against ecological and psychophysiological responses. Once a personalization threshold is exceeded, an assistive or mediative event occurs, and the system again monitors the individual’s response — weighing this against the current sensory input.
  • the individual can receive a mediation.
  • mediations are no longer effective or not enhancing a user’s experience, they can be disabled by the individual (e.g., via a user interface of the wearable device or mobile device) or any of the stakeholders.
  • cues that are no longer distracting can be removed from the identification process.
  • Tables 47A-47B show some example design specifications, including latency parameters, for implementing audiometric sensing, physiological/psychophysiological sensing, and transmission in accordance with some implementations of the disclosure. It should be appreciated that system specifications can vary depending on the available hardware.
  • the multi-sensory assistive wearable technology described herein can be utilized across a myriad of applications to supply a myriad of potential advantages.
  • the technology described herein can potentially reduce distractibility, improve attention and performance, lower anxiety, and/or increase employee output and/or satisfaction.
  • Metrics that could potentially be improved in the employment application include improved onboarding and training of neurodiverse, autistic, and neurotypical applicants and new hires, reduced employee turnover, increased productivity rate, diversity and/or inclusion, increased profit per employee, lowered healthcare costs, and/or ROI, employee net promoter score, cost of HR per employee, employee referral, combinations thereof and the like.
  • the technology described herein can potentially increase concentration and/or comprehension, and reduced, minimized and/or substantially eliminated hesitation and/or increased, enhanced and/or increased comfort.
  • Metrics that could potentially be improved in an academic application include retention rates (next term persistence versus resignation), graduation rates, time to completion, credits to degree and/or conferrals, academic performance, educational goal tracking, academic reputation, and/or underemployment of recent graduates.
  • Metrics that could potentially be improved in a social application include primary socialization (leam attitudes, values, and/or actions appropriate to individuals and culture), secondary socialization (leam behavior of smaller groups within society), developmental socialization (leam behavior in social institution and/or developing social skills), anticipatory socialization (rehearse future positions, occupations, and/or relationships), and resocialization (discarding former behavior and/or accepting new patterns as part of transitioning one’s life).
  • a transportation lorry/trucking application the technology described herein can potentially increase and/or improve attention and/or performance, reduce fatigue, and improve response times.
  • Metrics that could potentially be improved in a transportation lorry/trucking application include logistics benefits including increased safety and/or productivity (shut down engine, recommend rest, crash data statistics and/or analysis, etc.), reduced logistical strain and/or financial burden (reduced shipping, delivery time, and/or transportation costs), effective planning, dispatch, and/or scheduling.
  • the technology described herein can potentially increase focus and/or performance, and reduce fatigue and/or apprehension reduction.
  • Metrics that could potentially be improved in a transportation aircraft setting include safety (e.g., fatality and/or accident rate, system risk events, runway incursions, hazard risk mitigation, commercial space launch incidents, world-wide fatalities), efficiency (taxi-in/out time, gate arrival/delay, gate-to-gate times, distance at level-flight descent, flown v.
  • the technology described herein can potentially integrate mechanical and digital machines, objects, animals, and/or people (each with unique identifiers) received transferred information from the wearable so that actionable commands and/or analyses can occur.
  • Metrics that could potentially be improved in the loT application include an increase in physiological/psychophysiological activity can provide alerts to parents, caregivers, and/or professionals (para and otherwise) in the event wearable thresholds are exceeded.
  • Integration to environmental control units (ECU) bridge between the wearable and appliances including, but not limited to TV's, radios, lights, VCR's, motorized drapes, and/or motorized hospital beds, heating, and/or ventilation units (air-con), clothes washers and/or driers.
  • the technology described herein can potentially improve procrastination, mental health, fatigue, anxiety, and/or focus.
  • Metrics that could potentially be improved in a performance enhancement application include testing (logic processing, advocacy, curiosity, technical acumen and/or tenacity), Leadership (mentorship, subject matter expertise, team awareness, interpersonal skills, reliability), Strategy & Planning (desire, quality, community, knowledge and functionality), Intangibles (communication, diplomacy, negotiations, self-starter, confidence, maturity and selflessness).
  • the technology described herein can potentially improve the ability for medical and healthcare practitioners to share data with wearable users to help fine tune therapies, Rx, dispatch for emergency assist, surgical suite monitoring and/or optimization, work-schedule, and/or logistics strategy, pupillometry indicating unsafe conditions, unsafe warnings if thresholds are crossed (performance or physiological/psychophysiological).
  • Metrics that could potentially be improved in a telemedicine, emergency medicine, and/or healthcare application include telemedicine metrics (e.g., consultation time, diagnoses accuracy, rate of readmission, quality of service/technology, patient and/or clinician retention, time and/or travel saved, treatment plan adherence, patient referral), surgical metrics (e.g., first case starts, turnover times, location use/time, complications, value-based purchasing, consistency of service, outcomes), emergency metrics (e.g., average patient flow by hour, length of processing/stay, Time-to-Relative Value Unit, Patients Seen, RVU produced, Current Procedural Terminology (CPTs) performance, average evaluation and management distribution percentage, total number of deficient charts.
  • telemedicine metrics e.g., consultation time, diagnoses accuracy, rate of readmission, quality of service/technology, patient and/or clinician retention, time and/or travel saved, treatment plan adherence, patient referral
  • surgical metrics e.g., first case starts, turnover times, location use/time, complications, value-based purchasing, consistency
  • the technology described herein can potentially abet metric parenting (and guardianship) whereby work-life balance is made possible by meeting actionable and measurable goals and deadlines to improve family dynamics, including being more present, aware, and/or tracking engagement of children (particularly those with exceptionalities, although it is not limited to gifted, neurodiverse but all children).
  • Metrics that could potentially be improved in a parental, guardian, and/or educational monitoring application include family time, engagement, academic improvement, reduction in digital media technologies, screen time, online and console gaming, schedule adherence, nutritional faithfulness, safety and/or exposure to substance abuse, seizure and/or location monitoring.
  • machine readable medium In this document, the terms “machine readable medium,” “computer readable medium,” and similar terms are used to generally refer to non-transitory mediums, volatile or non-volatile, that store data and/or instructions that cause a machine to operate in a specific fashion. Common forms of machine-readable media include, for example, a hard disk, solid state drive, magnetic tape, or any other magnetic data storage medium, an optical disc or any other optical data storage medium, any physical medium with patterns of holes, a RAM, a PROM, EPROM, a FLASH-EPROM, NVRAM, any other memory chip or cartridge, and networked versions of the same.
  • Instructions can be grouped in the form of computer programs or other groupings. When executed, such instructions can enable a processing device to perform features or functions of the present application as discussed herein.
  • a “processing device” can be implemented as a single processor that performs processing operations or a combination of specialized and/or general- purpose processors that perform processing operations.
  • a processing device can include a CPU, GPU, APU, DSP, FPGA, ASIC, SOC, and/or other processing circuitry.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Epidemiology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Primary Health Care (AREA)
  • Physics & Mathematics (AREA)
  • Developmental Disabilities (AREA)
  • Psychiatry (AREA)
  • Biophysics (AREA)
  • Hospice & Palliative Care (AREA)
  • Child & Adolescent Psychology (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Data Mining & Analysis (AREA)
  • Educational Technology (AREA)
  • Databases & Information Systems (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

Système et procédé de fourniture de soulagement sensoriel contre la distraction, l'inattention, l'anxiété, la fatigue et/ou des problèmes sensoriels pour un utilisateur en ayant besoin. L'utilisateur peut être autiste/neurodivers ou neurotypique. Le système peut être configuré pour obtenir des données de sensibilité sensorielle d'utilisateur indiquant la sensibilité visuelle, sonique ou intéroceptive d'un utilisateur ; déterminer, à l'aide d'au moins les données de sensibilité sensorielle de l'utilisateur, des seuils sensoriels spécifiques à l'utilisateur et des données de médiation correspondant à des médiations spécifiques à l'utilisateur ; stocker les seuils sensoriels et les données de médiation ; enregistrer, à l'aide d'un ou de plusieurs capteurs, un stimulus d'entrée sensorielle à l'utilisateur ; comparer le stimulus d'entrée sensoriel aux seuils sensoriels ; en réponse à la comparaison du stimulus d'entrée sensoriel avec les seuils sensoriels, déterminer, sur la base au moins des données de médiation, une médiation à fournir à l'utilisateur, la médiation étant configurée pour fournir au soulagement de l'utilisateur contre la distraction, l'inattention, l'anxiété, la fatigue ou des problèmes sensoriels.
PCT/US2022/042100 2021-08-30 2022-08-30 Technologie portable multi-sensorielle d'assistance, et procédé de fourniture d'un soulagement sensoriel à l'aide de celle-ci Ceased WO2023034347A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CA3230610A CA3230610A1 (fr) 2021-08-30 2022-08-30 Technologie portable multi-sensorielle d'assistance, et procede de fourniture d'un soulagement sensoriel a l'aide de celle-ci
EP22865462.0A EP4396842A4 (fr) 2021-08-30 2022-08-30 Technologie portable multi-sensorielle d'assistance, et procédé de fourniture d'un soulagement sensoriel à l'aide de celle-ci
JP2024556032A JP2025500073A (ja) 2021-08-30 2022-08-30 多感覚で支援型のウェアラブル技術及びそれを用いた感覚的な解放を与える方法

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202163238490P 2021-08-30 2021-08-30
US63/238,490 2021-08-30
US17/882,517 2022-08-05
US17/882,517 US11779275B2 (en) 2021-08-05 2022-08-05 Multi-sensory, assistive wearable technology, and method of providing sensory relief using same

Publications (2)

Publication Number Publication Date
WO2023034347A1 true WO2023034347A1 (fr) 2023-03-09
WO2023034347A9 WO2023034347A9 (fr) 2024-06-06

Family

ID=85413066

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/042100 Ceased WO2023034347A1 (fr) 2021-08-30 2022-08-30 Technologie portable multi-sensorielle d'assistance, et procédé de fourniture d'un soulagement sensoriel à l'aide de celle-ci

Country Status (4)

Country Link
EP (1) EP4396842A4 (fr)
JP (1) JP2025500073A (fr)
CA (1) CA3230610A1 (fr)
WO (1) WO2023034347A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4294053A1 (fr) * 2022-06-14 2023-12-20 Sivantos Pte. Ltd. Procédé et système d'adaptation d'une prothèse auditive à un utilisateur
CN119992783A (zh) * 2025-02-25 2025-05-13 郑州铁路职业技术学院 一种基于儿童青少年近视预警方法及系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019165698A1 (fr) * 2018-03-02 2019-09-06 重庆邮电大学 Procédé de partage de sécurité de données dans un mode de collaboration de multiples nœuds de périphérie dans un environnement en nuage industriel
US10524715B2 (en) * 2013-10-09 2020-01-07 Nedim T. SAHIN Systems, environment and methods for emotional recognition and social interaction coaching
WO2020146749A1 (fr) * 2019-01-11 2020-07-16 Metafyre, Inc. Systèmes, dispositifs et procédés associés à des architectures d'automatisation et de commande intégrées de l'internet des objets

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018222589A1 (fr) * 2017-05-30 2018-12-06 A.Y.Y.T. - Technological Applications And Data Update, Ltd. Système et méthode de traitement de troubles au moyen d'un système de réalité virtuelle
US11534571B2 (en) * 2019-01-04 2022-12-27 Apollo Neuroscience, Inc. Systems and methods of facilitating sleep state entry with transcutaneous vibration

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10524715B2 (en) * 2013-10-09 2020-01-07 Nedim T. SAHIN Systems, environment and methods for emotional recognition and social interaction coaching
WO2019165698A1 (fr) * 2018-03-02 2019-09-06 重庆邮电大学 Procédé de partage de sécurité de données dans un mode de collaboration de multiples nœuds de périphérie dans un environnement en nuage industriel
WO2020146749A1 (fr) * 2019-01-11 2020-07-16 Metafyre, Inc. Systèmes, dispositifs et procédés associés à des architectures d'automatisation et de commande intégrées de l'internet des objets

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4396842A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP4294053A1 (fr) * 2022-06-14 2023-12-20 Sivantos Pte. Ltd. Procédé et système d'adaptation d'une prothèse auditive à un utilisateur
CN119992783A (zh) * 2025-02-25 2025-05-13 郑州铁路职业技术学院 一种基于儿童青少年近视预警方法及系统

Also Published As

Publication number Publication date
EP4396842A1 (fr) 2024-07-10
JP2025500073A (ja) 2025-01-07
WO2023034347A9 (fr) 2024-06-06
CA3230610A1 (fr) 2023-03-09
EP4396842A4 (fr) 2025-09-24

Similar Documents

Publication Publication Date Title
US10901509B2 (en) Wearable computing apparatus and method
US11779275B2 (en) Multi-sensory, assistive wearable technology, and method of providing sensory relief using same
US12208213B2 (en) Multi-sensory, assistive wearable technology, and method of providing sensory relief using same
US20240164672A1 (en) Stress detection
US12093457B2 (en) Creation of optimal working, learning, and resting environments on electronic devices
US20240115831A1 (en) Enhanced meditation experience based on bio-feedback
US20230072403A1 (en) Systems and methods for stroke care management
CN116997880A (zh) 注意力检测
WO2023034347A1 (fr) Technologie portable multi-sensorielle d'assistance, et procédé de fourniture d'un soulagement sensoriel à l'aide de celle-ci
Tillmann et al. The effect of visual perceptual load on auditory awareness in autism spectrum disorder
CN117677345A (zh) 基于生物反馈的增强型冥想体验
Arinta et al. Brain spatial reconciliation through multisensory integration in the impact of pandemic fatigue on workplace
Heck Presentation adaptation for multimodal interface systems: three essays on the effectiveness of user-centric content and modality adaptation
Singh Exploring the Future Trajectory of AI-Powered Hearables: Industry Perspectives on Emerging Capabilities
Gamage et al. Smart Glasses for CVI: Co-Designing Extended Reality Solutions to Support Environmental Perception by People with Cerebral Visual Impairment
US12393273B1 (en) Dynamic recording of an experience based on an emotional state and a scene understanding
US20250225415A1 (en) System and method for real-time biodata analysis and personalized response adaptation
Bayuaji Designing Compassionate Technology for Mental Health
McGowan et al. Gathering Requirements for Stress Management in Autistic Adults for the Development of Augmented Reality Self-Interventions
González-Lara et al. Identifying covert cognition in disorders of consciousness
Zou Guidelines for Designing Inclusive Wearable Devices for the Aging Population
CN117120958A (zh) 压力检测
한구현 Designing Personally Adaptable Ambient Information Visualizations for Individualized Everyday Contexts
Halder et al. Mitigating the Challenges for AI-Assisted Intervention in Mental Health
Retallic et al. Health and well-being in eye care practice: Negotiating with your patient's ocular health and well-being

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22865462

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 3230610

Country of ref document: CA

WWE Wipo information: entry into national phase

Ref document number: 2022865462

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

ENP Entry into the national phase

Ref document number: 2022865462

Country of ref document: EP

Effective date: 20240402

ENP Entry into the national phase

Ref document number: 2024556032

Country of ref document: JP

Kind code of ref document: A