WO2025221862A1

WO2025221862A1 - Incident diagnosis with retrieval augmented llm for medical decision making

Info

Publication number: WO2025221862A1
Application number: PCT/US2025/024925
Authority: WO
Inventors: Peng Yuan; Luan Tang; Haifeng Chen; Yanchi Liu; Motoyuki Sato
Original assignee: NEC Laboratories America Inc
Current assignee: NEC Laboratories America Inc
Priority date: 2024-04-16
Filing date: 2025-04-16
Publication date: 2025-10-23
Anticipated expiration: 2026-10-16

Abstract

Methods and systems for anomaly analysis include generating (502) an incident timeline graph for an anomaly, based on a graph of relationships sensors and temporal information relating to anomalous sensor readings. Documents are retrieved (504) relating to the anomaly using a signature based on the incident timeline graph. A prompt is generated (506) using an anomaly description based on the incident timeline graph and examples taken from the retrieved documents. A report is generated (508) describing the incident using with the prompt as an input to a large language model. An action is performed (510) responsive to the anomaly based on information in the report.

Description

INCIDENT DIAGNOSIS WITH RETRIEVAL AUGMENTED LLM FOR MEDICAL DECISION MAKING

RELATED APPLICATION INFORMATION

[0001] This application claims priority to U.S. Patent Application No. 63/634,570, filed on April 16, 2024, and U.S. Patent Application No. 19/179,284 filed on April 15, 2025, both incorporated herein by reference in their entirety.

BACKGROUND

Technical Field

[0002] The present invention relates to anomaly detection and, more particularly, anomaly detection using retrieval augmented large language models.

Description of the Related Art

[0003] Internet of Things (loT) systems may include a number of individual sensors, each producing data over time to monitor the functioning and behavior of a complex system. This multivariate time series data may be used to analyze abnormal sensor records to generate an incident report. Such data may quickly grow to large volumes, making it impossible for domain experts to analyze.

SUMMARY

[0004] A method for anomaly analysis includes generating an incident timeline graph for an anomaly, based on a graph of relationships sensors and temporal information relating to anomalous sensor readings. Documents are retrieved relating to the anomaly using a signature based on the incident timeline graph. A prompt is generated using an anomaly description based on the incident timeline graph and examples taken from the retrieved documents. A report is generated describing the incident using with the prompt as an input to a large language model. An action is performed responsive to the anomaly based on information in the report.

[0005] A system for anomaly analysis includes a hardware processor and a memory that stores a computer program. When executed by the hardware processor, the computer program causes the hardware processor to generate an incident timeline graph for an anomaly, based on a graph of relationships between sensors and temporal information relating to anomalous sensor readings, to retrieve documents relating to the anomaly using a signature based on the incident timeline graph, to generate a prompt using an anomaly description based on the incident timeline graph and examples taken from the retrieved documents, to generate a report describing the incident using with the prompt as an input to a large language model, and to perform an action responsive to the anomaly based on information in the report.

[0006] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.

BRIEF DESCRIPTION OF DRAWINGS

[0007] The disclosure will provide details in the following description of preferred embodiments with reference to the following figures wherein:

[0008] FIG. 1 is a diagram illustrating a maintenance system that monitors a cyberphysical system using a set of sensors and a retrieval-augmented large language model (LLM), in accordance with an embodiment of the present invention;

[0009] FIG. 2 is a block diagram illustrating a process for report generation that includes retrieval augmented prompting to an LLM, in accordance with an embodiment of the present invention; [0010] FIG. 3 is a block/flow diagram of a method for identifying anomalies and creating a temporal graph framework, in accordance with an embodiment of the present invention;

[0011] FIG. 4 is a block/flow diagram of a method for ranking anomaly scores, in accordance with an embodiment of the present invention;

[0012] FIG. 5 is a block/flow diagram of a method for reporting on and responding to an anomaly using a retrieval augmented LLM, in accordance with an embodiment of the present invention;

[0013] FIG. 6 is a diagram of an exemplary neural network architecture that can be used to implement part of an LLM, in accordance with an embodiment of the present invention;

[0014] FIG. 7 is a diagram of an exemplary deep neural network architecture that can be used to implement part of an LLM, in accordance with an embodiment of the present invention;

[0015] FIG. 8 is a block diagram of a computing device that can analyze anomalies with a retrieval augmented LLM, in accordance with an embodiment of the present invention; and

[0016] FIG. 9 is a block diagram of a healthcare facility that uses anomaly analysis and treatment recommendation for making treatment recommendations, in accordance with an embodiment of the present invention.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0017] Large language models (LLMs) may be used to analyze large volumes of multivariate time series data. Such LLMs may identify abnormal behavior and can produce an explanation of what factors led to the detection of an anomaly. However, applying LLMs in this context can be challenging due to a lack of background knowledge of a deployed Internet of Things (loT) system and because the incidents may be complex events that affect many sensors and system components.

[0018] A retrieval augmented LLM may therefore be used to provide incident diagnosing and reporting for an loT system. Related system documents, based on incident features, are used to provide context to the LLM to analyze detected anomalies. The inclusion of this background information helps the LLM to identify root causes of the anomalies and to automatically generate incident reports. The LLM may furthermore identify a corrective action and may even automatically implement that corrective action by issuing instructions to components of the monitored system.

[0019] To that end, a graph is constructed from abnormal records to capture sensor interactions and dependencies. Features are generated that represent the incident timeline and any known influences. These features are used as a key to query and retrieve relevant documents, such as sensor descriptions, operation manuals, incident tickets, and historical reports. Prompts are generated for the LLM based on the retrieved documents and anomaly features. By incorporating such context-aware prompts, the LLM’s ability to understand loT-specific environments is improved. Incident reports and responsive instructions may then be generated automatically.

[0020] Referring now in detail to the figures in which like numerals represent the same or similar elements and initially to FIG. 1, a maintenance system 106 in the context of a monitored system 102 is shown. The monitored system 102 can be any appropriate system, including physical systems such as manufacturing lines and physical plant operations, electronic systems such as computers or other computerized devices, software systems such as operating systems and applications, and cyberphysical systems that combine physical systems with electronic systems and/or software systems. Exemplary systems 102 may include a wide range of different types, including railroad systems, power plants, vehicle sensors, data centers, satellites, and transportation systems. Another type of cyber-physical system can be a network of internet of things (loT) devices, which may include a wide variety of different types of devices, with various respective functions and sensor types.

[0021] One or more sensors 104 record information about the state of the monitored system 102. The sensors 104 can be any appropriate type of sensor including, for example, physical sensors, such as temperature, humidity, vibration, pressure, voltage, current, magnetic field, electrical field, and light sensors, and software sensors, such as logging utilities installed on a computer system to record information regarding the state and behavior of the operating system and applications running on the computer system. The sensor data may include, e.g., numerical data and categorical or binaryvalued data. The information generated by the sensors 104 can be in any appropriate format and can include sensor log information generated with heterogeneous formats.

[0022] The sensors 104 may transmit the logged sensor information to an anomaly maintenance system 106 by any appropriate communications medium and protocol, including wireless and wired communications. The maintenance system 106 can, for example, identify abnormal or anomalous behavior by monitoring the multivariate time series that are generated by the sensors 104. Once anomalous behavior has been detected, the maintenance system 106 communicates with a system control unit to alter one or more parameters of the monitored system 102 to correct the anomalous behavior. [0023] Exemplary corrective actions include changing a security setting for an application or hardware component, changing an operational parameter of an application or hardware component (for example, an operating speed), halting and/or restarting an application, halting and/or rebooting a hardware component, changing an environmental condition, changing a network interface’s status or settings, etc. The maintenance system 106 thereby automatically corrects or mitigates the anomalous behavior. By identifying the particular sensors 104 that are associated with the anomalous classification, the amount of time needed to isolate a problem can be decreased.

[0024] Each of the sensors 104 outputs a respective time series, which encodes measurements made by the sensor over time. For example, the time series may include pairs of information, with each pair including a measurement and a timestamp, representing the time at which the measurement was made. Each time series may be divided into segments, which represent measurements made by the sensor over a particular time range. Time series segments may represent any appropriate interval, such as one second, one minute, one hour, or one day. Time series segments may represent a set number of collection time points, rather than a fixed period of time, for example covering 100 measurements.

[0025] Sensors 104 may collect information about conditions of the system 106, such as information that relates to system control and operation mode. Sensors 104 may also collect information relating to key performance indicators (KPIs) such as temperature, humidity, motion, and pressure to characterize the system health and key parameters.

[0026] The maintenance system 106 includes retrieval augmented LLM 108, which helps to detect and analyze anomalous behavior. The retrieval augmented LLM 108 may have access to documentation relating to the monitored system 102 and to the sensors 104, including specifications of different components, expected behavior, troubleshooting information, and historical records. This information helps the retrieval augmented LLM 108 to identify problems and report them. [0027] In some cases, the monitored system may not be a cyber-physical system, but may instead by a biological system, such as a patient. In such embodiments, the sensors 104 may collect information relating to the patient’s physical condition and medical status (e.g., temperature, blood pressure, heart rate, test results, etc.). The sensors 104 may thus include, for example, pulse oximeters, thermometers, oxygen saturation sensors, electrocardiogram sensors, etc. An anomaly may indicate an abnormal condition for the patient, such as a disease, injury, or other health condition. Corrective actions in such embodiments may include administering a treatment to the patient, for example by automatically triggering an intravenous medication.

[0028] Referring now to FIG. 2, a retrieval augmented anomaly analysis system is shown. Incident analysis 200 identifies anomalous behavior, for example based on new sensor data and historical patterns. Document retrieval 210 uses information generated by the incident analysis 200 to search historical reports and other documents to generate context for report generation 220. Report generation uses the anomaly information and any documents found by document retrieval 210 to create a prompt for LLM 228.

[0029] In more detail, incident analysis 200 takes reported anomalies 204 from deployed sensors 104 as input, after anomaly noise removal and clustering based on sensor and time. These sensors 104 may be developed based on different mechanisms, for example with some focusing on numerical values and with others generating binary or categorical values. The sensors’ relationships to one another may be represented as a sensor graph 202 generated from relation analysis on historical sensor data to capture the correlation of the sensor readings or categorical values with time stamps.. The sensor relationship graph 202 and the anomaly clusters 204 are combined to generate an incident timeline graph 206. [0030] Document retrieval 210 retrieves incident signatures 214 from the incident timeline graph 206, including key features to identify each incident. The key features may include the top k abnormal sensors and their corresponding anomaly sequences. Such features are used as a query to a document database 212. The document database 212 may include historical tickets and reports as well as sensor information and operation manuals relating to the monitored system 102 and the sensors 104. Documents that most closely match the query are retrieved as related documents 216.

[0031] Report generation 220 uses the incident timeline graph 206 to generate an anomaly description 222. The retrieved documents 216 are used to create in-context examples 224. The anomaly description and the in-context examples 224 are combined into incident prompts 226, which are used as input to LLM 228. The LLM 228 can then generate a report on the anomaly, for example including a description of the anomaly, possible root causes, and instructions on how to correct the anomaly.

[0032] The sensor graph 202 captures relationships between the sensors 104. The sensors 104 may be installed in a system to oversee different components or subsystems, and sensors 104 in a same component or sub-system may have outputs that are correlated to one another. However, the group and relationships of sensors 104 is not necessarily provided in advance.

[0033] Sensor correlations between sensors x and y may be determined as a Pearson correlation: where x and y are the average values of the readings of sensors x and y, respectively, and n is a number of measurements. [0034] It can be difficult to compute the correlation among categorical sensors using the Pearson correlation, since the categorical readings are not necessarily represented with numerical values. However, these categorical values may be converted to numerical values.

[0035] A sensor graph is initialized with nodes representing each of the sensors 104. The Peason correlation is calculated between each pair of sensors. If the correlation is larger than a threshold 8_t, then an edge is added to the graph between the respective nodes. In most cases, the sensor signals within a same sub-system may have strong correlations with one another, while signals from different sub-systems may have low correlations. Sub-system divisions can then be identified by correlation analysis on historical data, with the sensor graph being the output.

[0036] For example, in a relational graph showing the different sensors within a car, there may be fifty sensors represented as nodes. Exemplary sub-systems would include the engine, door, battery, audio, mirror, and airbag sub-systems. Sensors within a same sub-system are likely to be connected to one another, and clusters of densely interconnected sensors can thereby be identified as sub-systems.

[0037] The monitoring system 106 processes data generated by the sensors 104 and detects anomalies for each sensor 104 in a continuous manner. The detection results may be reported as a matrix, where columns represent the sensors 104 and rows represent detection results at different times, for example stored as binary values, where a 1 indicates that an anomaly was detected in a given sensor 104 at a given time. In some embodiments the anomaly values may instead indicate a frequency of the anomaly, where 0 indicates normal behavior and 1-4 indicates differing frequencies. For example, a frequently occurring anomaly may be designated with a 1, while a rarely occurring anomaly may be designated with a 4. This matrix may be used to generate an alignment of abnormal sensors as a framework for a temporal graph, as discussed in greater detail below.

[0038] Using the sensor graph 202 and the temporal graph framework of incidents, the incident analysis 200 can produce incident timeline graph 206. The sensor graph 202 includes relationship edges for the sensors 104. The set of sensors 104 may be denoted as .S'. For each incident, there is a temporal graph framework that includes abnormal sensor set S_t, being a subset of .S'. The abnormal sensor set S_t is matched to the sensor graph G, retrieving all the edges related to S_t . The edges are added to the framework and are integrated as the incident timeline graph G_t . The nodes and alignment of the incident timeline graph 206 are from the temporal framework and the edges are added from the relationship graph. The interval between nodes on an x axis indicates the time difference between the anomalies.

[0039] For document retrieval 210, the signatures of the incident from each timeline are retrieved as the top k sensors in the incident. This ranking may be made according to a severity determination for the sensors. The assessment of an abnormal sensor’s severity can be represented using the temporal duration of the anomaly, the strength of the anomaly, and the reliability of the sensor itself. The temporal duration measures the length of the sensor’s anomaly. The strength of the anomaly may be determined by evaluating the significance of the abnormal signal in comparison to normal baselines. The reliability of the sensor can be established by examining whether the sensor only reports anomalies during system-level alert periods, or whether it also generates numerous false positives during normal operations. Additional detail on generating sensor abnormality scores is described below.

[0040] For report generation 220, a user may upload detected anomalies of the system incidents, or such incidents may be automatically loaded. The time period of the incident and the detailed anomalies are shown, along with overall anomaly scores and associated times. Anomaly data may be cleaned (e.g., denoised) and clustered to generate an anomaly timeline for visualization. The report may be generated automatically or responsive to a user’s instruction.

[0041] An incident report may include similar incidents drawn from a historical database. The retrieved incidents may be used to generate a prompt for the LLM 228, including background information and examples 224 from historical incidents, as well as anomaly descriptions 222 from the incident timeline graph 206. The LLM 228 can then generate a formal incident report and can furthermore generate instructions for a downstream response to the incident.

[0042] Referring now to FIG. 3, additional detail on detecting anomalies and generating the temporal graph is shown. An incident is defined as a system-level abnormal event that involves multiple sensors 104. Block 302 computes an incident threshold for the number of sensors that report abnormal signals. This includes checking the detection results during a period of normal operation and maintaining a maximum number of abnormal sensors. The incident threshold may then be computed as:

6 = (1 + a) • max(abnormal_sensor) where a is a safe buffer (e.g., 20%). When testing in a normal period, the threshold indicates a maximum number of anomalies in a time window that may result from normal dynamics (e.g., noise). This expression of the threshold sets a safe buffer higher than that maximum number. When a number of anomalies greater than the threshold is detected, then it can be determined that the system is in an anomalous state.

[0043] The incident threshold is applied to all detection results to generate incident candidate periods in block 304. A candidate period T_t may be defined by the range {start end-i}. In some cases, the incident threshold may be adjusted by a user and/or the list of identified incident periods may be manually adjusted by adding or deleting periods. Block 306 scans all candidate periods sequentially and checks each pair. If T_t and T_i+1 are close in time (e.g., start_i+1 — endi < 8_t , where 8_t is a predetermined time threshold), then block 306 merges T_t and T_i+1.

[0044] After an incident period is detected, information is retrieved from all sensors indicated as having abnormal signals. This information is aligned by time in block 308. Block 310 then outputs this temporal graph framework. The sensor nodes are aligned by the time of the first anomaly during the incident period. An x-axis may indicate time, with start and end points being the starting and ending times of the incident period. The intervale of the sensor nodes correspond to the time interval of their anomalies. There are no edges in this graph yet, as these are added later.

[0045] Referring now to FIG. 4, additional detail on computing abnormality scores is shown. Block 402 accumulates anomaly strength information from the sensors 104 over an incident period. This measure is represented as Ai. For example: where n is a number of abnormal windows, a_n is the n¹¹¹ abnormal sensor value, t_window is the length of each time window, and t_totai is a total time period over which the abnormal values are measured.

[0046] The anomaly strength information from the sensors 104 is also collected from a period of normal operation, which can include false positives and which is represented as Ai. This accumulated normal signal strength may be calculated as: where m is a number of false positives and o_m is the in"' false positive sensor value. [0047] The first measure Ai may be higher in proportion to the length of the anomalous activity, which can be confused with a more significant abnormal signal. Block 404 therefore normalizes the score for the incident as:

A_s = e ^{1 -H4}2

Block 406 then ranks the sensors according to the normalized anomaly scores, in particular identifying the top k scores. These sensors are those which have the most significant anomalous activity over the longest duration during the system alert period (Ai). These sensors should also have a low number of false positives during the system’ s normal period, according to their low Ai scores.

[0048] Referring now to FIG. 5, a method for analyzing and responding to an anomalous incident is shown. Block 502 generates the incident timeline graph, collecting sensor information relating to an anomaly and arranging it in a graph that captures temporal information as multiple sensors exhibit anomalous behavior. Block 504 retrieves related documents, using the temporal information graph to generate incident signatures that are used to query a database for information relating to the sensors, the system, and historical incident records.

[0049] Block 506 generates a prompt, combining an anomaly description 222 from the incident timeline graph 206 and in-context examples 224 drawn from the related documents 216. This prompt is used as input to LLM 228 in block 508, which provides an analysis of the incident and can further generate instructions on how best to respond. A user may then adjust parameters of the report, for example by adjusting the anomaly score threshold and/or manually adding or removing anomalies. This may cause the prompt to be regenerated in block 506, providing an updated output of the LLM 228.

[0050] Block 510 then performs a responsive action to address or correct the anomaly. As noted above, the action can include changing an operational state of a hardware component of the monitored system 102 or an environmental condition. The action can be performed automatically, for example triggering software through an application programming interface (API) or changing a configuration setting.

[0051] Referring now to FIGs. 6 and 7, exemplary neural network architectures are shown, which may be used to implement parts of the present models, such as the LLM 228. A neural network is a generalized system that improves its functioning and accuracy through exposure to additional empirical data. The neural network becomes trained by exposure to the empirical data. During training, the neural network stores and adjusts a plurality of weights that are applied to the incoming empirical data. By applying the adjusted weights to the data, the data can be identified as belonging to a particular predefined class from a set of classes or a probability that the input data belongs to each of the classes can be output.

[0052] The empirical data, also known as training data, from a set of examples can be formatted as a string of values and fed into the input of the neural network. Each example may be associated with a known result or output. Each example can be represented as a pair, (x, y), where x represents the input data and y represents the known output. The input data may include a variety of different data types, and may include multiple distinct values. The network can have one input node for each value making up the example’s input data, and a separate weight can be applied to each input value. The input data can, for example, be formatted as a vector, an array, or a string depending on the architecture of the neural network being constructed and trained.

[0053] The neural network “learns” by comparing the neural network output generated from the input data to the known values of the examples, and adjusting the stored weights to minimize the differences between the output values and the known values. The adjustments may be made to the stored weights through back propagation, where the effect of the weights on the output values may be determined by calculating the mathematical gradient and adjusting the weights in a manner that shifts the output towards a minimum difference. This optimization, referred to as a gradient descent approach, is a non-limiting example of how training may be performed. A subset of examples with known values that were not used for training can be used to test and validate the accuracy of the neural network.

[0054] During operation, the trained neural network can be used on new data that was not previously used in training or validation through generalization. The adjusted weights of the neural network can be applied to the new data, where the weights estimate a function developed from the training examples. The parameters of the estimated function which are captured by the weights are based on statistical inference. [0055] In layered neural networks, nodes are arranged in the form of layers. An exemplary simple neural network has an input layer 620 of source nodes 622, and a single computation layer 630 having one or more computation nodes 632 that also act as output nodes, where there is a single computation node 632 for each possible category into which the input example could be classified. An input layer 620 can have a number of source nodes 622 equal to the number of data values 612 in the input data 610. The data values 612 in the input data 610 can be represented as a column vector. Each computation node 632 in the computation layer 630 generates a linear combination of weighted values from the input data 610 fed into input nodes 620, and applies a non-linear activation function that is differentiable to the sum. The exemplary simple neural network can perform classification on linearly separable examples (e.g., patterns).

[0056] A deep neural network, such as a multilayer perceptron, can have an input layer 620 of source nodes 622, one or more computation layer(s) 630 having one or more computation nodes 632, and an output layer 640, where there is a single output node 642 for each possible category into which the input example could be classified. An input layer 620 can have a number of source nodes 622 equal to the number of data values 612 in the input data 610. The computation nodes 632 in the computation layer(s) 630 can also be referred to as hidden layers, because they are between the source nodes 622 and output node(s) 642 and are not directly observed. Each node 632, 642 in a computation layer generates a linear combination of weighted values from the values output from the nodes in a previous layer, and applies a non-linear activation function that is differentiable over the range of the linear combination. The weights applied to the value from each previous node can be denoted, for example, by wi, W2, ... Wn-i, Wn. The output layer provides the overall response of the network to the input data. A deep neural network can be fully connected, where each node in a computational layer is connected to all other nodes in the previous layer, or may have other configurations of connections between layers. If links between nodes are missing, the network is referred to as partially connected.

[0057] Training a deep neural network can involve two phases, a forward phase where the weights of each node are fixed and the input propagates through the network, and a backwards phase where an error value is propagated backwards through the network and weight values are updated.

[0058] The computation nodes 632 in the one or more computation (hidden) layer(s) 630 perform a nonlinear transformation on the input data 612 that generates a feature space. The classes or categories may be more easily separated in the feature space than in the original data space.

[0059] Referring now to FIG. 8, an exemplary computing device 800 is shown, in accordance with an embodiment of the present invention. The computing device 800 may be embodied as any type of computation or computer device capable of performing the functions described herein, including, without limitation, a computer, a server, a rack based server, a blade server, a workstation, a desktop computer, a laptop computer, a notebook computer, a tablet computer, a mobile computing device, a wearable computing device, a network appliance, a web appliance, a distributed computing system, a processor-based system, and/or a consumer electronic device. Additionally or alternatively, the computing device 800 may be embodied as one or more compute sleds, memory sleds, or other racks, sleds, computing chassis, or other components of a physically disaggregated computing device.

[0060] As shown in FIG. 8, the computing device 800 illustratively includes the processor 810, an input/output subsystem 820, a memory 830, a data storage device 840, and a communication subsystem 850, and/or other components and devices commonly found in a server or similar computing device. The computing device 800 may include other or additional components, such as those commonly found in a server computer (e.g., various input/output devices), in other embodiments. Additionally, in some embodiments, one or more of the illustrative components may be incorporated in, or otherwise form a portion of, another component. For example, the memory 830, or portions thereof, may be incorporated in the processor 810 in some embodiments.

[0061] The processor 810 may be embodied as any type of processor capable of performing the functions described herein. The processor 810 may be embodied as a single processor, multiple processors, a Central Processing Unit(s) (CPU(s)), a Graphics Processing Unit(s) (GPU(s)), a single or multi-core processor(s), a digital signal processor(s), a microcontroller(s), or other processor(s) or processing/controlling circuit(s). [0062] The memory 830 may be embodied as any type of volatile or non-volatile memory or data storage capable of performing the functions described herein. In operation, the memory 830 may store various data and software used during operation of the computing device 800, such as operating systems, applications, programs, libraries, and drivers. The memory 830 is communicatively coupled to the processor 810 via the I/O subsystem 820, which may be embodied as circuitry and/or components to facilitate input/output operations with the processor 810, the memory 830, and other components of the computing device 800. For example, the VO subsystem 820 may be embodied as, or otherwise include, memory controller hubs, input/output control hubs, platform controller hubs, integrated control circuitry, firmware devices, communication links (e.g., point-to-point links, bus links, wires, cables, light guides, printed circuit board traces, etc.), and/or other components and subsystems to facilitate the input/output operations. In some embodiments, the VO subsystem 820 may form a portion of a system-on-a-chip (SOC) and be incorporated, along with the processor 810, the memory 830, and other components of the computing device 800, on a single integrated circuit chip.

[0063] The data storage device 840 may be embodied as any type of device or devices configured for short-term or long-term storage of data such as, for example, memory devices and circuits, memory cards, hard disk drives, solid state drives, or other data storage devices. The data storage device 840 can store program code 840A for incident analysis, 840B for document retrieval, and/or 840C for report generation. Any or all of these program code blocks may be included in a given computing system. The communication subsystem 850 of the computing device 800 may be embodied as any network interface controller or other communication circuit, device, or collection thereof, capable of enabling communications between the computing device 800 and other remote devices over a network. The communication subsystem 850 may be configured to use any one or more communication technology (e.g., wired or wireless communications) and associated protocols (e.g., Ethernet, InfiniBand®, Bluetooth®, Wi-Fi®, WiMAX, etc.) to effect such communication.

[0064] As shown, the computing device 800 may also include one or more peripheral devices 860. The peripheral devices 860 may include any number of additional input/output devices, interface devices, and/or other peripheral devices. For example, in some embodiments, the peripheral devices 860 may include a display, touch screen, graphics circuitry, keyboard, mouse, speaker system, microphone, network interface, and/or other input/output devices, interface devices, and/or peripheral devices.

[0065] Of course, the computing device 800 may also include other elements (not shown), as readily contemplated by one of skill in the art, as well as omit certain elements. For example, various other sensors, input devices, and/or output devices can be included in computing device 800, depending upon the particular implementation of the same, as readily understood by one of ordinary skill in the art. For example, various types of wireless and/or wired input and/or output devices can be used. Moreover, additional processors, controllers, memories, and so forth, in various configurations can also be utilized. These and other variations of the processing system 800 are readily contemplated by one of ordinary skill in the art given the teachings of the present invention provided herein.

[0066] Referring now to FIG. 9, a diagram of information extraction is shown in the context of a healthcare facility 900. Anomaly analysis and treatment recommendation 908 may be used to process present sensor information and past medical records 906 from a patient to identify and analyze a patient’ s health conditions. The report generated may include specific treatment recommendations and, in some cases, can include automatic triggers for treatment actions.

[0067] The healthcare facility may include one or more medical professionals 902 who review information extracted from a patient’s medical records 906 to determine their healthcare and treatment needs. These medical records 906 may include selfreported information from the patient, test results, and notes by healthcare personnel made to the patient’s file. Treatment systems 904 may furthermore monitor patient status to generate medical records 906 and may be designed to automatically administer and adjust treatments as needed.

[0068] Based on information provided by the anomaly analysis and treatment recommendation 908, the medical professionals 902 may make medical decisions about patient healthcare suited to the patient’s needs. For example, the medical professionals 902 may make a diagnosis of the patient’ s health condition and may prescribe particular medications, surgeries, and/or therapies.

[0069] The different elements of the healthcare facility 900 may communicate with one another via a network 910, for example using any appropriate wired or wireless communications protocol and medium. Thus the anomaly analysis and treatment recommendation 908 can monitor health information from a patient may formulate a response based on information gleaned from stored medical records 906. The anomaly analysis and treatment recommendation 908 may coordinate with treatment systems 904 in some cases to automatically administer or alter a treatment. For example, if the anomaly analysis and treatment recommendation 908 indicates a particular disease or condition, then the treatment systems 904 may automatically begin or halt the administration of the treatment. [0070] Embodiments described herein may be entirely hardware, entirely software or including both hardware and software elements. In a preferred embodiment, the present invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0071] Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. A computer-usable or computer readable medium may include any apparatus that stores, communicates, propagates, or transports the program for use by or in connection with the instruction execution system, apparatus, or device. The medium can be magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. The medium may include a computer-readable storage medium such as a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk, etc.

[0072] Each computer program may be tangibly stored in a machine-readable storage media or device (e.g., program memory or magnetic disk) readable by a general or special purpose programmable computer, for configuring and controlling operation of a computer when the storage media or device is read by the computer to perform the procedures described herein. The inventive system may also be considered to be embodied in a computer-readable storage medium, configured with a computer program, where the storage medium so configured causes a computer to operate in a specific and predefined manner to perform the functions described herein.

[0073] A data processing system suitable for storing and/or executing program code may include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code to reduce the number of times code is retrieved from bulk storage during execution. Input/output or I/O devices (including but not limited to keyboards, displays, pointing devices, etc.) may be coupled to the system either directly or through intervening I/O controllers.

[0074] Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.

[0075] As employed herein, the term “hardware processor subsystem” or “hardware processor” can refer to a processor, memory, software or combinations thereof that cooperate to perform one or more specific tasks. In useful embodiments, the hardware processor subsystem can include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution devices, etc.). The one or more data processing elements can be included in a central processing unit, a graphics processing unit, and/or a separate processor- or computing element-based controller (e.g., logic gates, etc.). The hardware processor subsystem can include one or more on-board memories (e.g., caches, dedicated memory arrays, read only memory, etc.). In some embodiments, the hardware processor subsystem can include one or more memories that can be on or off board or that can be dedicated for use by the hardware processor subsystem (e.g., ROM, RAM, basic input/output system (BIOS), etc.).

[0076] In some embodiments, the hardware processor subsystem can include and execute one or more software elements. The one or more software elements can include an operating system and/or one or more applications and/or specific code to achieve a specified result.

[0077] In other embodiments, the hardware processor subsystem can include dedicated, specialized circuitry that performs one or more electronic processing functions to achieve a specified result. Such circuitry can include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and/or programmable logic arrays (PLAs).

[0078] These and other variations of a hardware processor subsystem are also contemplated in accordance with embodiments of the present invention.

[0079] Reference in the specification to “one embodiment” or “an embodiment” of the present invention, as well as other variations thereof, means that a particular feature, structure, characteristic, and so forth described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase “in one embodiment” or “in an embodiment”, as well any other variations, appearing in various places throughout the specification are not necessarily all referring to the same embodiment. However, it is to be appreciated that features of one or more embodiments can be combined given the teachings of the present invention provided herein.

[0080] It is to be appreciated that the use of any of the following “/”, “and/or”, and “at least one of’, for example, in the cases of “A/B”, “A and/or B” and “at least one of A and B”, is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of both options (A and B). As a further example, in the cases of “A, B, and/or C” and “at least one of A, B, and C”, such phrasing is intended to encompass the selection of the first listed option (A) only, or the selection of the second listed option (B) only, or the selection of the third listed option (C) only, or the selection of the first and the second listed options (A and B) only, or the selection of the first and third listed options (A and C) only, or the selection of the second and third listed options (B and C) only, or the selection of all three options (A and B and C). This may be extended for as many items listed.

[0081] The foregoing is to be understood as being in every respect illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is not to be determined from the Detailed Description, but rather from the claims as interpreted according to the full breadth permitted by the patent laws. It is to be understood that the embodiments shown and described herein are only illustrative of the present invention and that those skilled in the art may implement various modifications without departing from the scope and spirit of the invention. Those skilled in the art could implement various other feature combinations without departing from the scope and spirit of the invention. Having thus described aspects of the invention, with the details and particularity required by the patent laws, what is claimed and desired protected by

Letters Patent is set forth in the appended claims.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method for anomaly analysis, comprising: generating (502) an incident timeline graph for an anomaly, based on a graph of relationships between a plurality of sensors and temporal information relating to anomalous sensor readings; retrieving (504) documents relating to the anomaly using a signature based on the incident timeline graph; generating (506) a prompt using an anomaly description based on the incident timeline graph and examples taken from the retrieved documents; generating (508) a report describing the incident using with the prompt as an input to a large language model; and performing (510) an action responsive to the anomaly based on information in the report.

2. The method of claim 1, wherein the documents include documentation regarding the sensors.

3. The method of claim 1, wherein the documents include historical anomaly reports.

4. The method of claim 1, further comprising generating the signature by identifying a subset of the sensors that have a highest anomaly score.

5. The method of claim 1, wherein generating the incident timeline graph includes identifying incident candidate periods using a threshold value based on a maximum number of abnormal sensors.

6. The method of claim 5, wherein generating the incident timeline graph further includes merging candidate periods according to a threshold time difference.

7. The method of claim 1, further comprising generating the graph of relationships based on a correlation analysis of outputs of the plurality of sensors.

8. The method of claim 1, wherein the large language model is implemented as a machine learning model.

9. The method of claim 1, wherein the report is used to assist a medical professional in medical decision making.

10. The method of claim 1, wherein the plurality of sensors collect medical information from a patient and wherein the action includes automatically altering a treatment for the patient.

11. A system for anomaly analysis, comprising: a hardware processor (810); and a memory (840) that stores a computer program which, when executed by the hardware processor, causes the hardware processor to: generate (502) an incident timeline graph for an anomaly, based on a graph of relationships between a plurality of sensors and temporal information relating to anomalous sensor readings; retrieve (504) documents relating to the anomaly using a signature based on the incident timeline graph; generate (506) a prompt using an anomaly description based on the incident timeline graph and examples taken from the retrieved documents; generate (508) a report describing the incident using with the prompt as an input to a large language model; and perform (510) an action responsive to the anomaly based on information in the report.

12. The system of claim 11, wherein the documents include documentation regarding the sensors.

13. The system of claim 11, wherein the documents include historical anomaly reports.

14. The system of claim 11, wherein the computer program further causes the hardware processor to identify a subset of the sensors that have a highest anomaly score.

15. The system of claim 11, wherein generation of the incident timeline graph includes identification of incident candidate periods using a threshold value based on a maximum number of abnormal sensors.

16. The system of claim 15, wherein generation of the incident timeline graph further includes merging candidate periods according to a threshold time difference.

17. The system of claim 11, wherein the computer program further causes the hardware processor to generate the graph of relationships based on a correlation analysis of outputs of the plurality of sensors.

18. The system of claim 11, wherein the large language model is implemented as a machine learning model.

19. The system of claim 11, wherein the report is used to assist a medical professional in medical decision making.

20. The system of claim 11, wherein the plurality of sensors collect medical information from a patient and wherein the action includes automatically altering a treatment for the patient.