[go: up one dir, main page]

WO2018171940A1 - Dispositif et procédé de détermination d'un état d'un cycle d'opérations - Google Patents

Dispositif et procédé de détermination d'un état d'un cycle d'opérations Download PDF

Info

Publication number
WO2018171940A1
WO2018171940A1 PCT/EP2018/000108 EP2018000108W WO2018171940A1 WO 2018171940 A1 WO2018171940 A1 WO 2018171940A1 EP 2018000108 W EP2018000108 W EP 2018000108W WO 2018171940 A1 WO2018171940 A1 WO 2018171940A1
Authority
WO
WIPO (PCT)
Prior art keywords
model
state
workflow
dynamic sequence
knowledge model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2018/000108
Other languages
German (de)
English (en)
Inventor
Patrick PHILIPP
Yvonne Fischer
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Original Assignee
Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV filed Critical Fraunhofer Gesellschaft zur Foerderung der Angewandten Forschung eV
Publication of WO2018171940A1 publication Critical patent/WO2018171940A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/20—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms

Definitions

  • Embodiments of the present invention relate to methods and apparatus for determining a state of a workflow.
  • Workflow modeling can be used to help people get on with their work - providing the opportunity to increase efficiency, avoid mistakes, and increase satisfaction.
  • a workflow eg. As a medical intervention, recorded multi-sensorially.
  • the actual progress and anomalies can be recognized and converted into an interactive assistance for the acting person. This is necessary to be able to implement a needs-based assistance.
  • an assistant who can offer a suitably adequate assistance function depending on the current context (actual progress).
  • the condition for this is that a workflow model is available in a suitable way (machine-readable).
  • the development of such models is often a lengthy process. This is illustrated herein by way of example from the field of medicine. For example, if a model is developed only by technical experts, a lack of medical domain knowledge can lead to model errors and low acceptance by medical domain experts. A tutelage through defined workflow models, which are not sufficiently comprehensible for the physician, can be another hurdle.
  • Embodiments according to the invention provide a device with a sensor device, which is designed to detect a work area in which a workflow is executable, and to provide a sensor signal.
  • the apparatus further comprises processing means adapted to obtain a graphical knowledge model of the workflow, for example a deterministic model of the workflow, and to assign a dynamic sequence model of the workflow, for example a probabilistic model of the workflow, based on the graphical knowledge model create.
  • the processing device is designed to determine a state of a dynamic sequence model of the workflow based on the sensor signal.
  • the apparatus has an interface for providing a status signal, wherein the apparatus is configured to provide the status signal based on the determined state of the dynamic sequence model.
  • the present embodiment is based on the idea that a graphical knowledge model may be more accessible to a domain professional, such as a medical professional, than a dynamic sequence model, but dynamic sequence models may provide reliable depiction and tracking of a workflow. Since dynamic sequence models are mostly based on complex mathematical structures, these are difficult to access for non-specialist personnel.
  • graphic knowledge models for example, can be displayed in a manner similar to a conventional flow diagram.
  • the flowchart may contain, for example, textual information and therefore non-specialist persons, ie persons without knowledge of probabilistic workflow modeling, be very accessible.
  • the dynamic sequence model in terms of determining the state of a workflow, has the advantage of allowing interpretation of sensor information regarding the likelihood of even unexpected events, thereby allowing accurate determination of the state.
  • the observation of an unlikely object can be ignored by means of the dynamic sequence model, since the detection may be based on a faulty or noisy sensor signal.
  • a more reliable assignment of states of a planned workflow to an actual workflow can be assigned.
  • the device is designed to adapt the dynamic sequence model based on the sensor signal.
  • the device described can adapt flexibly to a field of application by adapting the dynamic sequence model to the conditions of use.
  • the device provides a self-learning mechanism to adapt to changing environmental conditions.
  • the device is configured to provide an instruction to an assistant configured to interfere with the workflow, in the current state or in a subsequent state of the dynamic sequence model, based on the state signal.
  • the described device can advantageously provide information that the assistant device can use to influence the workflow in a desired direction.
  • the assisting device may include a gripping arm that can assist a worker in lifting heavy objects or mounting workpieces.
  • the display of general information or possible options for action for example on a screen or using head-worn spectacles in the sense of a virtual or augmented reality, represents a further possibility to favorably influence the workflow.
  • the dynamic sequence model is based on a dynamic Bayesian network.
  • Bayesian networks are particularly advantageous for sequence modeling since they are suitable for incorporating prior knowledge of courses of action or work processes in state decisions and for determining the dynamic or temporal behavior of sensor signals and the sequence of features derived therefrom. to model. Furthermore, by means of Bayesian networks, events which have been detected in the sensor signal can be classified and evaluated.
  • the graphical knowledge model is based on an activity diagram and is designed to map the workflow.
  • An activity diagram may be advantageous to allow a simple modeling of a workflow.
  • activity diagrams are easy to understand, especially for people who can not readily understand a modeling in the form of the dynamic sequence model.
  • the activity diagram is based on the Unified Modeling Language (UML).
  • UML Unified Modeling Language
  • the use of UML allows well-structured and normalized creation of activity diagrams. As a result, a high degree of order can be achieved, which in turn can lead to an easier understanding of the graphical model of knowledge.
  • the processing device is designed to map output pins and objects of activity nodes of the graphic knowledge model to nodes of the dynamic sequence model.
  • the described mapping rule for mapping the starting points of the graphic knowledge model to nodes of the dynamic sequence model, can be advantageously used to efficiently and simply obtain a dynamic sequence model based on parts, for example the output pins, of the graphical knowledge model.
  • the processing device is designed to associate activity nodes of the graphical knowledge model with states of the workflow.
  • the processing device is designed to associate nodes of the dynamic sequence model with states of the dynamic sequence model.
  • the exemplary embodiment described can advantageously map elements of the graphic knowledge model onto the dynamic sequence model by means of a simple mapping rule. Thus, with little effort, for example, automated, portions of the graphical knowledge model can be mapped to the dynamic sequence model.
  • the processing device is designed to include annotations associated with the activity node or annotations and edges of the graphical knowledge model associated with branch nodes of the graphical knowledge model to associate nodes of the dynamic sequence model.
  • the device can simply translate or transmit a graphical knowledge model into a dynamic sequence model.
  • the processing means is adapted, for creating the dynamic sequence model, to link an activity during the workflow with an object of the workflow in the graphical knowledge model. Furthermore, the processing device is designed based on the link to obtain an object flow of the graphic knowledge model and to map the object flow onto a directional path of the dynamic sequence model, wherein an object flow can also be referred to as a data flow.
  • the directional path is arranged between a root node and a corresponding child node. The described mapping can be advantageously used to associate observations of the sensor signal of objects corresponding states of the dynamic sequence model.
  • the state is a first state and wherein the device is configured to detect an event in the work area based on the sensor signal. Furthermore, the device is designed to determine a second state of the dynamic sequence model based on the first state and based on a probability of occurrence of the event in the first or the second state.
  • the disclosed embodiment may advantageously use occurrence probabilities of events, such as tool observing, to utilize an assessment of the probability of the observation associated with a new condition, such as the second condition.
  • the device formed considers an observation to be probable for a state, then based on the dynamic sequence model, the device may enable the decision to be made in a more probable state. For example, a state transition from the first state to the second state may be initiated due to observation of a tool that is more likely to occur in the second state than in the first state.
  • the device is designed to obtain or derive the occurrence probability from the graphic knowledge model.
  • the described embodiment can advantageously occurrence probabilities of serving To take on staff that is not skilled in dealing with dynamic sequence models by enabling input via the graphical knowledge model.
  • a person who is not experienced in dealing with dynamic sequence models still very easy to parameterize the device.
  • expert knowledge of persons who are very familiar with the workflow, but who are not familiar with dynamic sequence models can thus be transferred into the dynamic sequence model.
  • the processing device is designed to determine the state of the dynamic sequence model using transition probabilities based on nodes of the dynamic sequence model. For example, if a node of the dynamic sequence model corresponds to a state of the workflow, the device may determine the likelihood of a state transition using the dynamic sequence model and based on the knowledge that the dynamic sequence model is in the state Change state, for example, in a state of a temporally subsequent step is assigned.
  • transition probabilities By using transition probabilities, a precise determination of a transition from one state to another state can be made possible.
  • the processing device is designed to adapt the occurrence probabilities and / or the transition probabilities with the aid of recorded sensor data.
  • the described embodiment may advantageously use the recorded sensor data describing the work area to adjust the probability values that parameterize the dynamic sequence model. As a result, any missing or incorrect initial parameterization can be compensated.
  • an occurrence probability of an event is correlated with a stay or change of a state of the dynamic sequence model.
  • a transition probability describes a likelihood of going from a first state to a second state, and the processing device is configured to estimate the current state during the operation based on evaluations of the sensor signal.
  • initial initialization may include providing occurrence probabilities of events, or transition probabilities of states, as known to a medical professional during an operation, for example.
  • the processing means is adapted to determine the state based on a time average of the sensor signals based on a plurality of time-spaced detections of the work area. Using temporal averaging, noise or spurious components of the sensor signal, ie the detection, can be suppressed. Thus, improved sensor signals may be provided to the processing device to more reliably determine the state of the dynamic sequence model.
  • the sensor device is configured to provide the sensor signal based on a visual detection of the work area.
  • a visual capture can be easily performed with a camera, which can easily capture a variety of information from the workspace.
  • the sensor device is designed to provide the sensor signal based on an acoustic detection of the work area.
  • An acoustic detection may be advantageous, for example, to detect the tools used in a work step based on noise, or to allow a classification of the current work step based on dialogues of personnel.
  • the sensor device is designed to provide the sensor signal based on a high-frequency (radio frequency) detection of the work area.
  • a high-frequency detection of the work area For example, allow the detection of tools used.
  • the device comprises a user interface adapted to receive input from a user that at least partially describes the workflow.
  • the apparatus is configured to provide the graphical knowledge model based on the input of the user.
  • the exemplary embodiment described can enable a simple parameterization of the graphic knowledge model for a user, for example via a computer by means of keyboard and mouse.
  • Embodiments according to the invention describe a system for workflow support, the system comprising a device as described herein, as well as a workspace and an assistant facility. The described system can advantageously assist a worker in a difficult or complicated job.
  • Embodiments according to the invention provide a method.
  • the method includes detecting a workspace in which a workflow is executable. Furthermore, the method includes providing a sensor signal based on the detected work area. Further, the method includes obtaining a graphical knowledge model of the workflow and creating a dynamic sequence model of the workflow based on the graphical knowledge model of the workflow. Furthermore, the method comprises determining a state of the dynamic sequence model based on the sensor signal. Incidentally, the method includes providing a state signal based on the determined state of the dynamic sequence model. The method may be extended to include all features and functionalities mentioned in connection with the device described herein. Fiqurenkurzbeschreibunq
  • FIG. 1 is a schematic block diagram of a device according to embodiments of the invention.
  • FIG. 2 is a flow chart of a graphical knowledge model according to embodiments of the invention.
  • Fig. 4 shows a creation of a workflow, by means of a graphical knowledge model and a dynamic sequence model, according to embodiments of the invention
  • Fig. 5 shows a dynamic portion of a dynamic Bayesian network for two
  • Time periods according to embodiments of the invention shows a dynamic Bayesian network according to embodiments of the invention, which is rolled out over three time steps;
  • FIG. 1 shows a schematic block diagram of a device 100 according to exemplary embodiments of the invention.
  • the device 100 has a sensor device 110, a processing device 120 and an interface 130.
  • the device 100 may include an optional user interface 140 and / or an optional assistant device 150.
  • the sensor device 110 is designed to detect a work area 160 in which a work sequence can be executed and to provide a sensor signal 12 based on the detection.
  • the processing device 120 is designed to obtain a graphical knowledge model 122 of the workflow and to create a dynamic sequence model 124 based on the graphical knowledge model 122. Furthermore, the processing device 120 is designed to determine a state of the dynamic sequence model 124 of the workflow based on the sensor signal 1 12. The processing device 120 can access both the graphical knowledge model 122 and the dynamic sequence model 124.
  • the interface 130 is configured to provide a status signal, wherein the device 100 is formed, to obtain the state signal based on the determined state of the dynamic sequence model 124.
  • the sensor device 110 can be set up in such a way that it detects the work area 160 in such a way that tools occurring or appearing in it can be detected.
  • the sensor signal 112 of the processing device 120 can provide information about occurring objects.
  • the processing device 120 may provide an estimate or statement about the current state or a subsequent state of the workflow based on the sensor signal and a current state of the dynamic sequence model 124 associated with a state of the workflow. For example, when the sensor signal detects a particular item, the apparatus 100 may rate the likelihood of the item occurring in the current condition.
  • the evaluation may include, for example, that the processing means evaluates how likely the occurrence of the item is in the current state.
  • the processing device 120 may change to a state in which the mentioned object is more likely to occur. Based on the determined state of the dynamic sequence model 124, the processing device 120 may estimate a state of the workflow and provide that estimate or state via the interface 130 in the form of the state signal. For example, interventions for the workflow may be scheduled based on the status signal. For example, a worker performing the operation may be warned if, for example, B. a step was skipped or a wrong tool is used. Also, based on the estimated state, for example, an assisting device, such as the assisting device 150, may interact with the workflow, in other words, act, intervene or participate in the course of action.
  • an assisting device such as the assisting device 150
  • the assistant device 150 could be a gripper arm assisting a worker.
  • the processing device 120 could detect that an object is too heavy to be lifted by a human, and, for example, instruct a gripper arm to lift the object.
  • the device 100 offers the advantage that the derivative of the dynamic sequence model 124 is based on a graphical knowledge model 122. This makes it easy to parameterize the workflow, since a graphical knowledge model, in general, is more accessible or easier to understand for people who are not experts in the field of probability calculus.
  • graphic knowledge modeling allows a simple graphical modeling, for example via a workflow diagram. Graphical modeling can be easier to understand for staff who are not otherwise involved in probabilistic workflow planning.
  • FIG. 2 shows a flow chart 200 of a graphical knowledge model according to embodiments of the invention.
  • flowchart 200 may describe a cholecystectomy, that is, a medical workflow to remove the gallbladder.
  • States 210, 215, 220, 225, 230, 235, 240, 245, and 250 represent states of the workflow, and the states associated with a workflow plan may also be referred to as operations.
  • a pneumoperitoneum can be generated, i. Gas is pumped into the abdomen of the patient to inflate the abdomen of the patient. For example, this process may take 186 seconds, as indicated by annotation 210a.
  • trocars may be placed, with trocars being tubes with sharpened endings.
  • the state 215 is associated with a probability of occurrence of 0.95, that trocars 200a are used or detected in the work area, that is, in the state of interest. H. with 95 percent probability, the sensor device z. B. the sensor device 1 10 detect trocars in the detection of the work area. State 215, for example, lasts for 195 seconds, which is indicated by annotation 215a.
  • the free preparation of the bile duct can be performed. For example, this state may take 558 seconds, which is indicated by annotation 220a.
  • the probability of occurrence of trocars 200a remains 95%. Coming in addition, but also other tools can be detected, such.
  • a fourth state 225 for example, stapling and cutting (stapling and cutting) of the bile duct can be performed.
  • the described fourth state 225 may last for example 14 seconds, characterized by Annotation 225a.
  • the known devices or tools 200a and 200b can occur with the known occurrence probabilities.
  • the tools 200e, z As a laparoscopic scissors (lap scissors) with a probability of 90% and 200f, z.
  • a stapling device (Abemmemm réelle, Tacker), occur with a probability of occurrence of 91%.
  • the gallbladder artery can be dissected free, which state can last, for example, 108 seconds, characterized by annotation 230a.
  • the known devices 200a, 200b, and 200c may occur with the known probabilities.
  • a sixth state 235 for example, the gallbladder artery can be clamped and cut.
  • this sixth state 235 may last for 1 to 15 seconds, indicated by annotation 235a.
  • the already known tools 200a, 200b, 200e and 200f with the known occurrence probabilities can be observed.
  • an optional state 240 can be reached in the workflow, this is indicated by the branch 236.
  • branch 236 a case distinction is made that is conditioned, for example, by the worker's knowledge.
  • a physician may recognize before or during the operation that the optional step 240 may be necessary.
  • the physician may indicate this to the device 100 via the user interface 140.
  • 600 seconds may be paused, as annotated 240.
  • optional state 240 may be associated with a task that requires bile duct imaging.
  • the sensor device should only detect the tools 200a, such as the trocars mentioned.
  • the dynamic system can also detect without input that step 240 is present, since it is highly probable that only trocars are used, whereas in step 245 more tools are used. Furthermore, it is known via the annotation 234 at junction 236 (expert knowledge) that intraoperative imaging (240) is necessary in 45% of the OPs, ie a transition from 235 to 240 takes place. After carrying out the optional work step 240, it is possible to return to the standard work path via the node 241. In another If, for example, 245, which according to annotation 245a, can last 534 seconds, the gallbladder can be removed (removed). During this state, the known tools 200a, 200b, 200c, 200d, 200e and 200f with the known occurrence probabilities can be observed.
  • coagulation of the liver bed can be performed.
  • the known tools 200a and 200d may occur with the known probabilities.
  • a workflow plan as per workflow 200, may also describe, for example, a workflow for assembling automobiles or other complex activities.
  • the workflow plan can have any number of states, which can be assigned to objects 200a-f in the form of output pins 200a'-f, or their occurrence probabilities for a particular state, such as by means of a user input.
  • the probability of occurrence of an object or event can vary in different workflow states, i.
  • Object 200a could be 95% expected in a first step and 60% expected in a second step. These probabilities may indicate how frequent or likely an event is during the state, such as a specific object being used or present in the workspace. Alternatively, the associated probability may also relate to another event, such as the change of an object with respect to location or state. This may include a partial or complete consumption of the object.
  • Paths between the states may describe a change from one state to another state, such as when the activities are performed in a state, an associated time period 210a, 215a, 220a, 225a, 230a, 235a, 240a, 245a or 250a has expired and / or another criterion is met.
  • the states 210, 215, 220, 225, 230, 235, 240, 245, and 250 may be understood as activity nodes that are expected to have an associated activity in the workflow.
  • an object flow connects an output pin 200a'-f to an object 200a-f.
  • the graphical knowledge model 200 is poorly suited for direct sensor signal processing, but allows comfortable and easy assignment of activities to be performed in the workflow to the states / activity nodes 210, 215, 220, 225, 230, 235, 240, 245, and 250 as well as an association of occurrence probabilities of certain events in the states, which is represented by the output pins 200a'-f. Relationships and transitions between states may be represented by paths 21 1, 216, 221, 226, 231 or 246.
  • the graphical knowledge model can represent an expected flow of the working diagram. However, deviations from the illustrated workflow are not taken into account or only inadequately considered, but this is made possible by dynamic sequence models, which also include dynamic Bayesian networks, which are explained in conjunction with FIGS. 5 and 6.
  • Devices in accordance with the present invention are configured to obtain a graphical knowledge model, such as the graphical knowledge model 200, and translate it into a dynamic sequence model and to sensibly capture the execution of the workflow and determine states in the dynamic sequence model using or evaluating the sensor data.
  • a graphical knowledge model such as the graphical knowledge model 200
  • the model 200 may be supplied to the device 100 via the user interface 140 and used there to coordinate or assist the described workflow.
  • Figure 2 may show the UML activity of a cholecystectomy. At the beginning is a sequential order of actions. The action "intraoperative cholangiography" is optionally performed, the activity being derived from a description in [14], [15] and [16]
  • Figures 3a-c show elements of flowcharts and / or complete flowcharts according to embodiments of the invention 3a and 3b show flowcharts with a start node 310 and an end point 390 and therefore represent complete flow charts. On the other hand, the operations arranged between the start node 310 and end node 390 may also serve as elements of other flow charts.
  • FIG. 3a shows an activity 1 which has a start node 310 and changes into the state A starting from this start node.
  • a transition from state A to state B is indicated. This sequential order can be mapped by processor 120 to a dynamic sequence model and then used for state determination.
  • the workflow flows to an endpoint 390 in Figure 3a.
  • Figure 3b shows an activity 2 describing a decision possibility or branching capability within a workflow. Starting from a starting point 310, the workflow runs into a decision node 320.
  • a decision criterion here, for example, a threshold decision, it can be decided here whether the workflow is performed in state A, ie the activities associated with state A.
  • the thresholding decision for selecting the branching path may, for example, be performed by the processing device 120 and the resulting state of the workflow therefrom determined by the processing device 120.
  • the decision node 320 is accompanied by an annotation which describes information about the probability of the change into the respective branches. Say: how often is a given branch taken? Thereafter, the flowchart merges back into node 330 to end at end node 390.
  • Figure 3c shows excerpts of activities, activity 3 and activity 4 that may be part of workflows.
  • Activity 3 shows the workflow state A, which is coupled to an object y, via an object flow, whereby the object y can occur in state A with a probability of 95%.
  • an annotation is attached, which can provide information about the state A.
  • the linking of the workflow state A with the object y can be used by the processing device 120 to conclude the state A via the dynamic sequence model if the sensor signal detects the object y.
  • Activity 4 can be interpreted analogously to activity 3, where state A is here associated with an object y in another representation with a probability of occurrence of 95% via an output pin of the activity node A and an object flow.
  • state A in activity 4 has an annotation that can provide information about state A.
  • the connection between state A and object y in activity 3 can also be characterized as flow or stream.
  • Figures 3a and b show two typical flowcharts of workflow models in the context of surgical procedures.
  • the actions A and B are carried out successively (consecutively).
  • a decision must be made to perform either A or B.
  • the decision is represented by a guardian (expression). If it evaluates to "true”, the corresponding action is taken
  • Fig. 3c) shows two equivalent representations of a streamed object, that is, an object flow may occur while A is still executing (not after execution) the activities text annotations that can be used to store additional information.
  • FIG. 4 shows an illustration of a workflow in a graphic knowledge model, here a UML activity diagram, and its conversion into a dynamic sequence model, here a dynamic Bayesian network.
  • the example shown in FIG. 4 is based on a medical example. For this purpose, knowledge from medical textbooks and guidelines is used either in combination with a medical expert or in a dialogue between domain experts to define a workflow. Domain experts can be, for example, engineers or physicians.
  • a UML activity diagram can be generated, which is then automatically translated into a dynamic Bayesian network.
  • the device 100 may receive the defined workflow, here in the form of the UML activity diagram, via the user interface 140 to define the graphical knowledge model 122.
  • the device can now determine the dynamic sequence model 124, here the dynamic Bayesian network, and use it to determine the state.
  • Figure 4 shows that medical knowledge can be converted to UML activity of the surgical procedure.
  • the formalization process may be performed by an expert dialogue or by the medical expert himself.
  • the resulting activity serves as an interface to more complex models used for actual detection of a surgical procedure.
  • Embodiments of the invention describe translation rules to translate a given UML activity into a Bayesian Dynamic Network (DBN)
  • Bayesian networks have been described as an example of a dynamic sequence model.
  • Other examples of bases of dynamic sequence models are neural networks, hidden Markov models, Markov networks or partially observable Markov decision processes.
  • states for example, individual phases of operation, are hidden from the observer (technical system).
  • Markov nets (Markov Random Fields) are similar to Bayesian nets (BN), but the dependencies are nondirective, i. Unlike BNs, cyclic dependencies can be modeled directly.
  • Partially observable Markov decision processes are similar to the Hidden Markov model, but it is not a purely observational model, but actions are modeled. A sequence of the partially observable Markov decision-making processes, however, is hidden from the actual observer (technical system), as in the Hidden Markov models.
  • FIG. 5 shows a dynamic part of a dynamic Bayesian network 500.
  • FIG. 5 shows a simplified graph of a two-part temporal Bayesian network (2TBN).
  • a 2TBN (or: ⁇ _) is used as a template for consecutive time steps t.
  • a DBN consists of two networks.
  • B 0 represents the starting distribution and ß_ describes how a network evolves over time.
  • a Bayesian network (B 0 ) serves as an initial starting point for consecutive time steps. For simplicity, was omitted, since in this
  • a transition to a second state may occur take place in a second time period t.
  • the condition observations can be associated with the state X °.
  • Observations can be, for example, which, for example, occurrence probability of 4 plant
  • the Bayesian network can be applied to the nodes by mapping phases of a workflow, for example, phases 210, 215, 220, 225, 230, 235, 240, 245, or 250 of the graphical knowledge model 200 of a Bayesian network in the processing device 120.
  • the Bayesian network can be generated by mapping objects of a workflow and their probabilities, for example, object probabilities of the objects 200a-f, onto nodes in the processing device 120.
  • the device 100 is designed, for example, to create the dynamic sequence model using the graphical knowledge model.
  • the processing device 120 can map at least one, several or all of the activity nodes 210, 215, 220, 225, 230, 235, 240, 245, 250 into a state or node of the dynamic sequence model.
  • the processing device can be designed to apply a mapping rule according to where V B is a set of nodes of the dynamic sequence model, v e is a node of the dynamic sequence model, (e i ( ey) is an edge e or path between an object e y and an action e , - of the graphical knowledge model , DF is an amount of object flows of the graphical knowledge model, a is the amount of actions or conditions of the graphical knowledge model, O is the amount of object nodes of the graphi ⁇ rule knowledge model, v r is a root node of the dynamic sequence model and wherein an object flow connects an action of the set A with an object of the set 0 in the graphical knowledge model.
  • V B is a set of nodes of the dynamic sequence model
  • v e is a node of the dynamic sequence model
  • (e i ( ey) is an edge e or path between an object e y and an action e , - of the graphical knowledge model
  • DF
  • the processing device 120 is configured to generate a node v e of the dynamic sequence model for an object e ; - to generate, which is associated with a state or an action e t and whose such link is included in the graphic knowledge model.
  • the quantity is calculated by combining the nodes v e with the
  • the apparatus 100 is, for example, designed to create the dynamic sequence model using the graphical knowledge model.
  • the processing device can for this purpose nodes of the dynamic sequence model based on activity nodes of the graphical knowledge model according to
  • v * characterizes the root node of the dynamic sequence model, for example, at an advanced time t, which differs from a previous time t - 1, describes child nodes, the be added to characterize the objects or their probability of occurrence.
  • the processing device 120 is configured to be nodes an original or initial node set of the dynamic sequence model that corresponds to a state (root node or an object (child node whose link is included in the graphic knowledge model to obtain the set.
  • the device 100 is, for example, designed to create the dynamic sequence model using the graphical knowledge model.
  • the processing device 120 may be directed thereto paths between nodes of the dynamic sequence model based on object flows of the graphical knowledge model according to where E Bo is the set of directed paths of the dynamic sequence model, v r is a root node of the dynamic sequence model that characterizes a state where DF is the set of object flows of the graphical knowledge model, A is the set of actions or states of the graphical knowledge model and 0 is the set of object nodes of the graphical knowledge model.
  • the processing device 120 is configured to generate a path ( r , e) of the dynamic sequence model to an object e- associated with a state or action e £ , and whose such link is included in the graphical knowledge model to link using the path.
  • the crowd gets through
  • the device 100 is, for example, designed to create the dynamic sequence model using the graphical knowledge model.
  • the processing device 120 may determine paths of the dynamic sequence model based on structures of the graphic knowledge model according to FIG.
  • An object describes a path between a current action and the one
  • root node is an object and a path between
  • a current root node For example, at time t, and a predetermined root node v r , for example, at time t - 1 describes.
  • the set of paths is not added because the child nodes of v r in the 2TBN at the time
  • point t-1 have no direct influence on the nodes at time t.
  • the processing device 120 is configured to connect temporally preceding root nodes to immediately succeeding root nodes, the root nodes describing states of the workflow and, in addition, creating paths between root nodes and child nodes, corresponding to links of actions or states to objects in the graphical knowledge model.
  • the processing device 120 is designed to create the 2TBN for the creation of the directed paths in such a way that in the time step t-1 only those random variables are contained which actually have an influence on the access have variable variables in time step t (so-called interface variables). According to embodiments, this is just the variable and thus node v r . This can also be understood that way
  • the root node is linked by the processing device 120.
  • the nodes can describe the same state of the workflow, or
  • the processing device is designed to associate annotations associated with the activity node or annotations and edges of the graphical knowledge model assigned to branch nodes of the graphic knowledge model with nodes of the dynamic sequence model.
  • Equation (1) As described in equations (1) and (3).
  • the matrix A according to equation (13) and ⁇ can be calculated or generated according to equation (12) and be assigned via equation (2).
  • FIG. 6 shows a dynamic Bayesian network 600 consisting of the pair which is rolled out over three time steps. For simplicity, some nodes have been omitted, which is graphically represented by "".
  • FIG. 7 shows a method 700 according to exemplary embodiments of the invention, comprising the following steps: Detecting 710 a work area in which a workflow is executable. Providing 720 a sensor signal based on the detected work area. Obtain 730 of a graphical knowledge model of the workflow. Create 740 a workflow dynamic sequence model based on the graphical knowledge model of the workflow. Determining 750 a state of the dynamic sequence model based on the sensor signal. Providing 760 a state signal based on the determined state of the dynamic sequence model.
  • graphic knowledge models can be generated offline, ie before execution of a workflow, and for parameterization of the dynamic sequence model.
  • dynamic sequence models can be characterized as real-time-capable sequence models or online-capable classifiers, since they are used to detect successive steps or states of a currently executed workflow, based on evaluation of the sensor signal.
  • the later target model (which can serve to evaluate the actual sensor values ("online", so while the sensor values enter the system is successively operated on a classification, that is, it tries to detect the current phase)
  • graphic knowledge models can also be referred to as parameterization models, since they are designed to allow a parameterization of a workflow.
  • dynamic sequence models can also be referred to as processing models, since they are designed to directly process data based on sensor signals in order to detect the data State of the workflow.
  • DBN Dynamic Sequence Model
  • An illustrated UML activity can be converted into a dynamic sequence model by means of the executed conversion or transfer of the parameters.
  • This mapping can be ambiguous, because a DBN can be executed differently (network structures etc.).
  • a concrete DBN is generated from the knowledge represented in the graphic knowledge model, which can actually be used for classification.
  • the transfer can thus, for example, smoothing properties have.
  • the output of the model is smoothed (ie the classification result). It is smoothed by the model storing other classification results during the current execution, thus relating the current classification to the previous classifications. So a sequence of sensor data is considered, not just single feature values at a time. This allows the model to pay less attention to "false” or "noisy” data, and so the output is smoothed out.
  • Phase_1 of a workflow prevails due to the incoming sensor data.
  • feature values are typical in the system which are typical for Phase_10 (e.g., due to sensor noise).
  • a system that does not consider the sequence would immediately classify Phase_10.
  • a system which incorporates the sequence may smooth out here by e.g. the previous hypothesis (Phase_1) is initially maintained.
  • the DBN contains additional knowledge than the UML model, which is introduced during the transfer into the DBN, for example in the form that e.g. a specific destination network structure has been selected. This knowledge is thus added by the described transmission, for example according to equations (6) to (11).
  • not all degrees of freedom of the target model are covered by the initial model. By defining a concrete transfer, these degrees of freedom are first set in the target model. For example, a degree of freedom may be the number of layers using a neural network, which may be determined depending on transmission rules.
  • a DBN is represented as a graphical knowledge model
  • this model can not be used to classify the state of the workflow, ie, no sensor data can be evaluated online and features can be classified.
  • the DBN also makes further assumptions which are necessary for a specific classification (eg concrete network structure of the DBN, or eg structure of the layers in a neural network).
  • the parameters can be present in the DBN in such a way that they can advantageously be used for a classification (eg by smoothing, suppression of noise).
  • the following describes backgrounds and examples of graphical knowledge modeling. In general, these are models that focus on the collection of expert knowledge (not the classification of observation sequences).
  • Such a model can be presented to the operator in exemplary embodiments according to the invention, and by using machine-supported dialogues, it is possible to determine the individual elements of the graphic knowledge model.
  • the knowledge can be represented in the form of a graph for easy comprehension.
  • UML activities for example, this concerns the definition of possible sequences of actions. This can be expressed there by directed edges (arrows) (A-> B).
  • a transition in such a model can be deterministic. Alternatively, it may be determined, for example, using a branch, that in 50% of the cases a first path is followed up and in 50% of the cases a second path is followed, whereby both the number of paths and the probabilities are arbitrarily variable.
  • sequence modeling In a classification task, the prediction of the current class can be dependent on other classification results, ie feature vectors that flow into the system must be in Example Speech Recognition: If at a time 1 the word "good” was predicted, at time 2, the prediction “goulash” (though most likely considered isolated) is highly unlikely in the context of the sequence, and eg the word "day” more likely.
  • Classic probabilistic methods are HMMs, DBNs. Neural networks are becoming increasingly important. Even if neural networks are not generally referred to as probabilistic models, they will be referred to and understood as such in the context of the probabilistic analysis performed. Properties of these models can be represented as graphs.
  • DBNs for example, the independence of random variables, in neural networks, for example, the interconnection of individual layers.
  • These graphs are not (or only with difficulty) suitable for an expert dialogue in the sense of formalizing the expert knowledge.
  • the graphic knowledge model is understood to mean those models that are designed for the graphic preparation of a workflow, such as UML activities, and therefore go beyond the mere possibility of graphical representability.
  • Devices and methods according to the invention provide for transferring the graphic knowledge model into the dynamic sequence model and for monitoring the Work area by sensory detection. The sensor data is evaluated to determine the state of the dynamic sequence model of the workflow. This means that the dependency of the graphic knowledge model on the procedure described therein is overcome by the devices, methods and computer programs described herein according to the invention.
  • Embodiments according to the invention describe a framework for expert-based workflow modeling (workplan modeling) for interactive work process support.
  • the framework can involve all involved experts in a dialogue for knowledge formalization.
  • an activity of the Unified Modeling Language can be selected as the formalization basis. This is so easy to understand that a formalization can be worked out together in the expert dialogue. This means (using the example of medicine) that e.g.
  • a further advantage is a transfer / processing of work process relationships into process assistant control processes that are adjusted based on observations of the sensor signal. This is done by using the dynamic sequence model obtained by translating the graphical knowledge model.
  • an automatic translation of the UML activity into a more complex model a dynamic Bayesian network (DBN) takes place.
  • DBN dynamic Bayesian network
  • This is used for the actual recognition eg of the actual progress of the intervention.
  • This model itself does not provide any suitable interfaces for expert knowledge To bring in dialogue (see above) directly. It is not understandable for all experts involved. For this reason, a transfer of the models is necessary.
  • the framework can be used to design a total of assistance systems that are accepted by the assistants, for example because of their simplicity, and that can actually be integrated into the working days as a relief tool.
  • Embodiments according to the invention can be used in all industries in which work processes are interactively supported. These can be: assembly processes in industry, surgical interventions in medicine, emergency processes in civil protection, etc.
  • aspects of the invention relate to surgical procedures, Bayesian Dynamic Networks, Unified Modeling Language, Assistance and Planning.
  • Embodiments of the invention relate to expert-based sequence modeling of workflows in the context of surgical procedures.
  • computer-based support both software and hardware
  • computer-based support can be implemented in a number of ways - therefore the next subsections outline three typical types of support and their relationship to situational awareness.
  • the following discusses a modeling approach and corresponding requirements for tailored support, taking into account the specific conditions of a current operation: firstly, a suitable representation of the underlying work process; second, reliable capture of current progress by synchronizing this workflow model with sensor data.
  • UML Unified Modeling Language
  • the Unified Modeling Language (UML) can be used to provide a graphical representation of the workflow through UML activities, facilitating the dialogue between technical and medical experts.
  • UML Unified Modeling Language
  • DBN dynamic Bayesian networks
  • UML activities To model workflows through UML activity, medical knowledge must be focused and formalized. This formalization process can be realized by means of a dialogue between medical and technical experts, see Figure 4. Due to its ease of understanding, UML activities also provide the medical expert with the ability to modify an already formalized workflow itself. Barriers can be broken as the representation of activity serves as a by-pass. The fact that physicians can understand and modify the formalized representation of the workflow reduces their fear of paternalism and allows them, for example, to easily run a workflow based on specific constraints adapt. For the actual implementation of a support function that supports the physician during an operation, UML activities are translated into more complex models. The structure and parameters of these models are generated automatically.
  • tet a cholecystectomy (the surgical removal of the gallbladder).
  • This approach can be easily adapted to other applications, eg. B. a hip joint replacement [1 1]. The following is a closer look at UML activities.
  • UML is used and accepted in various disciplines worldwide [12]. UML activities have been selected as an interface for medical and technical experts due to their ease of understanding [13]. Understanding the workflow representation is a necessary precondition for dialogue between experts.
  • FIG. 3 shows typical flowcharts that may occur in operations such as cholecystectomy.
  • the beginning of an activity is represented by a black dot (beginning node), the end of an activity is symbolized by a double circle (end of activity).
  • Individual actions of the workflow are symbolized by rounded squares. Arrows represent the flow and thereby the order in which the actions can be performed.
  • sub-figures a) and b) show two different types of flowcharts in FIG. 3
  • sub-figure c) represents two equivalent parts of an activity.
  • the fact that an object flow can occur while action A is being performed (and not only after A is completed) is specified by the keyword ⁇ stream ⁇ .
  • An abbreviated form is a black pin, which is shown in activity 4.
  • the activities in subfigures b) and c) include text annotations associated with action A and / or a decision node to store additional information.
  • FIG. 2 Modeling a surgical workflow Figure 2 can be interpreted as a formalized model of cholecystectomy (surgical removal of the gallbladder).
  • the top part of this UML activity shows a sequence of actions.
  • the first action 210 is the injection of carbon dioxide to inflate the abdomen (stomach).
  • the average time of this action is represented by the corresponding note.
  • the trocars are used 215 to break through the abdominal wall.
  • trocars are used to facilitate the placement of additional medical instruments during surgery.
  • Black pins output pins represent the observation probability (values in parentheses next to tools 200a-f) of trocars in this act ( Figure 2).
  • the gallbladder duct and artery are exposed 220 and 2230, as well as clamped and severed 225 and 235.
  • An intraoperative cholangiography 240 (radiographic imaging of the bile ducts with contrast agent) is optional, so a decision must be made.
  • the gallbladder is separated from the liver bed 245 and the liver bed is coagulated for hemostasis, i. Tissue is destroyed or burned by electricity.
  • DBN dynamic Bayesian networks
  • sequence models that have proven to be successful in practice (eg hidden Markov models) [10].
  • DBN models allow for improved modularity and interpretability. Unlike a hidden Markov model, its state space can be expressed in a factorized manner rather than just a single discrete random variable.
  • DBN allow arbitrary probability distributions (not just unimodal linear Gaussian distributions) with respect to Kalman filter models [16].
  • Dialogue is less suitable for medical and technical experts.
  • modification of the model by the medical expert himself does not seem feasible, for example because of the complexity.
  • translation rules are introduced to translate understandable UML activity into a DBN.
  • a Bayesian network is a probabilistic graphical model (PGM) that combines theoretical approaches with probability theory approaches [20].
  • a dynamic Bayesian network is an extension of a BN, which also takes into account the temporal dependencies of variables [19].
  • a DBN is given by a pair: the Bayesian network used the a-priori probability distribution over random variables in a time step with index 0 to specify. Further, S_ specifies the conditional probability distribution over discrete time steps t by using:
  • In (1) is the set of parents of in the corresponding graph.
  • Fig. 5 is an exemplary graph of which is also known as a two-bay temporal Bayesian network (2TBN). Since the child nodes in 2TBN have no direct influence on the nodes at time t-1 at time t-1, they are not connected by edges to the root-to-root at time t-1. In this case, with respect to equation (1): Those nodes which have a direct influence on the nodes at time t are also referred to as interface. For simplicity, the time t-1 of a 2TBN can be represented or defined via the corresponding interface.
  • the composite distribution can be represented graphically by "rolling" the DBN, where B 0 is the initial distribution and be used as a template for each subsequent time slice.
  • B 0 is the initial distribution and be used as a template for each subsequent time slice.
  • FIG. 6 Similar to HM, parameters of such DBN with N children can be grouped as follows (see Fig. 6):
  • a (i, j) is an adjacency matrix that is extended by transition probabilities for nonzero entries.
  • the probability distribution is given for observations concerning a child. It should be noted that an HMM can be specified by a single matrix B (i, j), since the corresponding probability distribution can not be factored as in the case of a DBN - ie, graphically, the root node would have only one child, the whole Probability distribution contains.
  • the set N can be further subdivided into different sets of nodes:
  • A set of actions, for example 210, 215, 220, 225, 230, 235, 240, 245 and 250,
  • O set of object nodes, for example 200a-f.
  • the set of object nodes is given by the set of data pins.
  • a node that is part of one of the node sets E, B is called a control node. Further, the amount. given by the following:
  • output pins of UML actions are converted into vertices (vertices) of the graph of S 0 .
  • an auxiliary node v r is added to the mesh to represent the root of the resulting Bayesian structure.
  • the object flow connecting an action to an object is translated into a directed edge from the root node v r of the graph of B 0 to the corresponding child.
  • the vertex amount of the graph of is given by (8) and (9).
  • Fig. 5 is the amount which contains and the vertex set with the vertex set united.
  • An asterisk (star) is used to avoid ambiguity and to mark vertices that belong to the time slice t of the graph of belongs. See Fig. 5 for an example of a simplified 2TBN.
  • the simplification in Fig. 5 is that the child nodes of v r are not shown, since they have no edges in the 2TBN and thus can not serve as an interface to the next time slice.
  • the amount of edges (10) contains edges of the root node to the appropriate children.
  • the edges are derived from the object flow that connects an action to an object.
  • a unique natural number is assigned. Without limiting the generality, the numbers assigned by / correspond to a lexicographical order. You get: in which is used to denote the set of values that a linked random variable can take. For all other random variables, a Bernoulli distribution is used. That is, random variables those with child nodes of are linked, are binary, eg, where used. Furthermore, the
  • conditional state distribution and the CPD (conditional probability distribution) of observations and state transformations are specified.
  • initial state distribution ⁇ is given by:
  • the bijective function is that of every act A assigns a unique natural number as defined above. Since there is only one earmarked initial state, the probability of its presence in the initial distribution is 1; all other states are assigned 0 as the initial probability (12).
  • the probability distribution of state transitions is given by (13). Their representation is similar to an adjacency matrix with transition probabilities as edge weights (similar to HMM).
  • the matrix A is given by the following:
  • the number u.an.du indicates the average duration of the phase, that with action 6.an.prob (u, ⁇ ) indicates the probability of a transition to a phase ⁇ , provided phase u is exited.
  • Both u.an.du and ö.an.prob (u, v) are in note for the corresponding action or branch node b 6 B, as explained in the section on Dynamic Bayesian networks. It should be noted that a directed edge from a source node to a destination node ⁇ is given by the edge (u, v), whereas ⁇ u, v ⁇ denotes an amount of a source and a destination node.
  • pins denote the features observed during an operation. Some of the pins oe 0 are attached to different actions but represent the same feature, ie they have the same name. 0 name is the set of unique pin names that exist in the activity. A surjective function is defined. This function gives the name of a given
  • This function h assigns each item of 0 name lexiko
  • each matrix ⁇ is given by:
  • the conditional probabilities are of the condition of the goal
  • Each destination node is linked to a probability value o.prob,
  • a UML activity can be translated into a DBN, where vector ⁇ is the initial state distribution, state transition matrix A and Matrices B k are used for the probability distributions of observations.
  • B 2 is shown, which encodes the probabilities of grasped grasped use according to the surgical phase.
  • the transposition (T) is used to facilitate the representation of the matrix B 2 in landscape format.
  • the 9 columns of the illustrated matrix in (15) represent the different surgical phases.
  • the first row corresponds to the probability that the gripper will not be used in one phase.
  • the second line is the probability that the gripper will be used depending on the phase. That is, the columns add up to 1.
  • the probabilities contain two sources of expertise: First, the likelihood that a specific instrument will be used depending on the specific phase (medical knowledge). Secondly, the likelihood of detecting a specific instrument used in the specific phase (technical knowledge). For example, even if a medical expert estimates a likelihood of using an instrument in a specific phase as 1, the technical expert must consider the accuracy of the algorithms used, e.g. B. Accuracy of 90% usage detection. Furthermore, the processing device 120 may also use, for example, prior knowledge of detection algorithms to adapt the occurrence probabilities, i. Uncertainty regarding the recognition already to be determined based on an uncertainty of the algorithm.
  • Embodiments according to the invention introduce a framework for modeling surgical procedures.
  • the framework uses UML activities as an interface for dynamic Bayesian networks (DBN). These models are used for the actual estimation of the progress of a surgical procedure.
  • DBN dynamic Bayesian networks
  • a framework is proposed that permits a formalization of medical devices. workflows allows. It is promoted through a dialogue between medical and technical experts and is based on the Unified Modeling Language (UML).
  • UML Unified Modeling Language
  • a simple, understandable UML activity serves as the starting point for automatically creating complex models that can be used to actually estimate the progress of a surgical procedure.
  • Translation rules allow a given UML activity to be transmitted into a dynamic Bayesian network (DBN).
  • DBN dynamic Bayesian network
  • the procedures are presented for the application of a cholecystectomy (surgical removal of the gallbladder) BIBLIOGRAPHIEVERZEICHNIS

Landscapes

  • Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Economics (AREA)
  • Health & Medical Sciences (AREA)
  • Strategic Management (AREA)
  • Human Resources & Organizations (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Educational Administration (AREA)
  • Marketing (AREA)
  • Development Economics (AREA)
  • Primary Health Care (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Game Theory and Decision Science (AREA)
  • Epidemiology (AREA)
  • Biomedical Technology (AREA)
  • Public Health (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

Un dispositif (100) comprend un moyen de détection (110) conçu pour détecter une zone de travail (160) dans laquelle un cycle d'opérations peut être exécuté et pour produire un signal de détection (112), un moyen de traitement (120) et une interface (130) destinée à produire un signal d'état. Le moyen de traitement (120) est conçu pour obtenir une description d'un modèle de connaissance graphique (122) du cycle d'opérations et pour créer et paramétrer un modèle de séquences dynamique (124) du cycle d'opérations sur la base du modèle de connaissance graphique; et pour déterminer un état du modèle de séquences dynamique du cycle d'opérations sur la base du signal de détection. Le dispositif est conçu pour produire le signal d'état sur la base de l'état déterminé du modèle de séquences dynamique.
PCT/EP2018/000108 2017-03-24 2018-03-22 Dispositif et procédé de détermination d'un état d'un cycle d'opérations Ceased WO2018171940A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
DE102017205048.9 2017-03-24
DE102017205048.9A DE102017205048B4 (de) 2017-03-24 2017-03-24 Vorrichtung und verfahren zur bestimmung eines zustands eines arbeitsablaufs

Publications (1)

Publication Number Publication Date
WO2018171940A1 true WO2018171940A1 (fr) 2018-09-27

Family

ID=61837704

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2018/000108 Ceased WO2018171940A1 (fr) 2017-03-24 2018-03-22 Dispositif et procédé de détermination d'un état d'un cycle d'opérations

Country Status (2)

Country Link
DE (1) DE102017205048B4 (fr)
WO (1) WO2018171940A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102022211634A1 (de) * 2022-11-04 2024-05-08 Forschungszentrum Jülich GmbH Orchestrator-basiertes Forschungs- und Entwicklungssystem und Verfahren zu dessen Betrieb

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140067413A1 (en) * 2012-08-29 2014-03-06 David S. GHIVIZZANI Operating room management system and related methods
US9532848B2 (en) 2007-06-15 2017-01-03 Othosoft, Inc. Computer-assisted surgery system and method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9532848B2 (en) 2007-06-15 2017-01-03 Othosoft, Inc. Computer-assisted surgery system and method
US20140067413A1 (en) * 2012-08-29 2014-03-06 David S. GHIVIZZANI Operating room management system and related methods

Non-Patent Citations (24)

* Cited by examiner, † Cited by third party
Title
"Unified Modeling Language Superstructure", 2011, OMG
A. HOFSTEDE; N. RUSSELL; W. AALST; M. ADAMS: "Modern Business Process Automation: YAWL and its Support Environment", 2010, SPRINGER
A. MCWILLIAMS: "Medical Robotics and Computer-Assisted Surgery: The Global Market", 2014, BAISHIDENG PUBLISHING GROUP INC
C. RUPP; S. QUEINS ET AL.: "UML 2 glasklar", 2012, CARL HANSER VERLAG GMBH CO KG
D. KOLLER; N. FRIEDMAN: "Probabilistic graphical models: principles and technique", 2009, MIT PRESS
F. J. DIEZ: "Parameter adjustment in Bayes networks. The generalized noisy ORgate", PROCEEDINGS OF THE NINTH INTERNATIONAL CONFERENCE ON UNCERTAINTY IN ARTIFICIAL INTELLIGENCE, 1993, pages 99 - 105
F. LALYS; P. JANNIN: "Surgical process modelling: a review", INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, vol. 9, no. 3, 2014, pages 495 - 511
FLORENT LALYS ET AL: "Surgical process modelling: a review", INTERNATIONAL JOURNAL OF COMPUTER ASSISTED RADIOLOGY AND SURGERY, vol. 9, no. 3, 1 May 2014 (2014-05-01), DE, pages 495 - 511, XP055468024, ISSN: 1861-6410, DOI: 10.1007/s11548-013-0940-5 *
G.-Q. DING; W. CAI; M.-F. QIN: "Is intraoperative cholangiography necessary during laparoscopic cholecystectomy for cholelithiasis?", WORLD JOURNAL OF GASTROENTEROLOGY: WJG, vol. 21, no. 7, 2015, pages 2147
H. STÖRRLE: "Semantics of control-flow in UML 2.0 activities", VISUAL LANGUAGES AND HUMAN CENTRIC COMPUTING, 2004 IEEE SYMPOSIUM, 2004, pages 235 - 242, XP010752311, DOI: doi:10.1109/VLHCC.2004.46
I. D, LEARMONTH; C. YOUNG; C. RORABECK: "The operation of the century: total hip replacement", THE LANCET, vol. 370, no. 9597, 2007, pages 1508 - 1519, XP022314334, DOI: doi:10.1016/S0140-6736(07)60457-7
K, P. MURPHY: "Ph.D. dissertation", 2002, UNIVERSITY OF CALIFOMIA, article "Dynamic bayesian networks: representation, inference and learning"
N. J. SOPER; L. L. SWANSTRÖM; S. EUBANKS: "Mastery of endoscopic and laparoscopic surgery", 2008, LIPPINCOTT WILLIAMS & WILKINS
N. PADOY: "Ph.D. dissertation", 2010, UNIVERSITE HENRI POINCARE-NANCY I, article "Workflow and activity modeling for monitoring surgical procedures"
N. PADOY; T. BLUM; H. FEUSSNER; M.-O. BERGER; N. NAVAB: "On-line recognition of surgical activity for monitoring in the operating room", 2008, AAAI, pages: 1718 - 1724
P. PHILIPP; L. SCHREITER; J. GIEHL; Y. FISCHER; J. RACZKOWSKY; M. SCHWARZ; H. WOERN; J. BEYERER: "Situation detection for an interactive assistance in surgical interventions based on dynamic bayesian networks", CRAS 2016, 6TH JOINT WORKSHOP ON NEW TECHNOLOGIES FOR COMPUTER/ROBOT ASSISTED SURGERY, 2016
P. PHILIPP; Y. FISCHER; D. HEMPEL; J. BEYERER: "Emerging Trends in Applications and Infrastructures for Computational Biology, Bioinformatics, and Systems Biology", 2016, ELSEVIER, article "Framework for an interactive assistance in diagnostic processes based on probabilistic modeling of clinical practice guidelines", pages: 371 - 390
P. PHILIPP; Y. FISCHER; D. HEMPEL; J. BEYERER: "Modeling of clinical practice guidelines for interactive assistance in diagnostic processes", HIMS 2015, INTERNATIONAL CONFERENCE ON HEALTH INFORMATICS AND MEDICAL SYSTEMS, 27 July 2015 (2015-07-27), pages 3 - 9
R. A. FAUST: "Robotics in surgery: history, current and future applications", 2007, NOVA PUB-IISHERS
S. GLÖSER, DTSCH ARZTEBL INTERNATIONAL, vol. 112, no. 23, 2015, pages 4, Retrieved from the Internet <URL:http://www.aerzteblatt.de/int/article.asp?id=170832>
STORRLE H: "Semantics of Control-Flow in UML 2.0 Activities", VISUAL LANGUAGES AND HUMAN CENTRIC COMPUTING, 2004 IEEE SYMPOSIUM ON ROME 26-29 SEPT. 2004, PISCATAWAY, NJ, USA,IEEE, 26 September 2004 (2004-09-26), pages 235 - 242, XP010752311, ISBN: 978-0-7803-8696-9, DOI: 10.1109/VLHCC.2004.46 *
T. DEAN; K. KANAZAWA: "A model for reasoning about persistence and causation", COMPUTATIONAL INTELLIGENCE, vol. 5, no. 2, 1989, pages 142 - 150
W. VAN DER AALST; K. M. VAN HEE: "Workflow management: models, methods, and systems", 2000, MIT PRESS
Z. HAN; L. ZHANG; J. LING: "Enterprise Interoperability IV", 2010, SPRINGER, article "Transformation of uml activity diagram to yawl", pages: 289 - 299

Also Published As

Publication number Publication date
DE102017205048B4 (de) 2025-03-06
DE102017205048A1 (de) 2018-09-27

Similar Documents

Publication Publication Date Title
DE69736552T2 (de) Intelligente anwenderunterstützungsfunktion
Zeleny Management support systems: Towards integrated knowledge management
Khang et al. Application of computer vision (cv) in the healthcare ecosystem
Forestier et al. Automatic matching of surgeries to predict surgeons’ next actions
Jabbar et al. Survey on development of expert system in the areas of Medical, Education, Automobile and Agriculture
De Grasf et al. A case study in structured knowledge acquisition
Shaalan et al. A multiagent approach for diagnostic expert systems via the internet
EP4627419A1 (fr) Procédé d&#39;auto-apprentissage pour un processus de fabrication et/ou de montage
WO2018171940A1 (fr) Dispositif et procédé de détermination d&#39;un état d&#39;un cycle d&#39;opérations
Langer et al. How morphological computation shapes integrated information in embodied agents
DE10132014A1 (de) Verfahren zum Erstellen eines Kausalen Netzes auf Grundlage einer Wissensakquisition
DE102022103737A1 (de) Computergestütztes Assistenzsystem und Verfahren
Song et al. Addressing challenges to problem complexity: Effectiveness of AI assistance during the design process
EP3923218A1 (fr) Procédé mis en uvre par ordinateur d&#39;aide numérique à la formation et système mis en uvre par ordinateur d&#39;aide numérique à la formation
Miah et al. Ontology development for context-sensitive decision support
WO2024008232A1 (fr) Procédé de détection dynamique d&#39;encliquetage dans un robot collaboratif par intelligence artificielle et compensation dynamique des trajectoires
DE102005009056A1 (de) Verfahren zum Betrieb eines medizinischen Informationssystems
DE102014000086A1 (de) Arbeitsverfahren für Behandlung von abstrakten Objekten (Gedanke-Substanzen) von einem Computersystem von Künstlicher Intelligenz von einem Cyborg oder einem Android.
Ogden Human computer interaction in complex process control: developing structured mental models that allow operators to perform effectively
Menzies Limits to Knowledge Level-B Modeling (and KADS)
DE19626142A1 (de) Computer-Aided-Text-Design-System
DE102007014970B3 (de) Konfiguration von medizintechnischen Prozessen
EP1480149A2 (fr) Procédé et dispositif de traitement et de délivrance d&#39;enregistrements de données comprenant des versions modifiées des indications de thérapie médicale
WO2025061379A1 (fr) Procédé mis en œuvre par ordinateur pour faire fonctionner un appareil médical, appareil médical, programme informatique et support de données lisible électroniquement
DE102023209205A1 (de) System und Verfahren zur Segmentierung medizinischer Bildgebungsdaten

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18714681

Country of ref document: EP

Kind code of ref document: A1

DPE1 Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101)
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18714681

Country of ref document: EP

Kind code of ref document: A1