WO2025169500A1 - Processing device, processing program, processing method, and processing system - Google Patents

Abstract

[Problem] To provide a more reliable processing device, processing program, processing method, and processing system. [Solution] Provided is a processing device that, for an assessment result that represents the possibility of morbidity of one or a plurality of diseases and is generated by acquiring, via a communication interface, at least one among a subject image acquired by an imaging device configured to capture an image of a subject who is a target person, medical interview information of the target person, and observation information of the target person and inputting the at least one among the subject image, the medical interview information, and the observation information which was acquired into a trained evaluation model, executes processing for generating degree-of-effect information representing the extent of the effect exerted by the at least one among the subject image, the medical interview information, and the observation information input into the learned evaluation model.

Description

Processing device, processing program, processing method, and processing system

The present disclosure relates to a processing device, processing program, processing method, and processing system configured to execute processing related to a determination result indicating the possibility of contracting a disease.

Technology for capturing subject images of a subject and assessing predicted diseases has been known for some time. Patent Document 1 describes "an information processing system including an acquisition unit that acquires input information including 3D data related to MRI images of the brain of a single subject, and an output unit that uses the input information as input data and outputs disease name information or disease assessment support information related to Parkinson's syndrome for the single subject based on a machine learning model." However, with this technology, the reliability of the output information obtained from the machine learning model based on the input information was unknown.

Japanese Patent Application Laid-Open No. 2023-161222

In light of the above-mentioned technologies, the present disclosure aims to provide, through various embodiments, a more reliable processing device, processing program, processing method, and processing system.

One aspect of the present disclosure provides a processing device including at least one processor, wherein the at least one processor is configured to acquire, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject, medical interview information about the subject, and finding information about the subject, and to execute processing to generate impact information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to a trained judgment model on a judgment result indicating the possibility of the subject being afflicted with one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model.

According to one aspect of the present disclosure, there is provided a processing program that, when executed by at least one processor, causes the at least one processor to function to acquire, via a communication interface, at least one of a subject image acquired by an imaging device configured to capture an image of a subject, medical interview information about the subject, and finding information about the subject, and to input at least one of the acquired subject image, medical interview information, and finding information into a trained judgment model to generate impact information indicating the degree of impact of at least one of the subject image, medical interview information, and finding information input into the trained judgment model on a judgment result indicating the possibility of the subject being afflicted with one or more diseases.

According to one aspect of the present disclosure, there is provided a processing method executed by at least one processor, the processing method including the steps of: acquiring, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject, medical interview information about the subject, and finding information about the subject; and generating impact information indicating the degree of impact of at least one of the subject image, medical interview information, and finding information input to a trained judgment model on a judgment result indicating the possibility of the subject being affected by one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model.

According to one aspect of the present disclosure, there is provided a processing system including a terminal device and a processing device communicatively connected to the terminal device, wherein at least one processor included in the terminal device selects at least one of a subject image acquired by an imaging device configured to capture an image of a subject, medical interview information about the subject, and finding information about the subject, and at least one processor included in the processing device acquires at least one of the selected subject image, medical interview information, and finding information via a communications interface, and executes processing to generate impact information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to a trained judgment model on a judgment result indicating the possibility of the subject being afflicted with one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model.

This disclosure makes it possible to provide a more reliable processing device, processing program, processing method, and processing system.

Note that the above effects are merely illustrative for the sake of convenience and are not limiting. In addition to or instead of the above effects, any effect described in this disclosure or any effect that would be obvious to a person skilled in the art may be achieved.

FIG. 1 is a block diagram showing the configuration of a processing system 1 according to an embodiment of the present disclosure. FIG. 2A is a block diagram showing the configuration of the server device 100 according to an embodiment of the present disclosure. FIG. 2B is a block diagram showing the configuration of the terminal device 300 according to an embodiment of the present disclosure. FIG. 3A is a diagram conceptually illustrating a subject management table stored in server device 100 according to an embodiment of the present disclosure. FIG. 3B is a diagram conceptually illustrating a contribution management table stored in the server device 100 according to an embodiment of the present disclosure. FIG. 3C is a diagram conceptually illustrating an attention area management table stored in the server device 100 according to an embodiment of the present disclosure. FIG. 3D is a diagram conceptually illustrating a subject image stored in the server device 100 according to an embodiment of the present disclosure. FIG. 4 is a diagram showing a processing sequence executed by the processing system 1 according to an embodiment of the present disclosure. FIG. 5 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. FIG. 6 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. FIG. 7 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. FIG. 8A is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 8B is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9A is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9B is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9C is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9D is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9E is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 11 is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure.

1. Overview of Processing System 1 The processing system 1 according to the present disclosure generates influence information indicating the degree of influence of at least one of the subject image, the medical interview information, and the finding information input to a trained determination model on a determination result indicating the possibility of a disease for at least one of one or more diseases based on at least one of the subject image, the medical interview information, and the finding information of the subject. In this way, the basis of the determination result determined by the processing system 1 can be confirmed based on the influence information, thereby making it possible to increase the reliability of the generated determination result.

In the present disclosure, the one or more diseases may be any disease for which a determination result indicating the possibility of affliction can be obtained based on at least one of the subject image, medical interview information, and finding information, as described above. Among such diseases, diseases in which findings are found in natural orifices are preferred. Furthermore, among such diseases, diseases in which findings are found in the oral cavity, pharynx, larynx, etc. are preferred, including infectious diseases such as influenza, coronavirus infection, streptococcal infection, adenovirus infection, Epstein-Barr virus infection, mycoplasma infection, hand, foot, and mouth disease, herpangina, candidiasis, and other infections; diseases presenting vascular or mucosal disorders such as arteriosclerosis, diabetes, and hypertension; tumors such as tongue cancer and pharyngeal cancer; periodontal diseases such as dental caries, gingivitis, and periodontal disease; and combinations thereof. Among such diseases, influenza, which exhibits a unique pattern in lymphoid follicles appearing in the deepest part of the pharynx located within the oral cavity, is particularly preferred. The following description uses influenza as an example of a disease, but of course, this is not limited to this.

Furthermore, in this disclosure, the determination result broadly includes the result of determining the possibility of disease based on at least one of the subject image, medical interview information, and finding information. Such determination results are obtained, for example, by analysis using a trained determination model. In other words, the determination result includes not only a definitive determination by a medical professional, etc., but also a determination result obtained using a trained determination model, or a result used to assist a medical professional in making a definitive determination.

Furthermore, in this disclosure, the subject, who is the subject of imaging using an imaging device, includes all people, such as patients, test subjects, persons being evaluated, and healthy individuals. Furthermore, in this disclosure, the operator who holds the imaging device and operates the imaging device is not limited to medical professionals such as doctors, nurses, and laboratory technicians, but also includes all people, such as the subject themselves and the subject's guardian. In other words, for the sake of convenience, the subject and operator are given names based on the actions they perform, but they may be different people or the same person. In the following, these people will be collectively referred to as users.

2. Configuration of Processing System 1 FIG. 1 is a block diagram showing the configuration of a processing system 1 according to an embodiment of the present disclosure. According to FIG. 1 , the processing system 1 includes a server device 100, an imaging device 200, and a terminal device 300, which are communicatively connected via a wired or wireless network. The server device 100 uses a trained determination model to determine the likelihood of a patient suffering from a disease based on at least one of a subject image, medical interview information, and finding information acquired from at least one of the imaging device 200 and the terminal device 300, and generates influence information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to the trained determination model on the determination result. The imaging device 200 has its tip inserted into the oral cavity of a subject to capture images of the oral cavity, particularly the pharynx, and transmits the captured subject images to the server device 100 via a wired or wireless network. The terminal device 300 inputs subject information, medical interview information, findings information, etc. required for processing in the server device 100, and receives and outputs the determination results of the possibility of contracting one or more diseases and impact information from the server device 100.

In this disclosure, the processing device refers to the server device 100, the terminal device 300, or a combination thereof. In other words, although the following describes the case where the server device 100 functions as a processing device, the terminal device 300 can also function as a processing device. In addition, in this disclosure, the storage and processing performed by the processing device may be distributed to other terminal devices, other server devices, etc. In other words, the processing device is not limited to devices configured from a single housing, but includes the server device 100, the imaging device 200, the terminal device 300, other terminal devices, other server devices, or a combination thereof.

Furthermore, in Figure 1, only one terminal device 300 and one imaging device 200 are shown. However, in a large-scale medical institution, for example, it is possible that multiple terminal devices 300 or multiple imaging devices 200 may be managed and operated.

FIG. 2A is a block diagram showing the configuration of a server device 100 according to an embodiment of the present disclosure. According to FIG. 2A, the server device 100 includes a processor 111, a memory 112, and a communication interface 113. These components are electrically connected to each other via control lines and data lines. The server device 100 does not need to include all of the components shown in FIG. 2A; it is possible to omit some components, or to add other components. For example, it is possible to use an external memory, a database device, a server device, etc., which are communicatively connected as memory. It is also possible to distribute some of the processing to processing devices, including other server devices. In other words, the server device 100 is not limited to a single device, but may be distributed across multiple devices depending on the information handling and processing load.

The processor 111 functions as a control unit that controls the other components of the processing system 1 based on a processing program stored in the memory 112. Based on the processing program stored in the memory 112, the processor 111 executes processing to generate impact information indicating the degree of influence that at least one of the subject image, medical interview information, and finding information input to the trained judgment model has had on the judgment result, primarily based on the processing program stored in the memory 112. Specifically, based on the processing program stored in the memory 112, the processor 111 executes processing such as "acquiring, via the communication interface 113, at least one of the subject image, medical interview information, and finding information of the subject, which are acquired via at least one of the imaging device 200 configured to image the subject or the terminal device 200" and "generating impact information indicating the degree of influence that at least one of the subject image, medical interview information, and finding information input to the trained judgment model has had on a judgment result indicating the possibility of contracting one or more diseases, which is generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model." The processor 111 is primarily composed of one or more CPUs, but may also be combined with a GPU, FPGA, etc. as appropriate.

Memory 112 is composed of RAM, ROM, non-volatile memory, HDD, SSD, etc., and functions as a storage unit. Memory 112 stores instructions and commands for various controls of processing system 1 according to this embodiment as processing programs. Specifically, memory 112 stores programs to be executed by processor 111, such as "a process of acquiring, via communication interface 113, at least one of a subject image, medical interview information about the subject, and finding information about the subject, acquired via at least one of an imaging device 200 configured to image the subject or a terminal device 200" and "a process of generating impact information indicating the degree of influence of at least one of a subject image, medical interview information, and finding information input to a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, generated by inputting at least one of the acquired subject image, medical interview information, and finding information into a trained judgment model." In addition to the program, memory 112 also stores various information stored in a subject management table, a contribution management table, an area of interest management table, etc. Note that this information does not need to be constantly stored in memory 112 within server device 100, but may instead be stored in a database device installed remotely. In this case, the database device is also included in memory 112.

The communication interface 113 functions as a notification unit for sending and receiving various information to and from the image capture device 200 and terminal device 300 connected via a wired or wireless network. Examples of the communication interface 113 include wired communication connectors such as USB and SCSI, broadband wireless communication devices such as wireless LAN, Bluetooth (registered trademark) and LTE, wireless communication transmission/reception devices such as infrared, and various connection terminals for printed circuit boards and flexible circuit boards. The communication interface 113, for example, receives subject images from the image capture device 200 and sends impact information to and from the terminal device 300.

2B is a block diagram showing the configuration of a terminal device 300 according to an embodiment of the present disclosure. According to FIG. 2B, the terminal device 300 includes a processor 311, a memory 312, an input interface 313, an output interface 314, and a communication interface 315. These components are electrically connected to each other via control lines and data lines. It is noted that the terminal device 300 does not need to include all of the components shown in FIG. 2B; some components may be omitted, or other components may be added. The terminal device 300 may be any device capable of communicating with the server device 100 via a wired or wireless network. Examples of the terminal device 300 include a smartphone, tablet, laptop PC, desktop PC, and camera. It is noted that the terminal device 300 does not necessarily need to be communicatively connected to the camera device 200, but it is preferable that they be communicatively connected. Such terminal devices 300 include medical institution terminal devices installed in medical institutions that diagnose the possibility of disease infection and used by medical professionals, subject terminal devices used by subjects and their associates (such as guardians), user terminal devices used by users who operate the imaging device 200, and information provider terminal devices used by information providers who provide information to medical professionals and others based on impact information. However, the terminal devices exemplified here are merely names given to distinguish them according to their functions and the attributes of the main users. In other words, one terminal device can function as multiple terminal devices.

The processor 311 functions as a control unit that controls the other components of the processing system 1 based on the processing program stored in the memory 312. Based on the processing program stored in the memory 312, the processor 311 sends a request to generate impact information and outputs impact information generated by the server device 100. Specifically, the processor 311 executes the following processes based on the processing program stored in the memory 312: "selecting, via the input interface 313, at least one of a subject image acquired via the imaging device 200 configured to image the subject, the subject's medical interview information, and the subject's findings information," "sending, via the communication interface 315, at least one of the selected subject image, the subject's medical interview information, and the subject's findings information to the server device 100," "acquiring impact information from the server device 100 via the communication interface 315," and "outputting impact information acquired via the output interface 314." The processor 111 is primarily composed of one or more CPUs, but may also be combined with a GPU, FPGA, etc. as appropriate.

The memory 312 is composed of RAM, ROM, non-volatile memory, HDD, SSD, etc., and functions as a storage unit. The memory 312 stores instructions and commands for various controls of the processing system 1 according to this embodiment as processing programs. Specifically, the memory 312 stores processing programs to be executed by the processor 311, such as "a process of selecting, via the input interface 313, at least one of a subject image acquired via the imaging device 200 configured to image the subject, the subject's medical interview information, and the subject's findings information," "a process of transmitting, via the communication interface 315, at least one of the selected subject image, the subject's medical interview information, and the subject's findings information to the server device 100," "a process of acquiring impact information from the server device 100 via the communication interface 315," and "a process of outputting the impact information acquired via the output interface 314."

The input interface 313 functions as an input unit that accepts user input operations for the terminal device 300. Examples of the input interface 313 include physical key buttons and a touch panel having an input coordinate system that corresponds to the display coordinate system of the display. In the case of a touch panel, icons are displayed on the display, and the operator selects each icon by inputting instructions via the touch panel. The method for detecting the user's instruction input via the touch panel may be any method, such as capacitive or resistive. The input interface 313 does not always need to be physically provided on the terminal device 300, and may be connected as needed via a wired or wireless network. Therefore, in addition to the above, a mouse, keyboard, etc. can also be used as the input interface 313.

The output interface 314 functions as an output unit for outputting information such as impact information received from the server device 100. An example of the output interface 314 is a display configured as a liquid crystal panel, an organic EL display, or a plasma display. However, the terminal device 300 itself does not necessarily need to be equipped with a display. For example, an interface for connecting to a display or the like that can be connected to the terminal device 300 via a wired or wireless network can also function as the output interface 314 that outputs display data to that display or the like.

The communication interface 315 functions as a communication unit for sending and receiving determination requests, generation requests, determination results, impact information, etc., to and from the server device 100 and the image capture device 200 connected via a wired or wireless network. Examples of the communication interface 315 include wired communication connectors such as USB and SCSI, wireless LAN, Bluetooth (registered trademark), wideband wireless communication such as LTE, wireless communication transmission/reception devices such as infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

3. Various Information Used in Processing in the Processing System 1 Figures 3A to 3C show various tables that store information that is stored in the server device 100 and provided to the imaging device 200 and terminal device 300 as the processing progresses. This information is updated and stored as the processing progresses. Note that the information shown in Figures 3A to 3C may be stored in the memory 112 of the server device 100, or may be stored in another remotely installed database device, electronic medical record device, or the like, and read out as the processing progresses.

3A is a diagram conceptually illustrating a subject management table stored in a server device 100 according to an embodiment of the present disclosure. According to FIG. 3A, the subject management table stores, in association with subject ID information, assessment result information, medical interview information, findings information, subject image information, and history information. "Subject ID information" is information unique to each subject that identifies each subject. For example, subject ID information is generated each time a new subject is registered by a user. However, subject ID information may be any information that can identify the subject as described above, and is not limited to such information. It may also be unique identification information assigned by institutions such as Japan or other foreign countries, their local governments, schools, or workplaces, or a composite key created by combining multiple types of identification information. "Assessment result information" is information indicating the results of an assessment of the subject's likelihood of contracting one or more diseases. For example, such assessment result information is generated by inputting at least one of the subject image, medical interview information, and findings information as input information into a trained assessment model in the server device 100.

"Medical interview information" is information entered by, for example, the subject or a medical professional, etc., and is used as a reference for a doctor's diagnosis, such as the subject's medical history and symptoms. Examples of such medical interview information include patient background information such as weight, allergies, and underlying diseases; body temperature, peak body temperature since onset, time elapsed since onset; heart rate, pulse rate, oxygen saturation, blood pressure, medication status, contact with other infectious disease patients; subjective symptoms and physical findings such as joint pain, muscle pain, headache, fatigue, loss of appetite, chills, sweating, cough, sore throat, runny nose/nasal congestion, tonsillitis, gastrointestinal symptoms, rash on the hands and feet, redness or white coating of the pharynx, swollen tonsils, history of tonsillectomy, strawberry tongue, and swollen anterior cervical lymph nodes with tenderness; history of infectious disease vaccination; and timing of vaccination. The medical interview information is selected or entered by accepting instructions input from the subject or medical professional via the input interface 313 in the terminal device 300. The information is then sent from the terminal device 300 to the server device 100 and stored in the subject management table.

"Finding information" is information entered by a medical professional such as a doctor, and indicates any abnormalities obtained through various examinations of the subject, such as visual examination, questioning, palpation, auscultation, or percussion, or tests to assist in diagnosis. Examples of such finding information include redness or white coating of the pharynx, swelling of the tonsils, the presence or absence of tonsillitis, redness or white coating of the tonsils, etc. The finding information is selected or entered by accepting instructions input by the medical professional via the input interface 313 in the terminal device 300. It is then sent from the terminal device 300 to the server device 100, where it is stored in the subject management table.

"Subject image information" is image data of a subject image that includes at least the subject's pharynx as a subject, captured using the imaging device 200 or the like. The image data may be one or more still images, one or more videos, or a combination thereof. The subject image information is stored by being received from at least one of the imaging device 200 and the terminal device 300 via the communication interface 113. The subject image information may be the image data itself captured by the imaging device 200, or may be data after image processing such as sharpening has been performed on the image data. The subject image information is typically useful for diagnosing the possibility of infection with an infectious disease in which findings are found in the oral cavity, including the pharynx. Specifically, the subject image information is used by medical professionals such as doctors to make reference to and make findings, or the subject image information is used in a processing device to diagnose the possibility of infection with one or more diseases or to generate information to assist in that diagnosis. Note that such subject image information may be various information analyzed from the image data, instead of or together with the image data. For example, it may include values, classifications, or categories obtained by inputting image data into a trained analysis model to quantify things like redness of the throat or swelling of the tonsils.

Here, Figure 3D is a diagram conceptually illustrating subject images stored in server device 100 according to an embodiment of the present disclosure. According to Figure 3D, subject image M1 is an image captured of the pharynx, which is the subject of imaging device 200, and the surrounding area. Subject image M1 shows that follicles, which are characteristic findings of influenza, are observed in areas 37 and 38. In this way, subject images can be suitably used to determine the possibility of contracting one or more diseases based on visual findings or a trained determination model.

Returning to Figure 3A, "history information" includes subject images previously taken of the subject, previously acquired findings, previously acquired interview information, previously acquired assessment result information, and previously acquired impact information, as well as information indicating the time at which each was acquired. Such history information is preferably used for comparison when outputting the current assessment result or impact information. As described above, this history information stores information acquired at multiple points in the past. Therefore, a history information management table may be provided for each subject ID information, separate from the subject management table, and history information may be managed in that table.

Although not specifically shown, other attribute information such as the subject's name, age, gender, and date of examination may also be included.

FIG. 3B is a diagram conceptually illustrating a contribution management table stored in server device 100 according to an embodiment of the present disclosure. According to FIG. 3B, the contribution management table stores contribution information in association with judgment item information. This contribution management table is generated in association with the subject ID information of the specified subject each time a request for generating an impact is received from terminal device 300. When processing based on the generated contribution information is completed, the information stored in the contribution management table is stored as history information in association with the associated subject ID information.

The "determination item information" includes at least one of the subject image, medical interview information, and finding information input as input information in the trained determination model to generate a determination result. The information may be automatically acquired from information input as input information when the server device 100 performs a determination process to obtain a determination result indicating the possibility of having one or more diseases. Alternatively, the information may be acquired based on an operator's selection from information input as input information when the server device 100 performs a determination process to obtain a determination result indicating the possibility of having one or more diseases.
"Contribution information" is one type of influence information. The contribution information is information indicating the degree of influence that at least one of the subject image, medical interview information, and finding information input to the trained judgment model, i.e., each piece of information acquired as judgment item information, has had on the judgment result acquired by the trained judgment model. More specifically, the contribution information is information indicating the degree to which each piece of information acquired as judgment item information deviates from the average predicted value of the trained judgment model as a result of inputting the information into the trained judgment model. An example of such contribution information is the Shapley value. The process of generating contribution information will be described in detail below.

As an example, contribution information may be expressed as a specific numerical value, with a higher value indicating a higher contribution, i.e., an item that the trained judgment model paid more attention to when generating the judgment result, and a lower value indicating a lower contribution, i.e., an item that the trained judgment model paid less attention to when generating the judgment result. Note that contribution information may also be a classification or category other than a numerical value, such as "high," "medium," or "low."

Note that Figure 3B gives examples of assessment item information, such as pulse rate, body temperature, contact with the patient, joint pain, and loss of appetite, but it may naturally include other interview information and findings information not shown here. Also, Figure 3B gives an example of only one specific subject image as the subject image, but it may naturally include multiple subject images.

FIG. 3C is a diagram conceptually illustrating a region of interest management table stored in server device 100 according to an embodiment of the present disclosure. According to FIG. 3C, the region of interest management table stores region of interest information in association with subject image ID information. This region of interest management table is generated in association with the subject ID information of the specified subject each time a request for generating an impact level is received from terminal device 300. When processing based on the generated region of interest information is completed, the information stored in the region of interest management table is stored as history information in association with the associated subject ID information.

"Subject image ID information" is information used to identify a subject image that includes at least the subject's pharynx as a subject, captured using the imaging device 200, etc. As an example, subject image information is generated in association with a subject image each time the subject image is captured using the imaging device 200.

"Area of interest information" is a type of influence information. This area of interest information indicates the degree of influence that a subject image input into a trained judgment model had on the judgment result obtained by the trained judgment model. More specifically, when a judgment result is generated by inputting a subject image into a trained judgment model, the area of interest information indicates the area in the subject image that had a large influence on the judgment result. This influence information indicates the degree of influence on the judgment result in correspondence with the position information of the subject image. The process of generating contribution information will be described in detail below.

As an example, the attention area information is represented by a specific numerical value corresponding to the coordinates of the subject image, with a higher value indicating a higher degree of attention, i.e., the area that the trained judgment model paid more attention to when generating the judgment result, and a lower value indicating a lower degree of attention, i.e., the area that the trained judgment model paid less attention to when generating the judgment result. Note that the contribution information may be a classification or category other than a numerical value, such as "high," "medium," or "low." Furthermore, an attention area image may be generated in which an image showing a numerical value is arranged in correspondence with the position information of the subject image, and this image may be used as the attention area information.

4 is a diagram showing a processing sequence executed by the processing system 1 according to an embodiment of the present disclosure. Specifically, Fig. 4 shows a processing sequence executed between the server device 100, the image capture device 200, and the terminal device 300. Of these, S11 to S16 mainly represent processing for acquiring a subject image from at least one of the image capture device 200 and the terminal device 300, S21 to S24 represent processing for generating a determination result in the server device 100, and S31 to S38 represent processing for generating impact information in the server device 100.

(1) Subject Image Acquisition Process First, the subject image acquisition process will be described. Referring to FIG. 4 , the power button or the like is pressed to turn on the image capture device 200, and the image capture device 200 is activated (S11). Then, the image capture device 200 selects subject information of a subject to be captured from the subject information of uncaptured subjects received from the server device 100, based on the operator's instruction input received via the input interface 210 (S12). Next, the image capture device 200 determines whether or not an auxiliary tool is attached to the tip of the image capture device 200 to assist in inserting the image capture device into the oral cavity. If the auxiliary tool is not yet attached, the image capture device 200 outputs a display prompting the user to attach the auxiliary tool via the output interface (S13). Note that this display is merely an example, and the user may also be prompted to attach the auxiliary tool by sound, flashing light, vibration, or other means. When the image capture device 200 detects that the auxiliary tool has been attached (S14), the image capture device 200 captures a subject image based on the operator's instruction input received via the input interface (S15). Although the wearing of the assistive device is detected here, this process may be skipped.

Once the subject image is captured, the imaging device 200 transmits the captured subject image (T11) to the server device 100 via the communication interface, along with the subject ID information of the subject who was photographed. When the server device 100 receives the subject image via the communication interface 113, it stores the subject image in the subject management table in association with the subject ID information received at the same time.

Note that in Figure 4, the server device 100 has been described as acquiring a subject image from the imaging device 200, but it is also possible for the imaging device 200 to capture a subject image and transmit it to the terminal device 200, and for the server device 100 to acquire the subject image from the terminal device 200.

(2) Processing for Generating Assessment Results Next, the processing for generating assessment results will be described. This processing is performed, for example, each time an assessment request is generated by the operator. As shown in FIG. 4 , the processor 311 of the terminal device 300 accepts selection of subject information for the subject to be assessed via the input interface 313 on a screen displaying a list of subject information for the subjects (S21). At this time, although not specifically shown, the terminal device 300 generates subject ID information, medical interview information, and findings information for the subject based on the input by the operator, and transmits the information to the server device 100 via the communication interface 315. Upon receiving the subject ID information, medical interview information, and findings information via the communication interface 115, the server device 100 stores the information in a subject management table.

Furthermore, when determining the possibility of contracting one or more diseases, the processor 311 of the terminal device 300 accepts operational input from the operator via the input interface 313 and selects a judgment threshold (S22). When the possibility of contracting one or more diseases is obtained as output information in the trained judgment model, accuracy information indicating the accuracy of the output information is also obtained. The judgment threshold is used to determine that there is a possibility of contracting the disease if the accuracy exceeds the judgment threshold, and to determine that there is no possibility of contracting the disease if the accuracy is equal to or less than the judgment threshold. In other words, S22 indicates that the operator can change this judgment threshold to any value.

Here, FIG. 8A is a diagram showing an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. Specifically, it is a diagram showing an example of a determination instruction screen 20 output on the terminal device 300 when selecting a determination threshold performed in S22 of FIG. 4. According to FIG. 8A, the determination instruction screen 20 includes, at the top of the screen, attribute information of the subject selected in S21 of FIG. 4, such information as the subject's name, age, sex, and examination date. The determination instruction screen 20 also includes, at the bottom, a medical interview information display area 21 and a finding information display area 22 that display the medical interview information and finding information generated in S21 of FIG. 4. Furthermore, the determination instruction screen 20 includes a subject image display area 23 that displays the subject image of the subject captured in S15 of FIG. 4 obtained from the server device 100 or the imaging device 200.

Furthermore, the judgment instruction screen 20 has a judgment threshold adjustment area 25 for adjusting the judgment threshold. The judgment threshold adjustment area 25 includes a slide bar and slide button 26 that allow the operator to select the desired judgment threshold between thresholds T1 and T2. When the processor 311 of the terminal device 300 receives an operation input from the operator using the slide button 26, it moves the slide button 26 on the slide bar in accordance with the operation input. This allows the operator to select the desired judgment threshold between judgment threshold T1 (a lower threshold value) and judgment threshold T2 (a higher threshold value).

Furthermore, the judgment instruction screen 20 includes, below the judgment threshold adjustment area 25, a sensitivity display area 27 and a specificity display area 28, which are display areas for other judgment parameters that are adjusted in conjunction with adjusting the judgment threshold. In this embodiment, a trained judgment model is typically used to judge the possibility of contracting a specified disease. The sensitivity display area 27 displays a slide bar and slide button indicating the sensitivity, which indicates the probability that the trained judgment model will correctly judge a positive result. Similarly, the specificity display area 28 displays a slide bar and slide button indicating the sensitivity, which indicates the probability that the trained judgment model will correctly judge a negative result. The processor 311 of the terminal device 300 controls the display so that these slide buttons also move in accordance with the movement of the slide button 26 in the judgment threshold adjustment area 25. For example, in the example of Figure 8A, when the slide button 26 in the judgment threshold adjustment area 25 is moved to the left (toward a lower threshold), the slide button in the sensitivity display area 27 moves accordingly in the direction of increasing sensitivity, and the slide button in the specificity display area 28 moves accordingly in the direction of decreasing specificity.

Once the adjustment of the judgment thresholds and the like is complete, the processor 311 of the terminal device 300 accepts the operator's input to the judgment button 24 via the input interface 314, and sends a judgment request shown as T21 in Figure 4 to the server device 100.

As such, Figure 8A allows the user to more intuitively adjust the judgment threshold using the slide button in the judgment threshold adjustment area 25. Furthermore, the slide buttons in the sensitivity display area 27 and specificity display area 28 also move in response to adjustments to the slide button, allowing the operator to more accurately grasp the relationship between the judgment threshold and the sensitivity and specificity. This allows, for example, when determining the possibility of contracting a specific disease, the judgment threshold can be adjusted to be lower and the sensitivity higher when screening infected patients or when more proactively preventing the spread of infection during an epidemic. Furthermore, when more reliable judgments are desired during non-epidemic periods, for example, the judgment threshold can be adjusted to be higher and the specificity higher.

Note that while a slide bar and slide button are used to select the judgment threshold in Figure 8A, this is merely an example. For example, a box may be provided in which a desired numerical value can be input or selected as the judgment threshold. Furthermore, while Figure 8A uses sensitivity and specificity as examples of judgment parameters that are adjusted in response to adjustment of the judgment threshold, other judgment parameters such as positive agreement rate and negative agreement rate may also be used in addition to or instead of these.

Furthermore, a different threshold value may be selected for each disease to be diagnosed, or the same threshold value may be selected for all diseases to be diagnosed.

Returning to FIG. 4, the processor 311 of the terminal device 300 then references the subject management table to read the subject ID information of the subject, and transmits this to the server device 100 via the communication interface 315, together with a request (T21) to determine the possibility of suffering from one or more diseases and a determination threshold. Upon receiving the determination request, the server device 100 references the subject image in the subject management table based on the subject ID information received together, and reads out the subject image, medical interview information, and findings information associated with the subject ID information. The server device 100 then determines the possibility of suffering from each disease based on the read subject image, and stores the determination result in the subject management table (S23). Details of the determination process will be described later.

Once the determination is made, the server device 100 outputs the stored determination result (T22) to the terminal device 300 that sent the determination request. The processor 311 of the terminal device 300 displays the determination result received via the communication interface 315 on the display via the output interface 314.

(3) Impact Information Generation Process Next, the process for generating impact information will be described. This process is performed, for example, each time an impact generation request is generated by an operator. As shown in FIG. 4 , the processor 311 of the terminal device 300 accepts an operator's input via the input interface 313 on the impact request screen for a specific subject and selects a judgment item for which a contribution level, which is one of the impact levels, is to be generated (S31). The processor 311 of the terminal device 300 also accepts an operator's input via the input interface 313 on the impact request screen and selects a subject image for which a region of interest, which is one of the impact levels, is to be generated (S32). The processor 311 of the terminal device 300 then accepts an operator's input via the input interface 313 on the impact request screen and selects an execute button to generate impact information (S33).

Then, the processor 311 of the terminal device 300 reads the subject ID information of the subject on whose impact request screen the impact request screen is displayed, and transmits it to the server device 100 via the communication interface 315, together with a request to generate impact information (T31) and information identifying the judgment items and subject image selected in S31 and S32.

Here, FIG. 8B is a diagram showing an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. Specifically, FIG. 8B is a diagram showing an example of an impact request screen 10 output on the terminal device 300. According to FIG. 8B, the subject's name, age, gender, and consultation date are output at the top of the screen as the subject's attribute information.

The impact request screen 10 also includes an interview information selection area 11, a findings information selection area 12, and a subject image selection area 13. The interview information selection area 11 accepts input from the operator via the input interface 313, and outputs the interview information (determination item) selected as the target for generating an impact from the interview information input as input information to the determined learning model. This selection may be made by any of the following methods: the operator inputs the desired interview information as text via the input interface 313; check boxes, etc. are prepared in association with each piece of interview information and the operator selects the check box corresponding to the desired interview information; or the operator copies and pastes the desired interview information from a list of interview information.

Note that selection of medical interview information is not necessarily required; for example, the medical interview information input into the trained judgment model may be acquired in advance, and the medical interview information may be automatically output to the medical interview information selection area 11.

The finding information selection area 12 accepts input from the operator via the input interface 313, and outputs the finding information (determination item) selected as the target for generating an impact score from the finding information entered as input information into the determined learning model. This selection may be made by any of the following methods: the operator inputs the desired finding information as text via the input interface 313; check boxes or the like are prepared in association with each finding information and the operator selects the check box corresponding to the desired finding information; or the operator copies and pastes the desired finding information from a list of finding information.

Note that selection of finding information is not necessarily required; for example, the finding information input to the trained judgment model may be acquired in advance, and the finding information may be automatically output to the finding information selection area 12.

Subject image selection area 13 accepts input from the operator via input interface 313, and outputs a subject image selected as a target for generating an influence score from among the subject images input as input information to the determined learning model. This selection can be made in any of the following ways: via input interface 313, the operator prepares check boxes or the like in association with each subject image and selects the check box corresponding to the desired subject image; or, alternatively, the operator copies and pastes the desired subject image from a list of subject images; or, alternatively, the operator copies and pastes the desired subject image from a folder in which subject images are stored.

Note that selection of a subject image is not necessarily required; for example, the subject image input to the trained determination model may be acquired in advance, and the subject image may be automatically output to the subject image selection area 13.

The impact request screen 10 also includes an execute button 14. The execute button 14 serves as a trigger for sending a request to generate an impact to the server device 100. In other words, by accepting operation input from the operator to the execute button 14 via the input interface 313, a request to generate an impact is sent to the server device 100 along with the subject ID information and the information selected in the interview information selection area 11, the findings information selection area 12, and the subject image selection area 13.

Returning to FIG. 4, when the processor 111 of the server device 100 receives the impact generation request, it reads out from the subject management table the subject ID information received together with the request, and information identifying the assessment items and subject image selected in S31 and S32. The processor 111 of the server device 100 then performs a contribution analysis process based on the subject image, medical interview information, and findings information read out as assessment items (S34). The server device 100 also performs an analysis process of the region of interest based on the read out subject image (S35), and performs a segmentation process based on the subject image (S36). Details of each process will be described later.

When the processes of S34 to S36 are completed, the processor 111 of the server device 100 generates impact information (S37) and transmits the generated impact information (T32) via the communication interface 113 to the terminal device 300 that sent the impact generation request. The processor 311 of the terminal device 300 displays the impact information received via the communication interface 315 on the display via the output interface 314.

Here, Figures 9A to 11 are diagrams showing example screens output on a terminal device 300 according to an embodiment of the present disclosure. Specifically, Figures 9A to 11 are diagrams showing example impact information screens 30a to 30e, impact information screen 60, and impact information screen 70 output on a terminal device 300.

In other words, Figure 9A is an example of an impact information screen, and on this impact information screen 30a, the subject's name, age, sex, and consultation date are output at the top of the screen as the subject's attribute information. Also, below that, the judgment result indicating the possibility of the disease for the disease generated in S23 is displayed. Note that although influenza is shown as an example of a disease here, it may of course be a disease other than influenza, or multiple diseases. Also, although the positivity rate is shown here, it may also be that the possibility of infection is "yes" or "no" based on a judgment threshold.

The impact information screen 30a also includes a contribution display area 31 and a focus area display area 32. The contribution display area 31 includes a bar graph corresponding to the contribution (impact) made to the judgment result for each of the subject image, medical interview information, and finding information selected as judgment items in S31. Specifically, the contribution made by each of the input subject image, medical interview information, and finding information to the judgment result obtained by a trained judgment model (for example, the second trained model in Figure 6) using the subject image, medical interview information, and finding information as input is shown.

FIG. 9A shows a bar graph corresponding to the numerical value of the contribution of the pulse rate (a value between K _-2 and K _-3 ) generated by the process of S34, for example. This indicates that the contribution of the pulse rate to the judgment result is relatively low compared to other judgment items (such as body temperature). On the other hand, FIG. 9A shows a bar graph corresponding to the numerical value of the contribution of the body temperature (a value exceeding _K3) generated by the process of S34, for example. This indicates that the contribution of the body temperature to the judgment result is relatively high compared to other judgment items (such as contact with other patients).

Note that the bar graph used to display the degree of influence in the contribution display area 31 as described above is merely one example. It is of course also possible to display the degree of influence numerically, or in other ways.

Furthermore, the attention area display area 32 includes, together with the subject image 40, an attention area image 34 that indicates the area that influenced the judgment result for the selected subject image. Specifically, the attention area in the input subject image is displayed for the judgment result obtained using a trained judgment model (e.g., the first trained model in Figure 6) with the subject image as input.

According to Figure 9A, attention area image 34 is an image in which the numerical value indicating the degree of attention generated by the processing of S35 is distinguished by color scheme or color shades so as to correspond to the position information of the subject image. In the example of Figure 9A, dark colors are displayed near areas 38 and 37 where follicular findings are observed, and gradually lighter colors are displayed as the distance from these areas increases, with no color outside attention area image 34. This suggests that the degree of attention near areas 38 and 37 is particularly high, and that the area near the follicles was focused on when generating the judgment result using the trained judgment model.

Furthermore, the attention area display area 32 shows a frame 39 corresponding to the posterior pharyngeal wall and a frame 33 corresponding to the uvula, which were identified as segments by the processing of S36, and also displays names 35 and 36 corresponding to the identified segments. This allows an operator, etc., who refers to the attention area display area 32, to more intuitively identify the area of interest.

In this way, in Figure 9A, the contribution (influence) of the medical interview information and findings information to the judgment result is displayed in the contribution display area 31, and the attention area of the subject image is displayed in the attention area display area 32, so that they can be compared side by side. Therefore, by relating the contribution (influence) of the medical interview information and findings information to the attention area of the subject image, it is possible to infer in more detail the basis for the judgment result.

Note that the attention level displayed in the attention area display area 32 in the so-called heat map format as described above is merely one example. It is of course also possible to display the attention level numerically, for example, or in other ways.

Furthermore, although only the subject image 40 is displayed in the attention area display area 32, it is also possible to accept an operator's input to an icon (not shown) via the input interface 313, and display similar images in sequence for the other subject images selected in S32. Alternatively, a certain number of subject images and their attention area images may be displayed side by side for comparison.

FIG. 9B is another example of an impact information screen. The impact information screen 30b in FIG. 9B includes a contribution display area 43, a first focus area display area 45, and a second focus image display area 46. Note that, except for the parts specifically mentioned, it is the same as the impact information screen 30a in FIG. 9A, and therefore a description of the points in common with the impact information screen 30a will be omitted.

Here, as will be described later in Figure 6, in this embodiment, multiple subject images are input to each trained judgment model to obtain a judgment result (S215 in Figure 6, etc.). Therefore, for each subject image input to the trained judgment model, it is possible to detect the contribution (influence) made to the judgment result and the area of interest in the subject image.

In FIG. 9B, the first attention area display area 45 and the second attention image display area 46 display information related to the judgment results obtained by inputting subject image A and subject image B, respectively, into a trained judgment model. Specifically, the first attention area display area 45 displays the attention area in the input subject image A for the judgment results obtained by a trained judgment model (e.g., the first trained model in FIG. 6) using subject image A as input. Similarly, the second attention area display area 46 displays the attention area in the input subject image B for the judgment results obtained by a trained judgment model (e.g., the first trained model in FIG. 6) using subject image B as input. In addition to these, the contribution display area 43 displays the contribution of each of the input subject image A, subject image B, medical interview information, and finding information to the judgment results obtained by a trained judgment model (e.g., the second trained model in FIG. 6) using subject image A, subject image B, medical interview information, and finding information as input.

In this way, by outputting the contribution (influence) that each subject image input into the trained judgment model made to the judgment result and the area of interest in the subject image, it is possible to estimate the influence that each subject image had on the judgment result. In other words, it is possible to estimate the subject images that had the greatest impact on the judgment result, and further to understand the basis for which area of those subject images the trained judgment model focused on.

Figure 9C is another example of an impact information screen. The impact information screen 30c in Figure 9C includes a contribution display area 47, a first attention area display area 48a, and a second attention area display area 48b. Note that, except for the parts specifically mentioned, it is the same as the impact information screen 30a in Figure 9A, and therefore a description of the points in common with the impact information screen 30a will be omitted.

As described above, in this embodiment, multiple subject images are input to each trained judgment model to obtain judgment results (e.g., S215 in FIG. 6). Therefore, for each subject image input to the trained judgment model, it is possible to detect the contribution (influence) made to the judgment result and the area of interest in the subject image. As shown in FIG. 9C , the first area of interest display area 48a displays the area of interest in the input subject image C for the judgment result obtained by the trained judgment model (e.g., the first trained model in FIG. 6) using subject image C as input. The second area of interest display area 48b displays the area of interest in the input subject image D for the judgment result obtained by the trained judgment model (e.g., the first trained model in FIG. 6) using subject image D as input. Furthermore, the contribution display area 47 displays the contribution of subject images C and D, medical interview information, and findings information to the judgment result obtained from the trained judgment model (e.g., the second trained model in FIG. 6) as the sum of the judgment results obtained from subject images C and D.

In this way, by showing the area of interest for each subject image, it is possible to understand the degree to which the subject image contributed to the judgment result, and then estimate which subject image contributed more and which areas the trained judgment model paid more attention to in the judgment.

Figure 9D is another example of an impact information screen. The impact information screen 30d in Figure 9D includes a contribution display area 47, a first attention area display area 49a, a second attention area display area 49b, a third attention area display area 49d, and a fourth attention area display area 49d. Note that, except for the parts specifically mentioned, it is the same as the impact information screen 30a in Figure 9A, and therefore a description of the points in common with the impact information screen 30a will be omitted.

As described above, in this embodiment, multiple subject images are input to each trained judgment model to obtain a judgment result (e.g., S215 in Figure 6). Therefore, for each subject image input to the trained judgment model, it is possible to determine the contribution (influence) made to the judgment result and detect areas of interest in the subject image. As shown in Figure 9C, contribution display area 47 displays the contribution made by subject images E to H, medical interview information, and findings information to the judgment result obtained from the trained judgment model (e.g., the second trained model in Figure 6), as a result of adding up the judgment results obtained from subject images E to H.

Furthermore, the first attention area display area 49a displays the attention area of the trained judgment model (for example, the first trained model in FIG. 6 ) for subject image E, the second attention area display area 49b displays the attention area of subject image F, the third attention area display area 49d displays the attention area of subject image G, and the fourth attention area display area 49d displays the attention area of subject image H. Here, it is assumed that the contribution of subject image E to the judgment result is K ₃ , the contribution of subject image F to the judgment result is K ₂ , the contribution of subject image G to the judgment result is K ₁ , and the contribution of subject image H to the judgment result is K ₋₁ . In other words, it is assumed that the contributions to the judgment result are greatest in the order of subject image E, subject image F, subject image G, and subject image H. In such a case, as shown in Figure 9D, it is possible to display the first attention area display area 49a, the second attention area display area 49b, the third attention area display area 49d, and the fourth attention area display area 49d in order of decreasing contribution.

In this way, by showing the area of interest for each subject image, it is possible to understand the degree to which the subject image contributed to the judgment result, and then estimate which subject image contributed more and which areas the trained judgment model paid more attention to in the judgment.

Furthermore, in Figure 9D, the regions of interest for each of the four subject images, subject image E, subject image F, subject image G, and subject image H, are displayed side by side. Follicles can be confirmed in the areas surrounded by dashed lines in each image in Figure 9D. For example, when looking at subject image E, subject image F, and subject image H in Figure 9D, it can be confirmed that in the region of interest seen on the right side of the center, the degree of attention increases in the order of subject image H, subject image F, and subject image E. This trend coincides with the range of follicles confirmed as findings. Therefore, it is possible to infer the correlation between the region of interest that a trained judgment model (e.g., the first trained model in Figure 6) focused on in its judgment and the findings.

It should be noted that the first attention region display region 49a, the second attention region display region 49b, the third attention region display region 49d, and the fourth attention region display region 49d can be modified in various ways as follows.
Although four regions are shown in FIG. 9D, there may be any number of regions, such as 1 to 3 or 5 or more.
Although the subject images displayed in each area are arranged in descending order of contribution, they may also be arranged in any order, such as descending order of contribution, order of photographing, order of high or low attention (for example, the calculated cumulative value of attention), or a combination of these.
A small number of attention area display areas may be set relative to the number of subject images input to the trained model, and the areas may be selected and displayed in order of contribution or attention.

Figure 9E is another example of an impact information screen. The impact information screen 30e in Figure 9E includes a contribution display area 50 and a focus area display area 51. Note that, except for the parts specifically mentioned, it is the same as the impact information screen 30a in Figure 9A, and therefore a description of the points in common with the impact information screen 30a will be omitted.

In Figure 9E, in the attention area display area 51, instead of displaying numerical values indicating the degree of attention using different colors or shades of color corresponding to the position information of the subject image (for example, attention area image 34 in Figure 9A), the degree of attention is shown as a graph for each segment. Specifically, in Figure 9E, for the uvula, posterior pharyngeal wall, and soft palate identified as segments by the process of S36 in Figure 4, the degree of attention area (degree of attention) contained within each segment is added up, and the resulting numerical value is shown as a graph for each segment.

In this way, by showing the cumulative attention level for each segment as a graph, the level of attention level for each segment can be more intuitively grasped, and it also becomes easier to compare attention levels between segments.

Note that while Figure 9E shows the integrated value for each segment, it is also possible to find the area of each segment that shows an attention level above a predetermined threshold, and display this area value as a graph. Also, while Figure 9E does not weight the attention level, it is also possible to weight the attention level of each area included in a segment according to its size, and then integrate the weighted values.

Figure 10 is another example of an impact information screen, and the impact information screen 60 of Figure 10 includes a contribution display area 61 and a contribution display area 62, which contain bar graphs corresponding to the contribution (impact) that each judgment item has to the judgment result. Note that, except for the parts specifically mentioned, it is the same as the impact information screen 30a of Figure 9A, and therefore a description of the points in common with the impact information screen 30a will be omitted.

The contribution display area 61 displays a bar graph corresponding to the current numerical value for each judgment item generated by the processing of S34, and the judgment result (influenza positivity rate) generated by the processing of S23. On the other hand, the contribution display area 62 displays a bar graph corresponding to the past numerical value (for example, three days ago) for each judgment item generated by a process similar to the processing of S34, and the past judgment result (influenza positivity rate) generated by the processing of S23. The numerical values shown in the contribution display area 62 are obtained by the processor 111 of the server device 100 by reading out the past numerical values for a specific date specified by the operator, for example, from the history information in the subject management table.

In this way, by outputting the current and past bar graphs, as well as the current and past influenza positivity rates side by side, it is possible to understand the progression of symptoms and also to estimate the impact that changes in symptoms have had on the influenza positivity rate assessment results.

Note that while Figure 10 shows only two areas, contribution display area 61 and contribution display area 62, there may naturally be three or more areas.

FIG. 11 is yet another example of an impact information screen, and the impact information screen 70 in FIG. 11 includes a focus area display area 71 and a focus area display area 72. Note that, except for the parts specifically mentioned, it is the same as the impact information screen 30a in FIG. 9A, and therefore a description of the points in common with the impact information screen 30a will be omitted.

The attention area display area 71 includes an attention area image 73, which is an image corresponding to a numerical value indicating the current attention level generated by the processing of S35, and a judgment result (influenza positivity rate) generated by the processing of S23, so as to correspond to the position information of the subject image. On the other hand, the attention area display area 72 includes an attention area image 74, which is an image corresponding to a numerical value indicating a past attention level generated by processing similar to S35 for each predetermined pixel of the subject image, and a past judgment result (influenza positivity rate) generated by the processing of S23. The numerical values shown in the attention area display area 72 are obtained by the processor 111 of the server device 100 by reading out past numerical values for a specific date specified by the operator, for example, from the history information in the subject management table.

In this way, by outputting the current and past influenza positivity rates side by side in addition to the current and past attention area images 73 and 74, it is possible to estimate the influence that the attention area had on the influenza positivity rate determination results.

Note that while Figure 11 shows only two areas, attention area display area 71 and attention area display area 72, there may naturally be three or more areas.

Furthermore, each of the impact information screens in Figures 9A to 9E, the impact information screen 60 in Figure 10, and the impact information screen 70 in Figure 11 may be selected according to the operator's wishes by receiving operational input from the operator via the input interface 313 of the terminal device 300.

5. Processing Flow Executed by Server Device 100 (1) Determination Result Generation Processing Fig. 5 is a diagram showing a processing flow executed by the server device 100 according to an embodiment of the present disclosure. Specifically, Fig. 5 is a diagram showing a processing flow executed by the server device 100 in the determination result generation processing of S21 to S24 in the processing sequence of Fig. 4. This processing flow is mainly performed by the processor 111 of the server device 100 reading and executing a program stored in the memory 112.

As shown in FIG. 5, the processor 111 receives a request from the terminal device 300 via the communication interface 113 to determine the possibility of the subject being affected by one or more diseases, along with the subject ID information of the subject selected as the subject to be determined (S111). The medical interview information and findings information entered by the operator on the terminal device 300 or the like are stored in association with the subject ID information in the subject management table. Therefore, based on the received subject ID information, the processor 111 references the medical interview information and findings information in the subject management table and reads out the medical interview information and findings information associated with the subject ID information of the subject (S112 and S113). Based on the same received subject ID information, the processor 111 references the subject image in the subject management table and reads out the subject image associated with the subject ID information of the subject (S114).

Next, the processor 111 uses the read-out medical interview information, findings information, and subject image to perform a process to determine the possibility of contracting one or more diseases (S115).

One example of the above-mentioned judgment process would be to input this information into a trained judgment model such as the one shown below and make a judgment. However, it goes without saying that this processing method is not the only one available, and any processing method can be used, such as a processing method that uses image analysis processing to make a judgment based on the degree of match with an image showing the diseased state.

FIG. 6 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. Specifically, FIG. 6 is a diagram showing a detailed processing flow of the determination processing executed in S115 of FIG. 5. This processing flow is mainly performed by the processor 111 of the server device 100 reading and executing a program stored in the memory 112.

As shown in Figure 6, the processor obtains the first positive rate, second positive rate, and third positive rate, which indicate the possibility of contracting one or more diseases, using different processes, and then performs ensemble processing on them to obtain a determination result.

<Subject image screening process>
First, the screening process for subject images will be described. The processor 111 acquires a plurality of subject images associated with the subject ID information of the subject. The processor 111 then executes a process of screening the acquired plurality of subject images for use in obtaining the determination result (S211). The screening process is performed by labeling the training images in advance based on whether or not they can be used effectively for the determination, and using a trained image selection model obtained by training based on the training images and the label information. Specifically, the processor inputs the received plurality of subject images into the trained image selection model and acquires the plurality of subject images for use in obtaining the determination result as output information.

<Preprocessing of subject images>
Next, preprocessing performed on the subject image will be described. The processor 111 acquires the subject image screened in S211 and performs predetermined preprocessing on the acquired subject image (S212). The processor 111 stores the preprocessed subject image in a subject management table. Examples of such preprocessing include filtering using bandpass filters (including high-pass and low-pass filters), averaging filters, Gaussian filters, Gabor filters, Canny filters, Sobel filters, Laplacian filters, median filters, and bilateral filters; vascular extraction using Hessian matrices; segmentation of specific regions (e.g., follicles) using machine learning; trimming of the segmented region; dehazing; super-resolution processing; and combinations of these, including high-definition processing, region extraction, noise removal, edge enhancement, image correction, and image conversion. These preprocessing methods can be appropriately selected depending on the purpose. By performing preprocessing in this way, regions of interest important for diagnosing one or more diseases can be extracted or enhanced in advance, thereby improving the accuracy of diagnosis.

<Process for obtaining determination results of first positive rate>
Next, the process of acquiring the determination result of the first positive rate will be described. The processor 111 accesses the subject management table and acquires preprocessed subject images. The processor 111 then provides the acquired preprocessed multiple subject images as input to a feature extractor (S213), acquires image features of the multiple subject images as output, and stores the output in the subject management table (S114). The processor 111 then provides the multiple subject images or the features of each subject image as input to a first trained determination model (S215), and acquires a first positive rate indicating a first possibility of contracting one or more diseases as output (S216). The processor 111 stores the acquired first positive rate in the subject management table.

The first trained judgment model is generated by providing a learner with a pair of a subject image or a feature of the subject image and label information labeled with the presence or absence of one or more diseases determined in the subject who was the subject of the subject image, and performing machine learning. Such trained decision models can also be generated using machine learning techniques such as neural networks, convolutional neural networks, multi-layer Herceptron (MLP), long short term memory (LSTM), gated recurrent unit (GRU), graph neural network (GNN), and transformer; gradient boosting decision tree (GBDT) techniques such as Light Gradient Boosting Machine (LightGBM), XGBoost, and CatBoost; ridge regression, logistic regression, support vector regression (SVR), nearest neighbor algorithms, decision trees, regression trees, and random forests. Among these, it is preferable to use a convolutional neural network as the trained decision model, from the perspective of processing subject images and identifying regions of interest.

<Process for obtaining the determination result of the second positive rate>
Next, the process for acquiring the second positive rate determination result will be described. The processor 111 accesses the subject management table and acquires at least one of the medical interview information and the findings information associated with the subject ID information of the subject to be determined (S217). The processor 111 also accesses the subject management table and acquires the feature values of the subject image calculated in S114 and stored in memory in association with the subject ID information (S214). The processor 111 then provides at least one of the acquired medical interview information and the findings information, the subject image, the features of the subject image, or the first positive rate obtained as a determination result by the first trained determination model as input to the second trained determination model (S218), and acquires as output a second positive rate indicating a second possibility of contracting one or more diseases (S219). The processor 111 stores the acquired second positive rate in the subject management table.

The second trained judgment model is generated by providing a learner with a set of subject images or at least one of the subject image's features, medical interview information, and findings information, and label information labeled with the presence or absence of one or more diseases determined in the subject who was the subject of the subject image, and performing machine learning. Such trained decision models can also be generated using machine learning techniques such as neural networks, convolutional neural networks, multi-layer Herceptrons (MLP), long short term memory (LSTM), gated recurrent units (GRU), graph neural networks (GNN), and transformers; gradient boosting decision trees (GBDTs) such as LightGradientBoostingMachine (LightGBM), XGBoost, and CatBoost; ridge regression, logistic regression, support vector regression (SVR), nearest neighbor methods, decision trees, regression trees, and random forests.

<Process for obtaining the determination result of the third positive rate>
Next, the process of acquiring the determination result of the third positive rate will be described. The processor 111 accesses the subject management table and acquires at least one of the medical interview information and the finding information associated with the subject ID information of the subject to be determined (S217). The processor 111 then provides at least one of the read medical interview information and the finding information as input to the third trained determination model (S220), and acquires as output a third positive rate indicating a third possibility of contracting one or more diseases (S221). The processor 111 stores the acquired third positive rate in the subject management table.

The third trained judgment model is generated by providing a learning device with a pair of at least one of the medical interview information and the findings information, and label information labeled with the presence or absence of one or more diseases determined in the subject for whom the medical interview information was input, and performing machine learning. Such trained decision models can also be generated using machine learning techniques such as neural networks, convolutional neural networks, multi-layer Herceptrons (MLP), long short term memory (LSTM), gated recurrent units (GRU), graph neural networks (GNN), and transformers; gradient boosting decision trees (GBDTs) such as LightGradientBoostingMachine (LightGBM), XGBoost, and CatBoost; ridge regression, logistic regression, support vector regression (SVR), nearest neighbor methods, decision trees, regression trees, and random forests.

<Post-processing (ensemble)>
Next, post-processing (ensemble) will be described. As described above, once the first positive rate, second positive rate, and third positive rate have been calculated, the processor 111 accesses the subject management table and acquires each positive rate. The processor 111 then performs ensemble processing using each acquired positive rate (S222). As an example of this ensemble processing, the acquired first positive rate, second positive rate, and third positive rate are input to a ridge regression model, and a result in which each positive rate is ensembled is acquired as a determination result of the possibility of contracting one or more diseases.

The ridge regression model used in S222 is generated through machine learning by processor 111 of server device 100 or processor 111 of another processing device. Specifically, processor 111 obtains the first positive rate, second positive rate, and third positive rate from the training images. Processor 111 also assigns correct labels to the subjects who are the subjects of the training images based on the diagnosis results of doctors in advance. Processor 111 then provides the ridge regression model with a set of each positive rate and its associated correct label, and repeats learning while adjusting the parameters provided for each positive rate so that the output is the same as the correct label information of the ridge regression model. This results in a ridge regression model to be used for ensemble processing.

Furthermore, while the use of a ridge regression model has been described as an example of ensemble processing, any method may be used, such as processing to obtain the average value of each positive rate, processing to obtain the maximum value, processing to obtain the minimum value, weighted addition processing, or processing using other machine learning methods such as bagging, boosting, stacking, lasso regression, linear regression, etc.

In this embodiment, as shown in S22 of FIG. 4, the judgment threshold is selected by input from the operator. Then, at T21 of FIG. 4, information on the judgment threshold is sent to the server device 100 along with the judgment supply. Therefore, the processor 111 reads the received judgment threshold (S223) and compares it with the post-ensemble positive rate obtained in S222. If the positive rate exceeds the threshold as a result of the comparison with the judgment threshold, the processor 111 generates a judgment result of "positive" for one or more diseases, and if the positive rate is equal to or less than the threshold, generates a judgment result of "negative" for one or more diseases (S224).

The processor 111 stores the determination result thus obtained in the subject management table in association with the user ID information (S224). This ends the processing flow.

Returning to FIG. 5, the judgment process of S115 is, for example, performed according to the processing flow shown in FIG. 6. That is, the processor 111 reads the medical interview information, findings information, and subject image acquired in S112 to S114 from the memory 112, and provides them as input information to the first trained judgment model, the second trained judgment model, and the third trained judgment model, respectively. The processor 111 then acquires from each trained judgment model a judgment result indicating the possibility of suffering from one or more diseases based on the input information.

The processor 111 stores the assessment result information, which indicates the likelihood of contracting each disease obtained through the assessment process, in the subject management table in association with the subject ID information. The processor 111 then outputs the assessment result information to the terminal device 300 that sent the assessment request via the communication interface 115 (S116). This completes the processing flow for generating assessment results.

(2) Impact Information Generation Process Fig. 7 is a diagram showing a processing flow executed by the server device 100 according to an embodiment of the present disclosure. Specifically, Fig. 7 is a diagram showing a processing flow executed by the server device 100 in the impact information generation processing of S31 to S38 in the processing sequence of Fig. 4. This processing flow is mainly performed by the processor 111 of the server device 100 reading and executing a program stored in the memory 112.

As shown in FIG. 7, the processor 111 receives an impact generation request from the terminal device 300 via the communication interface 113 (S311). The processor 111 then reads out the medical interview information, findings information, and subject image from the subject management table based on the subject ID information received together with the impact generation request and the information identifying the assessment items and subject image selected in S31 and S32 of FIG. 4 (S312).

The processor 111 executes a contribution analysis process to obtain contribution information, which is one type of influence information, based on each piece of read information (S313). Here, contribution information is information about the directionality and magnitude of the influence that local input information has on the predicted value. Examples of such contribution information include the Shapley value, which indicates the degree of deviation from the expected value of the prediction of the trained judgment model when certain judgment item information is input, or an approximation of the Shapley value, or the output value of LIME (Local Interpretable Model-agnostic Explanations).

In other words, as an example, the contribution information is generated by processor 111 executing the following process. For ease of explanation, the case of calculating the contribution of "pulse rate" will be described here. First, processor 111 uses the selected judgment items to generate all possible combinations of judgment items (for example, pulse rate, body temperature, loss of appetite, pulse rate and body temperature, pulse rate and loss of appetite, and pulse rate, body temperature and loss of appetite, etc.). Then, processor 111 provides each combination as input information to the trained judgment model shown in FIG. 6 (for example, the second trained judgment model), and obtains the expected value of the trained judgment model (a judgment result (positive rate) indicating the possibility of contracting one or more diseases).

Next, processor 111 provides each combination that does not include "pulse rate" to the trained judgment model as input information, and obtains a first output value (a judgment result (positive rate) indicating the possibility of contracting one or more diseases). Processor 111 then calculates the difference between the first output value and the expected value. Processor 111 also provides each combination that includes "pulse rate" to the trained judgment model as input information, and obtains a second output value (a judgment result (positive rate) indicating the possibility of contracting one or more diseases). Processor 111 then calculates the difference between the second output value and the expected value.

Processor 111 calculates the marginal contribution of "pulse rate" to the expected value of the trained judgment model by subtracting the difference calculated from the first output value from the difference calculated from the second output value. Processor 111 obtains the average value of all the calculated marginal contributions as contribution information. Processor 111 then performs the same process for other judgment items besides "pulse rate" and obtains each average value as contribution information.

Note that, when the judgment results of multiple trained judgment models (first trained judgment model, second trained judgment model, and third trained judgment model) are ensembled, as in the judgment process described in Figure 6, the contribution information can be further calculated as follows. For example, when trained judgment models with the same structure are used as each trained judgment model, the following method can be adopted. Processor 111 calculates the contribution (e.g., Shapley value) for each judgment item for each trained judgment model for each judgment result obtained from multiple trained judgment models (e.g., multiple second trained judgment models obtained by cross-validation) obtained by training judgment models with the same structure using training information from different subject groups. Processor 111 then adds up the calculated contributions and sets the resulting value as contribution information for each judgment item. Note that this method is not limited to this, and other methods are also possible for obtaining contribution information.

Furthermore, for example, when trained judgment models with mutually different structures are used as the respective trained judgment models, the following method can be adopted. Processor 111 calculates the contribution (e.g., Shapley value) for each judgment item obtained from multiple trained judgment models with mutually different structures (e.g., in the second trained judgment model, a trained judgment model using LightGBM and a trained judgment model using CatBoost). Then, processor 111 weights the contribution for each trained judgment model based on the weighting coefficient used when ensembling the judgment results of each trained model, and calculates a weighted average. Processor 111 regards the weighted average contribution for each judgment item as contribution information. Naturally, contribution information may be obtained by other methods as well, not limited to this method.

Next, the processor 111 performs an analysis process of the area of interest based on the retrieved subject image to obtain area of interest information, which is one type of influence information (S314). Here, the area of interest information indicates the area in the subject image that had a significant influence on the judgment result when the subject image is input into a trained judgment model (e.g., the first trained judgment model). For example, such area of interest information is calculated using Grad-CAM (Gradient-weighted Class Activation Mapping), SHAP Deep Explainer, LIME (Local Interpretable Model-agnostic Explanations), or a combination of these.

Note that when the judgment results of multiple trained judgment models (first trained judgment model and second trained judgment model) are ensembled, as in the judgment process described in Figure 6, the attention area information can be further calculated as follows. For example, when multiple trained judgment models with the same structure obtained by training using learning information from different subject groups (e.g., multiple first trained judgment models obtained by cross-validation) are used as each trained judgment model, the following method can be adopted. Processor 111 generates an attention level for each trained model (e.g., an attention level image output by the Grad-CAM example above) for the subject image input to each trained model. Processor 111 then averages the attention level images generated for each trained model and uses the averaged attention level image as attention level information. Note that attention level information may, of course, be obtained by other methods as well, not just this method.

Furthermore, for example, when using learned judgment models with different structures as each learned model (specifically, when the sizes of the input judgment images are different, or when the sizes of the output attention images are different due to different structures of the learned judgment models), the following method can be adopted.
(A) For each trained model, the processor 111 generates an image indicating the attention level of the subject image input to the trained model (for example, an attention level image output by the Grad-CAM exemplified above). Since the attention levels obtained here are different sizes, the processor 111 enlarges them to the same size. Then, the processor 111 weights each of the attention levels images enlarged or reduced to the same size based on the weighting coefficients used when ensembling the judgment results of each trained model, and calculates a weighted average. The processor 111 uses the weighted averaged attention level image as attention level information.

Next, processor 111 executes a segmentation process to identify the body parts contained in the subject image based on the read-out subject image (S315). This segmentation process uses a trained segmentation image model obtained by machine learning, which is provided to a learner with a set of training subject images and the position information of the labels obtained by assigning labels to each body part (e.g., position information of the posterior pharyngeal wall, uvula, etc.) based on operational input by a doctor or other professional for the training subject image. Processor 111 inputs the subject image into the trained segmentation model, and obtains the position information of each segmented body part (e.g., the posterior pharyngeal wall, uvula, etc.).

Next, the processor 111 stores the contribution information acquired in S313 in the contribution management table, and transmits it as impact information to the terminal device 300 that sent the impact generation request via the communication interface 113 (S316). Also, the processor 111 stores the attention area information acquired in S314 in the attention area management table, and transmits it as impact information to the terminal device 300 that sent the impact generation request via the communication interface 113 (S316). At this time, the attention area information can also include segmentation information obtained by the segmentation process of S315.

Here, as in S316 and S317, the contribution information and the area of interest information may be transmitted at different times. Specifically, the area of interest analysis process of S314, which involves analyzing the subject image, imposes a relatively greater processing load than the contribution analysis process of S313. As a result, it may take some time to acquire the area of interest information. Therefore, the processor 111 transmits the contribution information to the terminal device 300 in advance as in S316 when the contribution information is acquired, and then transmits the area of interest information to the terminal device 300 as in S317 when the area of interest information is acquired. By processing in this manner, at least a portion of the influence information can be output before the terminal device 300 receives the area of interest information, allowing the operator, etc., to check the analysis results more quickly. This concludes the processing flow for generating influence information.

As shown in this processing flow, by generating influence information that indicates the degree of influence that at least some of the subject image, medical interview information, and findings information input into the trained judgment model have on the judgment results for one or more diseases, it is possible to increase the reliability of the judgment results. In particular, by using contribution information as influence information, it is possible to grasp the contribution of each item selected as a judgment item, and to accurately understand the basis for the judgment made by the trained judgment model. Furthermore, by using attention area information as influence information, it is possible to grasp the area that the trained judgment model focused on when obtaining the judgment result, which, in addition to the results of their own visual examination, allows medical professionals, for example, to feel more reassured.

As described above, in this embodiment, by generating influence information that indicates the degree of influence that at least some of the subject image, medical interview information, and findings information input into the trained judgment model has on the judgment results for one or more diseases, it is possible to increase the reliability of the judgment results.

6. Modifications As described above, one embodiment according to the present disclosure has been described with reference to FIGS. 1 to 11. However, various modifications can be applied without being limited to those described above. Note that, although each modification will be described below, it is also possible to use each modification in appropriate combination. Furthermore, although details of the modification will be described below, the remaining parts can be implemented in the same manner as the embodiment described with reference to FIGS. 1 to 11.

(1) Influence Information Screen In S316 and S317 of Fig. 7, the contribution information and the attention area information are transmitted, respectively, and based on the information, the influence information screens shown in Figs. 9A to 11 are output by the terminal device 300. However, in addition to this, various other information may be transmitted and output on the influence information screen.

For example, by using contribution information, it is possible to identify information on judgment items that had a significant impact on the judgment result. Therefore, processor 111 may generate recommended information for prescription drugs or countermeasures based on the contribution information, and output the recommended information on the impact information screen. Such recommended information is generated by processor 111 extracting judgment items from the generated contribution information according to predetermined criteria (for example, the top three or judgment items with a contribution above a predetermined threshold), and generating a prompt from the judgment items and a pre-prepared template. Processor 111 then provides the generated prompt to a large-scale language model, and obtains the recommended information as output. Note that instead of this method using a large-scale language model, a recommendation management table indicating the correspondence between judgment items extracted according to predetermined criteria and the recommended information generated accordingly may be stored, and the recommendation information may be generated using the recommendation management table.

Furthermore, processor 111 may generate explanatory text and diagrams explaining the basis for the judgment result based on the contribution information and attention area information, and may transcribe the generated explanatory text and diagrams into the subject's electronic medical record information stored in an electronic medical record device, for example. For such explanatory text and diagrams, processor 111 generates a prompt from the generated contribution information and attention area information and pre-prepared standard phrases. Processor 111 then provides the generated prompt to a large-scale language model and obtains the explanatory text and diagrams as output. Furthermore, processor 111 may generate an explanatory text of the treatment plan based on prescription information entered by the operator in addition to the contribution information and attention area information, and may transcribe the explanation into the subject's electronic medical record information stored in an electronic medical record device, for example. For such explanatory text, processor 111 generates a prompt from the generated contribution information, attention area information, and prescription information and pre-prepared standard phrases. Processor 111 then provides the generated prompt to a large-scale language model and obtains the explanatory text as output. The prescription information may be input by an operator as described above, obtained from an electronic medical record device, or generated by a large-scale language model based on the judgment results of a trained judgment model or the diagnosis results of a medical professional.

(2) Process for Generating Determination Results and Process for Generating Impact Information In the example of FIG. 4 , the process for generating determination results indicating the possibility of affliction with one or more diseases and the process for generating impact information were all executed by the same server device 100. Also, the case was described in which the determination request, which triggers the process for generating determination results, and the impact generation request, which triggers the process for generating impact information, are both transmitted from the terminal device 300. However, it is also conceivable that the process for generating determination results is transmitted from the terminal device of a medical professional, and the impact generation request is transmitted from the terminal device of a provider of the determination service. Therefore, the terminal device 300 in the process for generating determination results and the terminal device 300 in the process for generating impact information may be separate terminal devices. Similarly, the server device 100 in the process for generating determination results and the server device 100 in the process for generating impact information may be separate terminal devices.

Furthermore, for each process performed by the server device 100 shown in Figures 5 to 7, it is also possible to distribute the process to multiple server devices as appropriate depending on the processing load. For example, in Figure 6, each process, such as the screening process, preprocessing, and judgment process using each trained judgment model, may be performed on a different server device.

(3) Processing Device In the above embodiment, the server device 100 mainly functions as a processing device and executes the process of generating the judgment result and the process of generating the impact information. However, this is not limited to this, and the terminal device 300 may execute the process of generating the judgment result and the process of generating the impact information. Furthermore, the server device 100 may execute the process of generating the judgment result and the terminal device 300 may execute the process of generating the impact information, or the terminal device 300 may execute the process of generating the judgment result and the server device 100 may execute the process of generating the impact information.

(6) Subject Image In the above embodiment, the subject image was an image captured by the imaging device 200 that can be inserted into the oral cavity and capture images of the oral cavity as a subject. However, the subject image is not limited to this, and subject images captured by a CT device, MRI device, X-ray device, ultrasound diagnostic device, endoscopic device, angiography device, or a combination thereof may also be used. Furthermore, the subject image is merely one type of medical information used in the subject image determination process, and medical information measured by other devices such as a blood pressure monitor device or an electrocardiogram device may be used instead of or in combination with the subject image.

The processes and procedures described in this specification can be realized not only by those explicitly described in the embodiments, but also by software, hardware, or a combination of these. Specifically, the processes and procedures described in this specification are realized by implementing logic corresponding to the processes in media such as integrated circuits, volatile memory, non-volatile memory, magnetic disks, and optical storage. Furthermore, the processes and procedures described in this specification can be implemented as computer programs and executed by various computers, including processing devices and server devices.

Even if the processes and procedures described herein are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple components, and/or multiple modules. Furthermore, even if the various pieces of information described herein are described as being stored in a single memory or storage unit, such information may be stored in multiple memories provided in a single device or in multiple memories distributed across multiple devices. Furthermore, the software and hardware elements described herein may be realized by integrating them into fewer components or by breaking them down into more components.

1 Processing system 100 Server device 200 Imaging device 300 Terminal device

Claims (11)

A processing device comprising at least one processor,
the at least one processor:
Acquire, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject of the subject, interview information of the subject, and finding information of the subject;
generating impact information indicating the degree of influence of at least one of the subject image, the medical interview information, and the finding information input to the trained judgment model on a judgment result indicating the possibility of the subject having one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, the medical interview information, and the finding information into the trained judgment model;
a processing unit configured to perform processing for: the determination result is generated by inputting at least two or more pieces of information, the subject image, the medical interview information, and the finding information, into the trained determination model;
the impact information is generated for each of at least two pieces of input information,
The generated impact information is output in a manner that allows comparison with each other.
The processing device of claim 1 . The processing device of claim 1, wherein the influence information indicates the degree to which the input of at least one of the subject image, the medical interview information, and the finding information deviates from the average predicted value of the trained judgment model. The processing device of claim 1, wherein the impact information is a Shapley value or an approximation of the Shapley value. The processing device according to claim 1, wherein the influence information indicates an area in the subject image that had a large influence on the judgment result when the judgment result was generated by inputting the subject image into the trained judgment model. The processing device according to claim 4, wherein the influence information indicates the degree of influence on the determination result in correspondence with the position information of the subject image. The processing device according to claim 1, wherein the generated impact information is output to a terminal device connected via a communication interface. the influence information includes both deviation degree information indicating the degree of deviation from an average predicted value of the trained judgment model due to input of at least one of the subject image, the interview information, and the finding information, and area information indicating an area in the subject image that has a large influence on the judgment result when the judgment result is generated by inputting the subject image to the trained judgment model;
the deviation degree information is output to the terminal device before the area information.
The processing device of claim 7 . When executed by at least one processor,
Acquire, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject of the subject, interview information of the subject, and finding information of the subject;
generating impact information indicating the degree of influence of at least one of the subject image, the medical interview information, and the finding information input to the trained judgment model on a judgment result indicating the possibility of the subject having one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, the medical interview information, and the finding information into the trained judgment model;
A processing program that causes the at least one processor to function in such a manner. A processing method executed by at least one processor, comprising:
acquiring, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject of a subject, interview information of the subject, and finding information of the subject;
generating impact information indicating the degree of influence of at least one of the subject image, the medical interview information, and the finding information input to the trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, the medical interview information, and the finding information into the trained judgment model;
A processing method comprising: A processing system including a terminal device and a processing device communicatively connected to the terminal device,
At least one processor included in the terminal device selects at least one of a subject image acquired by an imaging device configured to image a subject of the subject, interview information of the subject, and finding information of the subject;
At least one processor included in the processing device
acquiring at least one of the selected subject image, the interview information, and the finding information via a communication interface;
generating impact information indicating the degree of influence of at least one of the subject image, the medical interview information, and the finding information input to the trained determination model on a determination result indicating the possibility of the subject having one or more diseases, the determination result being generated by inputting at least one of the acquired subject image, the medical interview information, and the finding information into the trained determination model;
a processing system configured to perform processing for