WO2025225015A1 - Image processing device, image display method, image display program, recording medium, and image diagnosis system - Google Patents

Abstract

An image processing device 1 comprises: an identification result reception unit 102 that receives a lesion identification result a from an image; a text generation unit 102a that generates, as text, basis information of the identification; a classification unit 103 that classifies constituent elements of the text into an object to be emphasized and an object not to be emphasized; an emphasis unit 104 that emphasizes the object to be emphasized; and an output unit 105 that outputs at least the object to be emphasized in the text to a monitor.

Description

Image processing device, image display method, image display program, recording medium, and image diagnosis system

In recent years, advances in IT technology have led to the practical application of image diagnosis using AI (Artificial Intelligence) in a variety of fields. In image diagnosis, a technology is known that presents supporting information for inferences made by AI (see, for example, Patent Document 1).

Patent No. 7161979

In the medical field, it is possible that AI for CAD (computer-aided diagnosis), used in endoscopic diagnosis and other procedures, will be required to provide all supporting information to the doctor who uses it. However, the time available for diagnosis is limited.

The present invention therefore aims to provide an image processing device, image display method, image display program, storage medium, and image diagnostic system that can generate display images that allow medical professionals and others to quickly identify the underlying information when referring to image diagnostic results of medical images generated by CAD or other methods.

An image processing device according to one aspect of the present invention includes a classification result receiving unit that receives lesion classification results from an image, and a sentence generation unit that generates sentences based on information on the basis of the classification. The image processing device also includes a classification unit that classifies components of the sentences into emphasis targets and non-emphasis targets. The image processing device also includes an emphasis unit that emphasizes the emphasis targets. The image processing device also includes an output unit that outputs at least the emphasis targets from the sentences to a monitor.

An image processing device according to one aspect of the present invention includes one or more processors. The processors receive the image classification results, generate the basis information for the classification as text, classify the components of the text into those to be emphasized and those not to be emphasized, emphasize the emphasized components, and output at least the emphasized components of the text and the classification results to a monitor.

An image display method according to one aspect of the present invention receives the results of image classification, generates the basis information for the classification as text, classifies the components of the text into highlight and non-highlight items, highlights the highlight items, and outputs at least the highlight items of the text to a monitor.

An image display program according to one aspect of the present invention receives the results of image classification, generates text as the basis for the classification, classifies the components of the text into emphasis targets and non-emphasis targets, emphasizes the emphasis targets, generates an image associated with the emphasis targets, and outputs at least the emphasis targets of the text to a monitor.

A storage medium according to one aspect of the present invention stores an image display program that receives the results of image classification, generates text based on information on the basis of the classification, classifies the components of the text into emphasis targets and non-emphasis targets, emphasizes the emphasis targets, generates images associated with the emphasis targets, and outputs at least the emphasis targets of the text to a monitor.

An image diagnostic system according to one aspect of the present invention includes a classification unit that receives evidence information for the differentiation of lesions and classifies the components of the text that make up the evidence information into emphasis targets and non-emphasis targets. The image diagnostic system also includes an emphasis unit that emphasizes the emphasis targets. Furthermore, the image diagnostic system includes an output unit that outputs at least the emphasis targets from the text to a monitor.

The present invention has the advantage of being able to generate a display image that allows medical professionals and others to quickly identify the underlying information when referring to the image diagnosis results of medical images obtained using CAD or other methods.

1 is a block diagram illustrating an example of the configuration of a medical system including an image processing apparatus according to a first embodiment of the present invention. FIG. 2 is a schematic block diagram illustrating a detailed configuration of the endoscope device. FIG. 1 is a block diagram for explaining the functional configuration of an image processing apparatus according to a first embodiment. 10 is a flowchart illustrating an example of a discrimination algorithm. FIG. 2 is a diagram illustrating an example of a display image according to the first embodiment. FIG. 2 is a diagram illustrating an example of a display image according to the first embodiment. FIG. 2 is a diagram illustrating an example of a display image according to the first embodiment. FIG. 2 is a diagram illustrating an example of a display image according to the first embodiment. FIG. 10 is a diagram illustrating an example of a display image according to the second embodiment. FIG. 10 is a diagram illustrating an example of a display image according to the second embodiment. FIG. 10 is a diagram illustrating an example of a display image according to the second embodiment. FIG. 11 is a diagram illustrating an example of a display image according to the third embodiment. FIG. 13 is a diagram illustrating an example of a display image according to the fourth embodiment. FIG. 13 is a view for explaining an example of a display image according to the fifth embodiment. FIG. 13 is a view for explaining an example of a display image according to the fifth embodiment. FIG. 20 is a view for explaining an example of a display image according to the sixth embodiment. FIG. 23 is a view for explaining an example of a display image according to the seventh embodiment. FIG. 23 is a view for explaining an example of a display image according to the seventh embodiment. FIG. 23 is a view for explaining an example of a display image according to the seventh embodiment. FIG. 23 is a diagram illustrating an example of a display image according to the eighth embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(First embodiment)

FIG. 1 is a block diagram illustrating an example of the configuration of a medical system including an image processing device according to a first embodiment of the present invention. The medical system according to this embodiment is mainly composed of an image processing device 1, an endoscope device 3, a display device 41, and a network 5. As shown in FIG. 1, it may also include a server 2. The medical system shown in FIG. 1 analyzes medical images acquired by the endoscope device 3 and the like, and outputs information to assist doctors in making diagnoses by differentiating lesions and the like.

The image processing device 1 differentiates lesions from medical images and outputs the results of the differentiation. The image processing device 1 includes a processor 11 and a storage device 12.

The processor 11 is connected to the network 5 and includes a central processing unit (hereinafter referred to as the CPU) 111 and hardware circuits 112 such as ROM and RAM. The processor 11 may also include an integrated circuit such as an FPGA instead of or separate from the CPU.

The storage device 12 stores various software programs. Each software program is read and executed by the CPU 111 of the processor 11. Note that all or part of the various programs may be stored in the ROM of the processor 11.

The storage device 12 stores a program (not shown) used to control the operation of the image processing device 1, as well as an image display program 121. The image display program 121 is a program that performs processing related to the generation of display images based on the results of lesion differentiation in medical images. The storage device 12 also stores various setting values and parameters required for executing the image display program 121. The storage device 12 is also configured to be able to store medical images, etc. obtained from the server 2 and the endoscope device 3.

The server 2 is connected to the network 5 and includes a processor 21 and a storage device 22. The processor 21 includes a CPU and other components. The storage device 22 is a large-capacity storage device such as a hard disk drive. The server 2 stores medical images output from endoscope devices 3 and other devices connected to the network 5. The server 2 also has the function of transmitting medical images in response to requests from image processing devices 1 and other devices connected to the network 5.

The endoscope device 3 is a device used to observe the inside of a subject's body. Figure 2 is a schematic block diagram illustrating the detailed configuration of the endoscope device. The endoscope device 3 includes an endoscope 31 and a video processor 32.

The endoscope 31 has an elongated insertion section 311, an operating section 312 provided at the base end of the insertion section 311, and a universal cable 313 extending from the operating section 312.

The image sensor 314 is provided at the tip of the insertion section 311. Light reflected from the subject illuminated by illumination light emitted from an illumination window (not shown) is incident on the imaging surface of the image sensor 314 via an observation window (not shown). The image sensor 314 outputs an imaging signal of the subject image.

The operation unit 312 has various operation members 312a, including an up-down bending knob, a left-right bending knob, and various operation buttons. The user of the endoscope 31 can operate these operation members 312a to bend the bending section (not shown) provided in the insertion section 311, or give instructions to record endoscopic images.

The endoscope 31 is connected to the video processor 32 via a connector 313a provided at the base end of the universal cable 313.

The imaging signal from the imaging element 314 is supplied to the video processor 32 via a signal line 315 that passes through the insertion portion 311, the operation portion 312, and the universal cable 313.

The video processor 32 includes a control unit 321, an image processing unit 322, a communication interface (hereinafter abbreviated as communication I/F) 323, and an operation panel 324. The video processor 32 is connected to the network 5.

The control unit 321 controls the overall operation of the endoscope device 3 and the execution of various functions. The control unit 321 includes a CPU, ROM, RAM, etc., and programs for various operations and controls are recorded in the ROM.

The CPU reads from ROM and executes programs in response to operation signals from the various operation members 312a of the operation unit 312 or the operation panel 324, thereby controlling the overall operation of the endoscope device 3 and executing various functions.

Under the control of the control unit 321, the image processing unit 322 drives the image sensor 314 and receives an image signal from the image sensor 314. Under the control of the control unit 321, the image processing unit 322 generates an endoscopic image based on the image signal and outputs it to the control unit 321.

The communication I/F 323 is a circuit that connects the control unit 321 to the network 5. The control unit 321 outputs endoscopic images to the server 2 via the communication I/F 323.

The operation panel 324 has various buttons and the like that allow the user to specify the execution of various functions. The operation panel 324 is, for example, a display device with a touch panel. The user can operate the operation panel 324 to instruct the endoscope device 3 to execute the desired function.

The control unit 321 generates an image signal for the endoscopic image to be displayed on the display device 41 and outputs it to the display device 41. The endoscopic image based on the image signal is displayed on the screen of the display device 41.

The display device 41 is included in the notification device 4. The notification device 4 may include an audio device 42 in addition to the display device 41. The display device 41 includes a monitor, etc. The video processor 32 may send image data to the display device 41 via the network 5, or may send image data to the display device 41 via the network 5 and the image processing device 1. When the evidence described below is presented as text, depending on the evidence, it may be sensitive information for the subject undergoing the endoscopic examination, such as text suggesting the possibility of cancer. For this reason, the relative angle and height of the display device 41 and the examination table on which the subject is lying may be adjusted so that the evidence information is not seen by the subject. Furthermore, goggles with a monitor may be used as the display device 41 instead of a stationary monitor.

Next, the function of generating a display image from a medical image in the image processing device 1 will be described. Figure 3 is a block diagram for explaining the functional configuration of the image processing device according to the first embodiment. The image processing device 1 is configured to have a basis information generation unit 102, a sentence generation unit 102a, a classification unit 103, an emphasis unit 104, and an output unit 105. It may further include a detection unit 100 or a discrimination unit 101, or at least one of these may be separate from the detection unit 100.

The discrimination unit 101 is equipped with artificial intelligence (AI) that has undergone deep learning for CADx (Computer-Aided Diagnosis). The discrimination results by the discrimination unit 101, such as whether it is a lesion, blood vessel, or nerve, or the type of lesion, infection history with Helicobacter pylori or other bacteria, type of blood vessel, type of organ, or location within the organ, are output to the display device 41 via the output unit 105.

The results of the medical image classification performed by the classification unit 101 are output to the basis information generation unit 102, which serves as a classification result receiving unit. The basis information generation unit 102 estimates the basis for the classification, i.e., why the classification unit 101 output such a classification result. Although not shown, a configuration may be provided that allows the display of the basis information to be turned on and off, so that the results of the classification are output to the basis information generation unit 102 only when the display of the basis information is turned on.

The sentence generation unit 102a converts the basis for discrimination into sentences that can be understood by humans. The classification unit 103 classifies multiple types of sentences into emphasis targets and non-emphasis targets. Specific examples will be described later, but when there are multiple types of sentences generated by the sentence generation unit 102a, each sentence may be classified into emphasis targets and non-emphasis targets, or emphasis targets and non-emphasis targets may be classified within a single sentence. When classifying emphasis targets and non-emphasis targets within a single sentence, this may be done on a phrase-by-phrase basis, on a word-by-word basis, or a combination of these. It is also possible to switch emphasis targets and non-emphasis targets over time.

The highlighting unit 104 generates information for highlighting the highlight target according to the classification result, and outputs it to the display device 41 via the output unit 105 as a highlighting process. Examples of highlighting methods used by the highlighting unit 104 include: "underlining the highlight target and not underlining the non-highlight target", "making the font of the highlight target bolder than that of the non-highlight target", "making the font size of the highlight target larger than that of the non-highlight target", "making the font type of the highlight target different from that of the non-highlight target", "making the color of the highlight target different from that of the non-highlight target", "making the tilt angle of the font of the highlight target different from that of the non-highlight target", or "making the blinking speed of the highlight target faster than that of the non-highlight target".

As an example of a method of highlighting using color, there is a method of selecting the color of the characters to match the color of the background, as shown in Table 1.
[Table 1]

The highlighting unit 104 may be equipped with an image generating unit 104a. The image generating unit 104a generates an highlighted image by adding marker images to locations on the differentiated medical image that correspond to the components to be highlighted. The marker image may be an icon, such as an arrow, that points to the area to be highlighted, or a bounding box that surrounds the area to be highlighted, or may be linked to the evidence information. For example, if the evidence information includes information regarding the outline of the lesion, a frame that traces the outline of the lesion may be displayed. It is preferable that the marker image be an image that is visually consistent with the highlighting process of the component to be highlighted. For example, it is preferable that the display color of the marker image be similar or the same color as the display color of the component to be highlighted. Similar colors refer to colors that fall within a specified range on the color wheel.

The results of detection of the position and contour of the lesion by the detection unit 100 may be input to the image generation unit 104a, and the detection information of the position and contour of the lesion may be used to generate a marker image. The detection unit 100 may be equipped with artificial intelligence (AI) that has undergone deep learning for Computer-Aided Detection (CADe).

Specifically, it is assumed that the sentence generation unit 102a generates multiple types of sentences, "sentences verbalizing the first basis," "sentences verbalizing the second basis," and "sentences verbalizing the third basis," from the evidence information generated by the evidence information generation unit 102. When the classification unit 103 determines that the "sentences verbalizing the first basis" is to be emphasized and the "sentences verbalizing the second basis" and "sentences verbalizing the third basis" are not to be emphasized, the highlighting unit 104 uses the above-mentioned highlighting method or the like to highlight the "sentences verbalizing the first basis" so that, when viewed by a doctor's eyes, they stand out more than the "sentences verbalizing the second basis" and "sentences verbalizing the third basis."

Of course, as mentioned above, it is not necessary to emphasize sentence-by-sentence. For example, if the sentence generation unit 102a generates a "sentence verbalizing the basis" that includes a "first word," a "second word," and a "third word," the classification unit 103 may determine that the "first word" is to be emphasized and that the "second word" and "third word" are not to be emphasized.

The classification unit 103 may add timing information to the emphasis target information and non-emphasis target information and output the information to the emphasis unit 104. Even if there are multiple areas that you want the doctor to pay attention to, by emphasizing them in order at different times, the doctor's burden can be reduced while still being able to input information.

For example, it is possible to output information such as, "At a first timing, the 'sentence verbalizing the first basis' is to be emphasized, and the 'sentence verbalizing the second basis' and the 'sentence verbalizing the third basis' are to be de-emphasized," "At a second timing later than the first timing, the 'sentence verbalizing the second basis' is to be emphasized, and the 'sentence verbalizing the first basis' and the 'sentence verbalizing the third basis' are to be de-emphasized," and "At a third timing later than the second timing, the 'sentence verbalizing the third basis' is to be emphasized, and the 'sentence verbalizing the first basis' and the 'sentence verbalizing the second basis' are to be de-emphasized."

The diagnostic order can be used, for example, to determine the order of highlighting. The diagnostic order can be based on a user-customized and registered order, or a diagnostic algorithm can be used. Figure 4 shows the diagnostic procedure using MESDA-G as an example of a diagnostic algorithm. MESDA-G is a diagnostic algorithm used to differentiate gastric cancer. It first determines color changes (whitish or reddish) or morphological changes (protrusion, flattening, or depression) on the gastric mucosal surface from the image. Next, it identifies the demarcation line (DL) between the lesion and the background mucosa (S1). If there is no DL, it is diagnosed as non-cancerous (benign lesion). If DL is present, the presence of an irregular microvascular pattern (IMVP) and an irregular microsurface pattern (IMSP) is then observed (S2). If irregular microvascular and/or microsurface patterns are present within the borderline, the diagnosis is cancer. If irregular microvascular and/or microsurface patterns are not present within the borderline, the diagnosis is non-cancerous.

As a specific example, let us consider a case where the differentiation unit 101 differentiates gastric cancer from a medical image and obtains a result of cancer with a diagnostic certainty of 90%. From the evidence information generated by the evidence information generation unit 102, the sentence generation unit 102a generates multiple types of sentences, such as the sentence "there is a clear boundary" as the "first evidence," the sentence "it has an irregular texture" as the "second evidence," and the sentence "it is reddish in color" as the "third evidence." These sentences are then merged to generate the evidence information sentence "there is a clear boundary, it has an irregular texture, and it is reddish in color." In this case, in accordance with the diagnostic algorithm, the classification unit first highlights sentences, phrases, or words related to the color of the gastric mucosal surface. Then, the next highlighting target is sentences, phrases, or words related to the boundary line between the lesion and the background mucosa. The next highlighting target is sentences, phrases, or words related to the microvascular pattern. Therefore, the components in the sentence "It has clear boundaries, an irregular texture, and is reddish in color" are emphasized in the order "redness," "clear boundaries," and "irregular texture."

If there are similar changes, such as "changes in the color of the gastric mucosal surface" and "morphological changes," the highlighting timing can be set to be simultaneous, or the order can be determined by user settings.

In addition, diagnostic algorithms may be switched to perform differentiation depending on the type of organ. If the target organ is the large intestine, for example, the JNET classification, NICE classification, Sano classification, Jikei classification, Showa classification, Hiroshima classification, Kudo-Tsuruta classification, and Akita-Nisseki classification are used. If the target organ is the esophagus, for example, the Inoue classification, Japan Esophageal Society classification, Arima classification, and BING classification are used. If the target organ is the stomach, for example, the MESDA-G, Koyama classification, Jikei classification, Yao classification, and Yagi classification are used. In addition to the type of organ, diagnostic algorithms may be switched to perform differentiation depending on the target lesion, the type of endoscope used to capture the medical images, etc.

When determining the order of emphasis in accordance with the diagnostic sequence, for example, a keyword can be set for each component of the diagnostic sequence, and the order of emphasis for each component of the evidence information can be determined based on the degree of match between each component of the evidence information and the keyword.

For example, when using the above-mentioned MESDA-G, the keywords related to the first item in the diagnostic order are "red, bleeding, protuberance, flat, depression," the keyword related to the second item in the diagnostic order is "boundary," and the keyword related to the third item in the diagnostic order is "capillary, capillary, texture, texture" are stored in the classification unit 103, or the classification unit 103 can access this information stored in a separate structure. Then, if the sentence generation unit generates sentences such as "There is a clear boundary" and "The color is reddish," then "The color is reddish," which has a higher degree of match with the keyword related to the first item in the diagnostic order, is emphasized first in chronological order.

The sentence generation unit 102a may generate sentences using medical terms contained in a medical dictionary set by the user or the like. For example, prior to image diagnosis, an arbitrary dictionary may be selected and set from multiple medical dictionaries, and the dictionary may be used to generate sentences for the evidence information. The generated sentences are output to the classification unit 103.

The output unit 105 generates and outputs a display image to be displayed on the display device 41. The display image includes the results of the diagnosis (disease name, diagnostic certainty), text of the evidence information that has been highlighted, and an highlighted diagram. Figures 5A, 5B, 5C, and 5D are diagrams illustrating an example of a display image according to the first embodiment. Figure 5A shows an example of a display image 411 displayed on the display device 41. The display image 411 has two display areas D1 and D2. The highlighted diagram G1 generated by the diagram generation unit 104a is placed in the display area D1, and the results of the diagnosis (disease name, diagnostic certainty) and text of the evidence information generated by the highlighting unit 104 are placed in the display area D2. Figure 5A shows a display image when the "first evidence" is to be highlighted. In this case, the marker image L1 in the highlighted diagram G1 is added to the location corresponding to the "first evidence." In the display image 411, the bolded portions of the text of the evidence information and the marker image L1 indicated by a bold line are displayed in green, for example. The rest of the evidence information text is displayed in black, for example. In this way, by making the display color of the component to be highlighted different from the display colors of the other components in the evidence information text, the user can quickly identify the evidence information to be highlighted. Furthermore, by making the display color of the component to be highlighted the same as the display color of the marker image L1, the user can quickly identify the evidence information to be highlighted and the location of the corresponding lesion.

It is desirable to use different colors for the components to be highlighted and the marker images. Figure 5B is an example of a display image in the example of gastric cancer differentiation described above, where the "first basis" of "there is a clear boundary" is to be highlighted. Figure 5C is an example of a display image in the example of gastric cancer differentiation described above, where the "second basis" of "has an irregular texture" is to be highlighted. Figure 5D is an example of a display image in the example of gastric cancer differentiation described above, where the "third basis" of "the color is reddish" is to be highlighted.

In the display image 411 shown in Figure 5B, the emphasis (bold part) in the text of the evidence information and the marker image L1 indicated by a thick line are displayed in green, for example. In the display image 411 shown in Figure 5C, the emphasis (left italic part) in the text of the evidence information and the marker image L2 indicated by a double line are displayed in blue, for example. In the display image 411 shown in Figure 5D, the emphasis (right italic part) in the text of the evidence information and the marker image L3 indicated by a thick dotted line are displayed in red, for example. In this way, when the components classified as emphasised are different, they may be displayed using different colors.

In this way, when generating a display image based on the results of differentiation of a medical image, the image processing device of the first embodiment differentiates the display color of the components to be emphasized from the display color of the other components in the text of the evidence information to be displayed in the display image. This allows the user to quickly identify the evidence information to be emphasized. Furthermore, the image processing device of the first embodiment adds marker images to the medical image that has been differentiated at locations corresponding to the components to be emphasized, to generate an emphasized image. At this time, the display color of the marker images is made the same as the display color of the components to be emphasized. This allows the user to quickly identify the evidence information to be emphasized and the location of the corresponding lesion.

In generating sentences for evidence information in the sentence generator 102a, in addition to the medical dictionary settings, the description method settings for colors may be switchable. For example, two modes, a normal mode and a detailed mode, may be set, and when the detailed mode is set, a sentence that describes in detail the color of a component that includes color may be generated.
Second Embodiment

In the first embodiment described above, the process of visually highlighting components classified as those to be highlighted is to set the display color of the components to be highlighted in the displayed image to a color different from the display color of components not to be highlighted. In contrast, in this embodiment, the texture of the components to be highlighted in the displayed image is made different from the texture of the components not to be highlighted. In other words, the processing performed by the highlighting unit 104 in this embodiment differs from the first embodiment.

The image processing device in this embodiment and the following embodiments has the same configuration as the image processing device 1 in the first embodiment. The same components are designated by the same reference numerals and their descriptions are omitted.

In this embodiment, the highlighting unit 104 visually highlights the text of the component to be highlighted by underlining it or by making it bolder than the text of the component not to be highlighted. Figures 6A, 6B, and 6C are diagrams illustrating an example of a display image according to the second embodiment. Figure 6A is an example of a display image in the example of gastric cancer differentiation described above, where the "first basis" "there is a clear boundary" is to be highlighted. Figure 6B is an example of a display image in the example of gastric cancer differentiation described above, where the "second basis" "has an irregular texture" is to be highlighted. Figure 6C is an example of a display image in the example of gastric cancer differentiation described above, where the "third basis" "the color is reddish" is to be highlighted.

In Figures 6A to 6C, marker image L4 in the highlighted image G1 is added to the location corresponding to the component to be highlighted. In the display image 411, the component to be highlighted in the text of the evidence information is shown in bold and underlined. The other parts of the text of the evidence information are shown in thin, ununderlined text. In this way, by making the texture of the characters of the component to be highlighted in the text of the evidence information different from the texture of the other components, the user can quickly identify the evidence information to be highlighted.

In the above description, the texture of the component to be emphasized is made different from that of other components depending on the thickness of the characters and whether or not they are underlined, but it is also possible to make the texture different by changing the size, font, or tilt of the characters, or by combining these.
(Third embodiment)

In the first embodiment described above, a display image is generated in which an emphasis image is placed when one specific component is to be emphasized. Therefore, when another component is to be emphasized, it was necessary to generate a display image in which an emphasis image corresponding to that component is placed. In contrast, this embodiment differs from the first embodiment in that an emphasis image is placed in a single display image in which each component is to be emphasized.

In this embodiment, the display image 411 generated by the output unit 105 includes text of the evidence information that has been subjected to emphasis processing, a medical image that has been differentiated, and an emphasis diagram. Figure 7 is a diagram illustrating an example of a display image according to the third embodiment. The display image 411 has three display areas D1, D2, and D3. Highlighted diagrams G1, G2, and G3 are arranged in display area D1, and text of the evidence information is arranged in display area D2. Furthermore, the medical image G0 that has been differentiated is arranged in display area D3.

The text of the evidence information arranged in display area D2 is visually highlighted in different ways for each component. For example, the "first evidence" "has a clear boundary" is displayed in green, the "second evidence" "has an irregular texture" is displayed in blue, and the "third evidence" "is reddish in color" is displayed in red. Marker image L1 in the highlighted image G1 arranged in display area D1 is added to the location corresponding to the "first evidence." The color of marker image L1 is the same as the display color of the "first evidence." Marker image L2 in the highlighted image G2 is added to the location corresponding to the "second evidence." The color of marker image L2 is the same as the display color of the "second evidence." Marker image L3 in the highlighted image G3 is added to the location corresponding to the "third evidence." The color of marker image L3 is the same as the display color of the "third evidence."

In this way, by arranging the highlighted images corresponding to the respective components of the sentence of the evidence information in one display image, it is possible to quickly compare the components.
(Fourth embodiment)

In the first embodiment described above, a display image is generated in which an emphasis image is placed when a specific component is the emphasis target. In contrast, this embodiment differs in that a display image is generated in which a reference image containing a lesion similar to the component is also placed. The reference image is, for example, an atlas image published in a medical encyclopedia or the like, which contains a lesion similar to the component.

In this embodiment, the display image 411 generated by the output unit 105 includes text of the evidence information that has been subjected to emphasis processing, a medical image that has been differentiated, and a reference image. Figure 8 is a diagram illustrating an example of a display image according to the fourth embodiment. The display image 411 has three display areas D1, D2, and D4. An emphasis image G1 is placed in display area D1, and text of the evidence information is placed in display area D2. Furthermore, a reference image G11 is placed in display area D4. A marker image L11 is added to the reference image G11. The marker image L11 is added to a location in the reference image G11 that corresponds to the component to be emphasized. The marker image L11 is an image that is visually consistent with the marker image L1. For example, the display color of the marker image L11 is the same as the display color of the marker image L1 and the display color of the component to be emphasized.

In this way, by arranging the evidence information text, the highlighted image, and the reference image in one displayed image, comparison with other cases can be made quickly.
Fifth Embodiment

This embodiment differs from the above-described embodiments in that the display images are switched sequentially based on the order of diagnoses in the differentiation. For example, when differentiation is performed using the MESDA-G diagnostic procedure shown in Figure 4, a display image (Figure 9A) is first displayed in which the components based on the diagnosis shown in S1 are highlighted. Next, a display image (Figure 9B) is displayed in which the components based on the diagnosis shown in S2 are highlighted. Figures 9A and 9B are diagrams illustrating an example of a display image according to the fifth embodiment.

The differential diagnosis results shown in display area D1 preferably display not only the disease name and diagnostic certainty, but also the judgment result for each diagnosis (for example, "DL+" in Figure 9A or "IMVP/IMSP+" in Figure 9B). In this case, it is preferable to display the judgment result in the same color as the component being highlighted.

The display color of the component to be highlighted and the display color of the marker image may be changed depending on the judgment status, such as red if the judgment result is "+" and green if the judgment result is "-".
Sixth Embodiment

This embodiment differs from the above-described embodiment in that, as a result of the differentiation, evidence information is displayed on the displayed image only if the lesion is determined to be a disease, and evidence information is not displayed if the lesion is determined not to be a disease.

FIG. 10 is a diagram illustrating an example of a display image according to the sixth embodiment. The display image when the lesion is determined to be a disease is the same as that of the above-described embodiments (for example, FIG. 5B). When the lesion is determined not to be a disease, for example, a display screen as shown in FIG. 10 is generated. That is, an emphasized image G1a is placed in display area D1, and the rejected disease name and diagnostic certainty are placed in display area D2. The emphasized image G1a is generated by adding a marker image L5 to the location of the lesion on the medical image.

In this way, if the result of the differential diagnosis is that there is no disease, the amount of information in the displayed image is reduced, allowing the user to quickly perform image diagnosis.
Seventh Embodiment

This embodiment differs from the above-described embodiments in that the display images are sequentially switched based on the importance of the basis for the diagnosis. Figures 11A, 11B, and 11C are diagrams illustrating an example of a display image according to the seventh embodiment. Figures 11A, 11B, and 11C are the display images shown in Figures 5B, 5C, and 5D, respectively, to which the importance of the basis for the diagnosis has been added.

For example, if the DL identification result has the highest importance for disease diagnosis (e.g., importance 100), the IMVP/IMSP observation result has the next highest importance (e.g., importance 70), and the color change of the gastric mucosal surface has the lowest importance (e.g., importance 30), the display image shown in Figure 11A will be displayed first. Next, the display image shown in Figure 11B will be displayed. Finally, the display image shown in Figure 11C will be displayed.

In this way, by displaying an image in which components are highlighted in descending order of importance of the basis for judgment, the user can quickly perform image diagnosis.
Eighth Embodiment

This embodiment differs from the above-described embodiment in that, as a result of the differentiation, evidence information is displayed on the displayed image only when the lesion is determined to be a disease and the diagnostic certainty is high (higher than a set value); it does not display evidence information when the lesion is determined not to be a disease, or when the diagnostic certainty is low even if the lesion is determined to be a disease.

FIG. 12 is a diagram illustrating an example of a display image according to the eighth embodiment. The display image when the lesion is determined to be a disease is the same as that of the above-described embodiments (for example, FIG. 5B). The display image when the lesion is determined not to be a disease is, for example, the image shown in FIG. 10. When the lesion is determined to be a disease but the diagnostic certainty is low, a display screen such as that shown in FIG. 12 is generated. That is, an emphasized image G1a is placed in display area D1, and the determined disease name and diagnostic certainty are placed in display area D2. The emphasized image G1a is generated by adding a marker image L5 to the location of the lesion on the medical image.

In this way, even if the lesion is determined to be a disease as a result of differentiation but the diagnostic certainty is low, the amount of information in the displayed image can be reduced, just as in the case where the lesion is determined to be not a disease, allowing the user to perform image diagnosis quickly.
Ninth Embodiment

In the above embodiment, we introduced a case where the rationale is presented as text on the display device 41, but the present invention is not limited to this, and the rationale information may also be input to the doctor via voice. Voice may also be combined with Examples 1 to 8.

In this case, the emphasis unit 104 synthesizes audio data that has been subjected to emphasis processing, such as increasing the volume of the emphasized items compared to the non-emphasized items, or reading out only the emphasized items first, followed by all of the evidence, and transmits this data to the audio device 42 via the output unit 105. The audio device 42 may be a general speaker, or it may be a directional speaker, earphones, headphones, or a bone conduction device. When a directional speaker, earphones, headphones, or bone conduction device is used, the evidence information can be transmitted only to a specific party (the doctor) via these, thereby reducing the possibility that the subject will see the evidence information.

The present invention is not limited to the above-described embodiments, and can be embodied by modifying the components in the implementation stage without departing from the spirit of the invention. Furthermore, various inventions can be created by appropriately combining the multiple components disclosed in the above-described embodiments. For example, some of the components shown in the embodiments may be deleted. Furthermore, components from different embodiments may be combined as appropriate.

Note that even if the operational flows in the claims, specifications, and drawings are explained using terms such as "first," "next," etc. for convenience, this does not mean that they must be performed in that order. Furthermore, it goes without saying that the steps that make up these operational flows may be omitted as appropriate if they do not affect the essence of the invention.

Furthermore, of the technologies described here, the controls mainly illustrated in the flowcharts can often be configured by program, and may be stored on a recording medium or recording unit. The recording medium or recording unit may be recorded at the time of product shipment, may use a distributed recording medium, or may be downloaded via the internet.

In the examples, the parts described as "parts" (sections or units) may be configured by combining dedicated circuits or multiple general-purpose circuits, and, if necessary, by combining a microcomputer, a processor such as a CPU, or a sequencer such as an FPGA that operates according to pre-programmed software. It is also possible to design the system so that some or all of the control is taken over by an external device, in which case a wired or wireless communication circuit is involved. Communication can be via Bluetooth (registered trademark), Wi-Fi, telephone lines, USB, etc. The dedicated circuits, general-purpose circuits, and control units may be integrated into an ASIC.

Claims (16)

a differentiation result receiving unit that receives a differentiation result of a lesion from the image;
a sentence generation unit that generates the basis information for the discrimination as a sentence;
a classification unit for classifying the components of the sentence into emphasis targets and non-emphasis targets;
an emphasis unit that emphasizes the emphasis target; and an output unit that outputs at least the emphasis target of the sentence to a monitor. The output unit also outputs the discrimination result to a monitor,
a picture generating unit that generates an emphasis picture associated with the emphasis target,
The output unit
The image processing device according to claim 1 , wherein the emphasized image is linked to the result of the discrimination and output to the monitor so that the output state of the object to be emphasized and the display of the emphasized image are linked. the output unit outputs both the emphasis target and the non-emphasis target of the sentence,
The highlighting section is
Underline what is emphasized and do not underline what is not emphasized,
Make the font of emphasized items bolder than that of non-emphasized items,
Make the font size of emphasized items larger than the font size of non-emphasized items,
Use a different font type for the highlighted text than for the unhighlighted text.
The color of the highlighted object is different from that of the non-highlighted object. The tilt angle of the highlighted object is different from that of the non-highlighted object. And
2. The image processing device according to claim 1, wherein the image processing device performs at least one of the following: making the blinking speed of the highlighted object faster than the blinking speed of the non-highlighted object. The diagram generating unit
2. The image processing device according to claim 1, wherein the color of the outline used in the highlighted image is the same color as the highlighted object. The diagram generating unit
The image processing device according to claim 1 , wherein a diagram is generated as the emphasized diagram, the diagram enclosing or pointing out a portion of the image that corresponds to the explanatory text to be emphasized. The classification unit
The emphasis target is classified into a first emphasis target and a second emphasis target,
the highlighting unit highlights the second emphasis target in an appearance or at a timing different from that of the first emphasis target,
the image generating unit generates a first emphasized image associated with the first emphasized object and a second emphasized image associated with the second emphasized object;
The output unit
Linking the highlighting state of the first highlight object with the display of the first highlight image;
The image processing device according to claim 1 , wherein the emphasis state of the second emphasis target and the display of the second emphasis image are linked. The highlighting unit highlights the second emphasis target in an appearance different from that of the first emphasis target by:
The image processing device according to claim 6 , wherein the second emphasis target is displayed in a color different from the color of the first emphasis target. the highlighting unit highlights the first emphasis target and the second emphasis target at different timings;
The image processing device according to claim 6 , wherein one of the first and second emphasis targets that is not at the timing of being emphasized is displayed in the same manner as a non-emphasis target. the highlighting unit highlights the first emphasis target and the second emphasis target at different timings;
The image processing device according to claim 6 , wherein the output unit does not output to the monitor one of the first emphasis target and the second emphasis target that is not at the timing to be emphasized. the highlighting unit highlights the first emphasis target and the second emphasis target at different timings;
The image processing apparatus according to claim 6 , wherein the order of enhancement is based on a predetermined diagnostic order. If the discrimination result matches a predetermined condition,
The image processing device according to claim 1 , wherein the emphasis unit emphasizes the emphasis target. the image processing device includes one or more processors;
The processor:
Receive the image identification results,
generating the basis information for the discrimination as a sentence;
Classifying the components of the sentence into emphasis targets and non-emphasis targets;
highlighting the emphasis target,
At least the emphasis target of the sentence is output to a monitor. Receive the image identification results,
generating the basis information for the discrimination as a sentence;
Classifying the components of the sentence into emphasis targets and non-emphasis targets;
highlighting the emphasis target,
and outputting at least the emphasis target of the sentence to a monitor. Receive the image identification results,
generating the basis information for the discrimination as a sentence;
Classifying the components of the sentence into emphasis targets and non-emphasis targets;
highlighting the emphasis target,
generating a diagram associated with the object to be emphasized;
an image display program that outputs at least the emphasis target of the sentence to a monitor. Receive the image identification results,
generating the basis information for the discrimination as a sentence;
Classifying the components of the sentence into emphasis targets and non-emphasis targets;
highlighting the emphasis target,
generating a diagram associated with the object to be emphasized;
A storage medium storing an image display program for outputting at least the highlighted portion of the text to a monitor. a classification unit that receives evidence information for the differentiation of a lesion and classifies components of the sentence that constitute the evidence information into emphasis targets and non-emphasis targets;
An image diagnostic system including: an emphasis unit that emphasizes the emphasis target; and an output unit that outputs at least the emphasis target of the sentence to a monitor.