JP7587815B2 - Medical image processing method, medical image processing device, and medical image processing program - Google Patents

Description

This disclosure relates to a medical image processing method, a medical image processing device, and a medical image processing program.

Methods for identifying target tissue from an area in which the subject's tissue and its surroundings are imaged are being considered. For example, Patent Literature 1 describes the use of a full-layer convolutional neural network (CNN) to create a model that segments at least one part of an anatomical structure from an MRI image.

Special Publication No. 2020-510463

However, while MRI images allow for more detailed examination of horizontal cross sections, it is difficult to grasp, for example, the condition of the subject's tissues in the vertical direction.

The present disclosure has been made in consideration of the above, and aims to provide a technology that can more appropriately grasp the condition of a target tissue in a direction different from the direction in which the image is acquired.

To achieve the above object, a medical image processing method according to one embodiment of the present disclosure includes: preparing a plurality of pixel information constituting a plurality of images of a cross section including a first axis, obtained by imaging a tissue to be analyzed at different positions along one direction; generating image information relating to an image of a second axis cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from the plurality of pixel information prepared; and identifying, using machine learning, an area in the image of the second axis cross section where the tissue to be analyzed is estimated to be imaged.

A medical image processing device according to one embodiment of the present disclosure includes an image preparation unit that prepares a plurality of pixel information constituting a plurality of images of a cross section including a first axis obtained by imaging a tissue to be analyzed at different positions along one direction, an image reconstruction unit that generates image information relating to an image of a second axis cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, among the plurality of pixel information prepared, and a target tissue identification unit that uses machine learning to identify an area in the image of the second axis cross section where the tissue to be analyzed is estimated to be imaged.

A medical image processing program according to one embodiment of the present disclosure is a medical image processing program for causing a computer to function as a medical image processing device, and causes a computer to execute the following: preparing a plurality of pixel information constituting a plurality of images of a cross section including a first axis, obtained by imaging a tissue to be analyzed at different positions along one direction; generating image information relating to an image of a second-axis cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared; and identifying, using machine learning, an area in the image of the second-axis cross section where the tissue to be analyzed is estimated to have been imaged.

According to the above medical image processing method, medical image processing device, and medical image processing program, image information relating to an image of a second axial section, which is an image of a plane including the second axis, is generated by combining pixel information relating to pixels included in the same plane including the second axis from multiple pixel information constituting multiple images of a section including the first axis. In addition, machine learning is used to identify an area of the image of the second axial section in which the tissue to be analyzed is estimated to be imaged. In this way, by generating image information relating to the image of the second axial section and then identifying an area from the image in which the tissue to be analyzed is estimated to be imaged, it becomes possible to more appropriately grasp the condition of the target tissue in a direction along the second axis, i.e., in a direction different from the direction in which the image was acquired.

In identifying the region presumed to have imaged the tissue to be analyzed, the embodiment may further include identifying the region presumed to have imaged the tissue to be analyzed among pixels included in the image of the cross section including the first axis, and creating three-dimensional data using information related to the region presumed to have imaged the tissue to be analyzed.

In this case, it is possible to obtain three-dimensional data of the target tissue, making it possible to more appropriately grasp the three-dimensional shape of the target tissue, including the second axis.

The tissue to be analyzed may be a tissue that performs cyclical operations, and in preparing the plurality of pieces of pixel information, the method further includes preparing a plurality of pieces of pixel information constituting a plurality of images of a cross section including the first axis at each of a plurality of time points during one cycle, generating image information corresponding to the second axial cross section for each of the plurality of time points, generating image information related to the image of the second axial cross section, and in identifying an area in which the tissue to be analyzed is presumed to have been imaged, identifying an area in the image of the second axial cross section for each of the plurality of time points in which the tissue to be analyzed is presumed to have been imaged, and calculating information related to the change in the tissue to be analyzed along the time series during one cycle from the result of identifying the area in which the tissue to be analyzed is presumed to have been imaged.

In this case, if the subject of analysis is a tissue that changes periodically, it is possible to obtain information related to the changes in the tissue over time, making it possible to properly grasp the periodic changes in the subject tissue in the direction along the second axis.

The present disclosure provides a technology that can more appropriately grasp the condition of a target tissue in a direction different from the direction in which the image was captured.

FIG. 1 is a block diagram illustrating the configuration of a medical image processing apparatus according to an embodiment. FIG. 2 is a diagram illustrating the hardware configuration of the medical image processing apparatus. FIG. 3 is a flow diagram illustrating the medical image processing method. FIG. 4 is a diagram for explaining an image of a horizontal cross section. FIG. 5 is a diagram showing the result of roughly specifying an area in a horizontal cross-sectional image in which the target tissue is presumed to be imaged. FIG. 6 is a diagram for explaining a process of reconstructing an image from a horizontal cross-sectional image and identifying a region in the vertical cross-sectional image obtained as a result of the reconstruction, in which a target tissue is estimated to be imaged. FIG. 7 is a diagram for explaining a case where the identification results in the XZ image and the YZ image are applied to the processing of the XY image. FIG. 8 is a diagram showing an example of a change in the vertical direction of a tissue.

Below, the embodiments for implementing the present disclosure will be described in detail with reference to the attached drawings. Note that in the description of the drawings, the same elements are given the same reference numerals, and duplicate descriptions will be omitted.

FIG. 1 is a block diagram for explaining the configuration of a medical image processing device according to one embodiment of the present disclosure. The medical image processing device 1 according to this embodiment is a device that calculates feature amounts related to an organ to be analyzed from image data obtained by capturing an image of the tissue (organ, etc.) of a subject. The target tissue (organ) is not particularly limited, but for example, an organ existing in the body can be the target. In addition, an organ that moves (deforms) at a specific cycle can be selected as the target organ. Examples of such organs include, but are not limited to, the heart (atrium or ventricle), lungs, liver, prostate, etc. For example, the heart repeatedly deforms at a cycle corresponding to the heartbeat. The medical image processing device 1 is also capable of calculating feature amounts focusing on changes along a time series. Therefore, it is also possible to calculate feature amounts focusing on periodic changes in tissues (organs) that repeatedly move (deform) at a specific cycle. Furthermore, the medical image processing device 1 is capable of calculating feature amounts related to deformation in the subject's body axis direction (vertical direction), which will be described in detail later. Therefore, it is particularly useful when targeting tissue that periodically changes (deforms) in the body axis direction, such as the right ventricle.

When the target tissue is the heart, an imaging test on a subject, for example an MRI test, produces a cine image of the short axis of the heart. The direction parallel to the cross section of this cine image is the first axis. From the cine image of the short axis of the heart, it is possible to grasp the movement of the heart along the first axis. On the other hand, from this cine image, it is difficult to grasp the movement of the heart in a direction perpendicular to the first axis, for example. In the following embodiment, an image of an axis different from the first axis, for example an axis (second axis) perpendicular to the first axis, is automatically reconstructed by a computer, and information on the target tissue in the second axis direction is added, thereby extracting the target tissue and grasping the information on the target tissue.

In the following embodiments, the right ventricle of the heart may be selected as the target tissue (organ, etc.). The right ventricle is a type of tissue whose shape changes periodically due to pulsation. In the following embodiments, the medical image processing device 1 is used to acquire features related to the change in shape of the right ventricle due to pulsation. In the following embodiments, the first axis extends horizontally (perpendicular to the body axis) and the second axis extends vertically (in the body axis direction). However, this setting is merely an example and may be changed depending on the characteristics of the target tissue, the method of acquiring images used by the medical image processing device 1, etc. For example, the first axis and the second axis may not be perpendicular to each other.

As shown in FIG. 1, the medical image processing device 1 has an image acquisition unit 11 (image preparation unit), an image processing unit 12, a feature amount calculation unit 13, a storage unit 14, and an output unit 15. The image processing unit 12 also includes an image reconstruction unit 21 and a target tissue identification unit 22.

The hardware of the medical image processing device 1 is composed of, for example, one or more control computers. For example, the medical image processing device 1 has a circuit 120 shown in FIG. 2. The circuit 120 has one or more processors 121, a memory 122, a storage 123, an input/output port 124, and a timer 125. The storage 123 has a storage medium readable by a computer, such as a hard disk. The storage medium stores a program (medical image processing program) for causing the medical image processing device 1 to execute the procedure related to medical image processing described below. The storage medium may be a removable medium such as a non-volatile semiconductor memory, a magnetic disk, or an optical disk. The memory 122 temporarily stores the program loaded from the storage medium of the storage 123 and the calculation results by the processor 121. The processor 121 configures each of the above-mentioned functional units by executing the above-mentioned program in cooperation with the memory 122. The input/output port 124 inputs and outputs electrical signals between each functional unit of the medical image processing device 1 according to instructions from the processor 121. The timer 125 measures the elapsed time, for example, by counting a reference pulse at a fixed interval.

Returning to FIG. 1, each functional unit of the medical image processing device 1 will be described. The image acquisition unit 11 of the medical image processing device 1 has a function of acquiring multiple image data obtained by imaging the target tissue (organ) of the subject for which the feature amount is to be calculated from an external device or the like. The multiple image data are multiple image data obtained by imaging the target organ. The multiple image data are, for example, multiple image data obtained by imaging a plane perpendicular to the body axis at different positions along the body axis of the subject. For example, a magnetic resonance imaging (CMR: Cardiovascular MRI) can be used as such image data. More specifically, a cine MRI image can be used. In general, a CMR image is obtained by imaging multiple times along the body axis (along one direction) to acquire three-dimensional information of the target tissue (organ) and its surroundings. However, the type of image data is not limited to the above. In addition, the type of image data can be changed depending on the target organ. In the medical image processing device 1, multiple image data including information related to three axes, namely the body axis and two axes perpendicular to the body axis, are acquired and the subsequent processing is performed. Note that the body axis generally extends in the vertical direction, and the plane perpendicular to the body axis corresponds to a horizontal plane. A CMR image corresponds to an image of a horizontal cross section of the subject.

In order to obtain time-series changes in the subject's target organ, the image data acquired by the image acquisition unit 11 may be image data captured multiple times at a specified interval. As an example, if the organ changes periodically, a group of image data acquired at multiple timings that capture the changes during one cycle may be acquired. For example, if the target organ is the heart (atrium or ventricle), expansion and contraction occur during one cycle (one beat). Therefore, in the case of the heart, image data at multiple timings including the diastole and systole may be prepared for calculating the feature amount, and the medical image processing device 1 may acquire this image data.

The image processing unit 12 has a function of reconstructing an image from multiple image data related to a target organ in order to obtain features related to tissues of other events, and performing processing to identify the tissues included in the reconstructed image. The image processing unit 12 has an image reconstruction unit 21 and a target tissue identification unit 22.

The image reconstruction unit 21 has a function of performing various processes using multiple image data to calculate feature amounts in the target tissue identification unit 22. The image reconstruction unit 21 has a function of reconstructing an image along a plane (a cross section including the vertical direction) including the body axis from information (pixel information) of each pixel included in an image of a cross section (a horizontal cross section) perpendicular to the body axis.

The target tissue identification unit 22 also has the function of identifying the target tissue (organ, etc.) in the image reconstructed by the image reconstruction unit 21.

The image reconstruction unit 21 and the target tissue identification unit 22 use machine learning and statistical processing using luminance information at each position (pixel) included in the image information to exercise the functions of each unit. Although details will be described later, image processing is performed by repeatedly performing machine learning and statistical analysis. Machine learning can include supervised learning (linear regression, support vector machine (SVM), logistic regression, etc.), unsupervised learning (principal component analysis, k-means method, etc.), reinforcement learning, deep learning (neural network, etc.), deep reinforcement learning, etc. In addition, principal component analysis, clustering, etc., which are considered to be a type of statistical analysis method, are also included in "machine learning" in this embodiment. In the following explanation, machine learning, statistical analysis, etc. will be collectively described as "machine learning, etc." When machine learning, etc. is used, an appropriate method may be selected depending on the type and number of data to be used.

The feature calculation unit 13 has a function of calculating a preset feature from the image after processing in the image processing unit 12. Examples of feature include various parameters related to the size, shape, and volume of the target tissue (organ, etc.) at a certain time, as well as differences in the rate of change of these over time and differences in the timing of change. Machine learning, statistical processing, etc. may also be used when calculating feature. The feature calculation unit 13 may have a function of calculating a preset feature from the image acquired by the image acquisition unit 11.

The memory unit 14 has the function of storing image data acquired by the image acquisition unit 11, image data reconstructed by the image processing unit 12, analysis results by the feature calculation unit 13, etc.

The output unit 15 has a function of outputting the calculation result by the feature amount calculation unit 13. Examples of the output destination include a monitor provided in the medical image processing device 1, an external device, and the like. The output contents, etc. are not particularly limited, and for example, the feature amount calculated by the feature amount calculation unit 13 may be output as is, or some evaluation may be performed based on the calculated feature amount and the result may be output. When performing an evaluation based on the feature amount, logic for performing the evaluation may be stored in the storage unit 14, and the evaluation may be performed using the logic after the feature amount is calculated. Furthermore, the output unit 15 may have a function of outputting image data reconstructed by the image processing unit 12, image data in which tissues are identified by the image processing unit 12, and the like. The output unit 15 may also be configured to generate and output three-dimensional data from the image data in which tissues are identified by the image processing unit 12.

Next, the procedure for image processing (image reconstruction) and feature calculation for multiple image data capturing target tissues (organs) by the medical image processing device 1 will be described with reference to Figures 3 to 6. Figure 3 is a flow diagram explaining the process for calculating feature amounts performed by the medical image processing device 1.

First, in the medical image processing device 1, the image acquisition unit 11 acquires a plurality of image data capturing an area including the target tissue (organ) (step S01: image acquisition step). An example of image data acquired by the image acquisition unit 11 is shown in FIG. 4. FIG. 4 shows a schematic diagram of a state in which the plurality of image data acquired in the image acquisition step are configured as an image. The plurality of image data are acquired, for example, at the same time (or within a period that can be considered to be the same time). The plurality of images are images captured along the body axis (Z axis) in an XY plane (horizontal plane) perpendicular to the body axis, and are data captured at different positions along the Z axis. The XY axes can be in any direction, but as an example, the X axis may be the left-right direction of the subject and the Y axis may be the front-back direction of the subject, as shown in FIG. 4.

In addition, when observing periodic deformation of tissue, it is necessary to obtain a plurality of image data a plurality of times along a time series t. As an example, FIG. 4 shows that a plurality of image data are obtained under the same conditions at three times, t1, t2, and t3. For example, it is assumed that at time t1, images P1 ₁ , P1 ₂ , ... P1 _n of horizontal cross sections are obtained along the body axis (Z axis), at time t2, images P2 ₁ , P2 ₂ , ... P2 _n of horizontal cross sections are obtained along the body axis (Z axis), and at time t3, images P3 ₁ , P3 ₂ , ... P3 _n of horizontal cross sections are obtained along the body axis (Z axis). Each image is captured along a plane perpendicular to the body axis, and corresponds to, for example, image P1 shown in FIG. 5. As an example, each of these horizontal cross-sectional images includes pixel information of 256 pixels in the XY direction. The images P1 ₁ , P1 ₂ , . . . P1 _n are images of parallel cross sections perpendicular to the body axis, and are images of the same position in the horizontal direction (X-axis direction and Y-axis direction).

For example, l pixels are provided at positions x1 to xl in the X-axis direction, and m pixels are provided at positions y1 to ym in the Y-axis direction. In this case, if image P1 ₁ is an image of a horizontal plane at Z=z1, the arrangement of pixels constituting the image data of image P1 ₁ can be expressed, for example, as coordinates (x1 to xl, y1 to ym, z1). Similarly, if image P1 _n is an image of a horizontal plane at Z=zn, the arrangement of pixels constituting the image data of image P1 _n can be expressed, for example, as coordinates (x1 to xl, y1 to ym, zn). As described above, if pixel information of 256 pixels is included in each of the X and Y directions, l=m=256.

When imaging is performed multiple times in a time series, the interval and number of imaging operations are appropriately changed depending on the target tissue. For example, the number and interval of imaging operations may be set so that imaging is performed multiple times within one cardiac cycle.

Next, in the medical image processing device 1, the image reconstruction unit 21 of the image processing unit 12 identifies an approximate area (rough area) that is estimated to be the area in which the target tissue to be reconstructed is imaged from the image (step S02). The reason for the "approximate area" is that the area identified at this stage is used as a reference for image reconstruction in the subsequent processing. Therefore, it is preferable that the "approximate area" is set so as to include the entire area in which the target tissue is imaged. FIG. 5 shows an image of the above-mentioned "approximate area" when identifying. In FIG. 5, for example, in the image P1, the area A1 corresponds to the "approximate area". It is estimated that the area with high brightness (whitish) in the area A1 is the area in which the target tissue is imaged, but the area A1 may be set as the "approximate area" so as to cover the entire area in which the target tissue is imaged. As a method for identifying the "approximate area", a method of identifying the "approximate area" in which the target tissue is imaged based on the difference in brightness in the image can be mentioned. In addition, machine learning or the like may be used as a method for identifying the approximate area in which the target tissue is imaged. As an example, the image reconstruction unit 21 may prepare multiple images in which the "approximate area" has been specified in advance as training data, and learn criteria for specifying the approximate area of the target from these images. As an example, multiple images in which the "approximate area" has been specified may be used to set a feature amount for distinguishing pixels that capture the "approximate area" from other pixels, and this may be used to set a feature space for distinguishing an image in which the "approximate area" has been captured from other pixels. In addition, the "approximate area" of the target may be identified from an image of the subject based on learning. When using machine learning, etc., preprocessing may be performed to adjust the luminance information of each pixel.

Next, in the medical image processing device 1, the image reconstruction unit 21 of the image processing unit 12 reconstructs the image for which the approximate region has been identified (step S03). This image reconstruction generates an image group corresponding to the XZ plane or the YZ plane from an image group captured along the XY plane.

Here, _the description will be given using images P1 ₁ to P1 _n . When the area (position of pixels ₎ captured by each image is expressed using three-dimensional coordinates using the XYZ axes shown in Fig. 4, it is expressed as follows (l, m, n are natural numbers):
P1 ₁ ; (x1, y1, z1)...(xl, ym, z1)
P1 ₂ ; (x1, y1, z2)...(xl, ym, z2)
P1 ₃ ; (x1, y1, z3)...(xl, ym, z3)
…
P1 _n ; (x1, y1, zn)...(xl, ym, zn)

For example, an image along the XZ plane is reconstructed from the information of each pixel contained in these images. In this case, the image along the XZ plane can be constructed by combining information of pixels whose X and Z coordinates are different from each other while keeping the Y coordinate constant. When the Y axis varies from 1 to m, for example, m images P1 _y1 to P1 _ym can be generated as reconstructed images.
P1 _y1 ; (x1, y1, z1)...(xl, y1, zn)
P1 _y2 ; (x1, y2, z1)...(xl, y2, zn)
P1 _y3 ; (x1, y3, z1)...(xl, y3, zn)
…
P1 _ym ; (x1, ym, z1)...(xl, ym, zn)

The image along the YZ plane can be reconstructed in a similar manner. The image along the YZ plane can be constructed by combining information on pixels with different Y and Z coordinates while keeping the X coordinate constant. When the X axis varies from 1 to l, for example, l images P1 _x1 to P1 _xl can be generated as reconstructed images.
P1 _x1 ; (x1, y1, z1)...(x1, ym, zn)
P1 _x2 ; (x2, y1, z1)...(x2, ym, zn)
P1 _x3 ; (x3, y1, z1)...(x3, ym, zn)
…
P1 _xl ; (xl, y1, z1)...(xl, ym, zn)

In this way, by changing the combination of pixel information contained in one image for each pixel contained in n images along the Z axis, an XZ image or YZ image can be reconstructed as a cross-sectional image related to a plane including the vertical direction (Z axis).

When reconstructing images in two planes in all patterns while changing all conditions as described above, reconstructed images in m+l patterns can be generated. Furthermore, when multiple images (for example, when t images are taken) are taken in a time series, reconstructed images in (m+l)×t patterns can be generated. In this case, in this step, instead of reconstructing images in all patterns, only a part of the images can be reconstructed. For example, image reconstruction can be performed on the XZ plane with a specific Y coordinate value, or on the YZ plane with a specific X coordinate value. The target for which image reconstruction is performed can be specified by, for example, the user (operator) of the device, or the image reconstruction unit 21 of the image processing unit 12 can determine which plane the image is to be reconstructed on based on a predetermined rule. When it is assumed that three-dimensional data is to be created, image reconstruction can be performed in multiple patterns in order to more accurately grasp the shape of the target tissue.

Next, in the medical image processing device 1, the target tissue identification unit 22 of the image processing unit 12 identifies the area in which the target tissue is imaged from the images (XZ image, YZ image) reconstructed by the image reconstruction unit 21 (step S04).

As a method for identifying the area where the target tissue is imaged, a method for identifying the area where the target tissue is imaged based on the difference in brightness in the image can be mentioned. Machine learning or the like can be used as a method for identifying the area where the target tissue is imaged. As an example, the image reconstruction unit 21 may prepare a plurality of images in which the area where the target tissue is imaged is identified in advance as training data, and learn a criterion for identifying the area where the target is imaged from these images. Also, the area where the target is imaged may be identified from the image of the subject based on the learning. When machine learning or the like is used, preprocessing such as adjusting the brightness information of each pixel may be performed. Note that the machine learning or the like used in this step S04 may be different from the machine learning or the like used in other steps. Also, the training data used in the machine learning or the like may be different depending on, for example, the XZ image, the YZ image, etc.

An example of the processing results in steps S03 and S04 is shown in FIG. 6. Image P1 shown in FIG. 6 is the same as image P1 shown in FIG. 5, and it is assumed that area A1 has been roughly identified. This image P1 corresponds to the XY image described above. Note that image P1 is an image in which the Y axis extends horizontally and the X axis extends vertically (see the bottom left of image P1).

Now, for example, suppose that an XZ image and a YZ image are reconstructed along a cross section passing through the center of gravity AC of area A1. Since the position of the center of gravity AC can be specified by three-dimensional coordinates, a reconstructed image can be obtained by combining information from pixels capturing an image of a plane having the same X or Y value as the coordinates of the center of gravity AC.

Image P2 is an XZ image taken along dashed line L1 in image P1, and image P3 is a YZ image taken along dashed line L2 in image P1. They are reconstructed images in a cross section passing through center of gravity AC. When images are acquired multiple times along a time series, the center of gravity is not calculated for each image, but an image of a plane passing through center of gravity AC in image data at a specific timing among the multiple images is reconstructed for each time. In other words, XZ and YZ images are reconstructed using the same center of gravity coordinates for each of the multiple image data along the time series. As a result, it is possible to identify changes in the shape of the target tissue in the same cross section along the time series.

Image P4 is an image obtained by extracting the area in image P2 where the target tissue is imaged. Image P5 is an image obtained by extracting the area in image P3 where the target tissue is imaged. Images P4 and P5 both display the area identified as the area where the target tissue is imaged with a higher brightness, so the identified area appears whiter than in images P2 and P3.

Next, in the medical image processing device 1, the target tissue identification unit 22 of the image processing unit 12 identifies, as necessary, the area in which the target tissue is imaged in the original image used when the image reconstruction unit 21 performs reconstruction, i.e., the XY image along the XY plane (step S05).

In the steps up to this point, only the "approximate area" in which the target tissue is imaged has been identified for the XY image (step S02). Therefore, the area in which the target tissue is imaged may also be identified again for the XY image using machine learning or the like. One method for identifying the area in which the target tissue is imaged is to identify the area in which the target tissue is imaged based on the difference in brightness in the image. Furthermore, machine learning or the like may be used as a method for identifying the area in which the target tissue is imaged, as with other planar images.

This step may be omitted depending on the type of feature to be calculated later. In other words, if it is determined that it is not necessary to identify the region of the target tissue in the XY image, this process may be omitted.

Next, if necessary, the area in which the target tissue is imaged is identified for the original image used for reconstruction by the image reconstruction unit 21, i.e., the XY image along the XY plane (step S05).

In the steps up to this point, only the "approximate area" in which the target tissue is imaged has been identified for the XY image (step S02). Therefore, the area in which the target tissue is imaged may also be identified again for the XY image using machine learning or the like. One method for identifying the area in which the target tissue is imaged is to identify the area in which the target tissue is imaged based on the difference in brightness in the image. Furthermore, machine learning or the like may be used as a method for identifying the area in which the target tissue is imaged, as with other planar images.

By configuring the system to identify the area in the XY image where the target is imaged (step S05) after identifying the area in the XZ image or YZ image where the target is imaged (step S04), the accuracy of identifying the area in which the target is imaged may be improved. This point will be explained with reference to FIG. 7.

In FIG. 7, it is assumed that imaging was performed 20 times (20 frames) along the time series t, and 15 images (15 slices) were acquired along the Z axis (body axis) for each imaging. The number of pixels in the XY plane can be, for example, 256 x 256 pixels. In FIG. 7, one image is shown as one cell C. Here, instead of showing 15 cells, only four cells are shown. Under such conditions, when identifying the area in which the target tissue is imaged in the XY image, the area in which the target is imaged is usually identified for all images, i.e., 15 x 20 images.

However, as a result of identifying the area in which the target tissue is imaged in the XZ image and the YZ image, an estimate is obtained as to which range in the Z-axis direction the area in which the target tissue is imaged exists. Therefore, it is possible to identify the image in which the area in which the target tissue is imaged exists among the 15 x 20 images. In FIG. 7, the cell C corresponding to the image in which the area in which the target tissue is imaged exists is hatched. In this way, by identifying the image in which the area in which the target tissue is imaged exists in advance, it is possible to perform a process of identifying the area in which the target tissue is imaged using only the identified XY image. By adopting such a process flow, it is possible to configure a configuration in which the process of identifying the area in which the target tissue is imaged is not performed for images in which it is estimated that the area in which the target tissue is imaged does not exist. Therefore, it is possible to prevent erroneous detection and improve workability.

This step may be omitted depending on the type of feature to be calculated later. In other words, if it is determined that it is not necessary to identify the region of the target tissue in the XY image, this process may be omitted.

Next, in the medical image processing device 1, the feature amount calculation unit 13 calculates feature amounts based on the area identified as the area in which the target tissue is imaged (step S06). The feature amounts to be calculated from the captured image (XY image) and the reconstructed images (XZ image, YZ image) can be set according to the type of target tissue or the disease to be analyzed. Therefore, the target and method for calculating the feature amounts can be changed as appropriate.

Figure 8 shows two images (YZ image) corresponding to image P5 and image P5' taken later in the time series under the same conditions. Image P5 corresponds to an image of the right ventricle in diastole, and image P5' corresponds to an image of the systole. Images P5 and P5' are shown adjacent to each other. Comparing images P5 and P5', it can be seen that the size of the target tissue in the Z direction (the area indicated by the white line in the figure) has changed. In this way, using an image of a plane including the Z axis makes it easier to detect changes in the tissue along the Z axis (such as organ deformation). It also makes it possible to calculate feature values based on the images, facilitating quantitative evaluation.

Returning to FIG. 3, the output unit 15 of the medical image processing device 1 outputs the results related to the feature amounts calculated by the above processing (step S07). The results to be output may be the feature amounts calculated by the above processing, or, if an evaluation is performed based on the feature amounts, may be the evaluation results.

When the results are output from the output unit 15, various evaluations and data processing may be performed as described above. As an example, when a process is performed to identify the area in which the target tissue is imaged in each of the XZ image, YZ image, and XY image, three-dimensional data may be created based on these identification results. In the above three types of images, images obtained at multiple different positions along each axis are acquired or reconstructed, and a process is further performed to identify the area in which the target tissue is imaged for each of them, so that the area in which the target tissue is imaged can be grasped three-dimensionally. In this case, if the information grasped three-dimensionally is combined and output as three-dimensional data, the user can grasp the shape of the tissue in three dimensions.

In addition, when the above processing is performed using images captured multiple times in a time series, it is possible to obtain information related to the change in the feature quantity according to the time series in the calculation of the feature quantity (step S06). In this case, if the images in the time series are acquired over one or more cycles, it is possible to obtain the periodic change in the feature quantity. Therefore, the configuration may be such that information related to this periodic change (e.g., maximum value, minimum value, etc.) is calculated and output.

When using images of the ventricles of the heart, the change in size of the ventricles in the body axis direction over time may be useful in analyzing cardiac disease or related diseases. For example, if remodeling of the heart (right ventricle) is occurring, the change in shape associated with pulsation may be useful information. Therefore, by acquiring features related to periodic tissue changes, it may be possible to more appropriately grasp the change in the shape of the right ventricle.

[Action]
According to the above-mentioned medical image processing method, medical image processing device, and medical image processing program, image information relating to an image of a vertical section (second axis section), which is an image of a plane including the vertical direction, is generated by combining pixel information relating to pixels included in the same plane including the vertical direction (second axis) from a plurality of pixel information constituting a plurality of images of a horizontal section (section including the first axis). In addition, a region in the image of the vertical section that is estimated to have imaged the tissue to be analyzed is identified using machine learning. In this way, by generating image information relating to the image of the vertical section and then identifying the region from the image that is estimated to have imaged the tissue to be analyzed, it is possible to more appropriately grasp the condition of the target tissue in the vertical direction, i.e., in a direction different from the direction in which the image was acquired.

Conventionally, analysis of target tissue has been performed using cine MRI images and the like. However, when grasping deformation of tissue in the vertical direction (extension direction of the second axis) along the body axis, it was generally necessary to estimate based on an image of a horizontal cross section (extension direction of the first axis). It is difficult to obtain a cross section of tissue in the vertical direction, and it has been required to grasp the shape of tissue in the vertical direction more accurately from an image of a horizontal cross section. In contrast, according to the above method, an image of a vertical cross section is generated using information contained in a horizontal cross section image, and a region in which the tissue to be analyzed is estimated to be imaged is identified from this image of the vertical cross section using machine learning. Since the region is identified using machine learning, it can be identified quickly and accurately compared to, for example, when a user individually identifies the region. Therefore, it is possible to grasp the shape of tissue in the vertical direction more accurately.

In addition, by identifying the area where the tissue to be analyzed is estimated to have been imaged, the area where the tissue to be analyzed is estimated to have been imaged in the horizontal cross-section image may also be identified. In this case, the result of identifying the area may be used to create three-dimensional data related to the tissue. In this case, since it is possible to obtain three-dimensional data of the target tissue, it is possible to more appropriately grasp the three-dimensional shape of the target tissue including the vertical direction, i.e., a direction different from the direction in which the image was acquired.

In addition, when the tissue to be analyzed is a tissue that performs cyclical behavior, multiple pieces of pixel information constituting multiple images of horizontal cross sections at multiple time points during one cycle may be prepared. Also, image information relating to images of vertical cross sections may be generated at each of the multiple time points, and the region may be identified. Furthermore, information relating to the chronological changes in the tissue to be analyzed during one cycle may be calculated from the identification results. In this case, when the tissue to be analyzed is a tissue that changes cyclically, information relating to the chronological changes in the tissue can be obtained, making it possible to appropriately grasp the cyclical changes in the target tissue in the vertical direction, i.e., in a direction different from the direction acquired as an image.

This disclosure is not limited to the above-described embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the disclosure. In addition, various disclosures can be formed by appropriately combining multiple components disclosed in the above-described embodiment.

For example, in the above embodiment, the image of the subject's target tissue (organ) is described as a cine MRI image, but the type of image is not limited as long as it is an image of a cross section of the tissue in a specific direction. Furthermore, the images including the first axis of the subject's target tissue (organ) may be images taken at different positions along the first axis, or may be images taken at different positions along a direction tilted relative to the first axis (one direction). Furthermore, the first axis and the second axis only need to intersect with each other, and are not limited to being "orthogonal" as described in the above embodiment. The configuration described in the above embodiment can also be applied to cases where the first axis and the second axis are not orthogonal.

In addition, in the above embodiment, a case where machine learning or the like is used in multiple steps has been described, but a method other than machine learning may be used except when identifying the area where the tissue to be analyzed is imaged. Also, the configuration may not include the rough area identification (step S02) described in the above embodiment. In this way, the procedure described in the above embodiment may be modified as appropriate.

1... medical image processing device, 11... image acquisition unit, 12... image processing unit, 13... feature amount calculation unit, 14... storage unit, 15... output unit, 21... image reconstruction unit, 22... target tissue identification unit.

Claims (6)

preparing a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along one direction;
generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared;
Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified;
A medical image processing method comprising:
In the step of identifying a region in which the tissue to be analyzed is estimated to have been imaged, the region in which the tissue to be analyzed is estimated to have been imaged is further identified from among pixels included in an image of a cross section including the first axis;
The medical image processing method further comprises generating three-dimensional data using information relating to a region presumably where the tissue to be analyzed is imaged. preparing a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along one direction;
generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared;
Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified;
A medical image processing method comprising:
the analyzed tissue is a tissue that performs a periodic operation,
In preparing the plurality of pieces of pixel information, a plurality of pieces of pixel information constituting a plurality of images of a cross section including the first axis are prepared at a plurality of time points in one period,
In generating image information corresponding to the second axial slice, image information relating to an image of the second axial slice is generated for each of the plurality of time points;
In identifying a region in which the tissue to be analyzed is estimated to have been imaged, an area in which the tissue to be analyzed is estimated to have been imaged is identified in the image of the second axial cross section for each of the plurality of time points;
The medical image processing method further includes calculating information relating to changes in the second axis along the time series of the tissue to be analyzed during one cycle from the result of identifying an area estimated to have imaged the tissue to be analyzed. an image preparation unit that prepares a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging the tissue to be analyzed at different positions along one direction;
an image reconstruction unit that generates image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared;
a target tissue specifying unit that specifies a region in the image of the second axial cross section in which the tissue to be analyzed is estimated to be imaged by using machine learning;
A medical image processing apparatus comprising:
the target tissue specifying unit, in specifying a region in which the analysis target tissue is estimated to have been imaged, further specifies a region in which the analysis target tissue is estimated to have been imaged among pixels included in an image of a cross section including the first axis;
The medical image processing apparatus further includes an output unit that creates three-dimensional data using information related to a region estimated to have been imaged of the tissue to be analyzed . an image preparation unit that prepares a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging the tissue to be analyzed at different positions along one direction;
an image reconstruction unit that generates image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared;
a target tissue specifying unit that specifies a region in the image of the second axial cross section in which the tissue to be analyzed is estimated to be imaged by using machine learning;
A medical image processing apparatus comprising:
the analyzed tissue is a tissue that performs a periodic operation,
the image preparation unit prepares a plurality of pieces of pixel information constituting a plurality of images of a cross section including the first axis at a plurality of time points in one cycle,
the image reconstruction unit generates image information relating to an image of the second axial slice for each of the plurality of time points in generating image information corresponding to the second axial slice;
the target tissue specifying unit, in specifying a region in which the analysis target tissue is estimated to have been imaged, specifies a region in which the analysis target tissue is estimated to have been imaged in the image of the second axial cross section for each of the plurality of time points;
The medical image processing device further includes a feature calculation unit that calculates information related to changes in the second axis along the time series of the tissue to be analyzed during one cycle from the result of identifying the area estimated to have been imaged of the tissue to be analyzed. A medical image processing program for causing a computer to function as a medical image processing device, comprising:
preparing a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along one direction;
generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared;
Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified;
on the computer ,
In the step of identifying a region in which the tissue to be analyzed is estimated to have been imaged, a region in which the tissue to be analyzed is estimated to have been imaged is further identified from among pixels included in an image of a cross section including the first axis,
The medical image processing program further causes the computer to create three-dimensional data using information related to a region estimated to have been imaged of the tissue to be analyzed . A medical image processing program for causing a computer to function as a medical image processing device, comprising:
preparing a plurality of pieces of pixel information constituting a plurality of images of a cross section including a first axis, the images being obtained by imaging a tissue to be analyzed at different positions along one direction;
generating image information relating to an image of a second axial cross section, which is an image of a plane including the second axis, by combining pixel information relating to pixels included in the same plane including a second axis intersecting the first axis, from among the plurality of pixel information prepared;
Using machine learning, a region in the image of the second axial cross section that is estimated to include the tissue to be analyzed is identified;
on the computer,
the analyzed tissue is a tissue that performs a periodic operation,
In preparing the plurality of pieces of pixel information, a plurality of pieces of pixel information constituting a plurality of images of a cross section including the first axis at each of a plurality of time points in one period are prepared,
In generating image information corresponding to the second axial cross section, image information relating to an image of the second axial cross section is generated for each of the plurality of time points;
In identifying a region in which the tissue to be analyzed is estimated to have been imaged, a region in which the tissue to be analyzed is estimated to have been imaged is identified in the image of the second axial cross section for each of the plurality of time points;
The medical image processing program further causes the computer to calculate information relating to changes in the second axis along the time series of the tissue to be analyzed during one cycle from the result of identifying the area estimated to have been imaged of the tissue to be analyzed.

Images