[go: up one dir, main page]

WO2016139925A1 - Appareil, procédé et système de traitement d'image - Google Patents

Appareil, procédé et système de traitement d'image Download PDF

Info

Publication number
WO2016139925A1
WO2016139925A1 PCT/JP2016/001091 JP2016001091W WO2016139925A1 WO 2016139925 A1 WO2016139925 A1 WO 2016139925A1 JP 2016001091 W JP2016001091 W JP 2016001091W WO 2016139925 A1 WO2016139925 A1 WO 2016139925A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
processing apparatus
images
image processing
characteristic
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2016/001091
Other languages
English (en)
Inventor
Shiori OSHIMA
Kazuhiro Nakagawa
Eriko Matsui
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2015206519A external-priority patent/JP6759550B2/ja
Application filed by Sony Corp filed Critical Sony Corp
Priority to US15/554,018 priority Critical patent/US10586333B2/en
Publication of WO2016139925A1 publication Critical patent/WO2016139925A1/fr
Anticipated expiration legal-status Critical
Priority to US16/719,583 priority patent/US20200118271A1/en
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing

Definitions

  • the present technology relates to an information processing apparatus, a program, an information processing method and an observation system usable to an image analysis in a biochemistry field.
  • an image analysis is used for analyzing cells and body tissues. Specifically, a movement analysis is done on images where an object to be observed such as cells is captured over time, a feature amount relating to a movement including a moving speed and a moving area can be extracted and visualized (see Patent Document 1, for example).
  • cells and body tissues that can be captured via a microscope etc. are used as an object to be analyzed.
  • analyses such as a cancer cell chemotaxis, a neuron cell outgrowth and a cardiac muscle cell pulsation.
  • Patent Document 1 Japanese Patent 2014-179061
  • an object of the present technology is to provide an information processing apparatus, a program, an information processing method and an observation system suitable for analyzing quantitatively a movement of an object to be observed.
  • an image processing apparatus includes circuitry configured to receive at least two images of a biological sample and determine motion information for a plurality of regions of the at least two images.
  • the motion information corresponds to motion of the biological sample.
  • the circuitry is further configured to generate a graphical representation of at least two characteristic amounts.
  • the at least two characteristic amounts correspond to a region of the plurality of regions and one characteristic amount of the at least two characteristic amounts is indicative of the motion information.
  • an imaging processing method includes receiving at least two images of a biological sample and determining motion information for a plurality of regions of the at least two images.
  • the motion information corresponds to motion of the biological sample.
  • the method includes generating a graphical representation of at least two characteristic amounts.
  • the at least two characteristic amounts correspond to a region of the plurality of regions and one characteristic amount of the at least two characteristic amounts is indicative of the motion information.
  • an imaging processing system includes at least one image sensor configured to acquire at least two images of a biological sample.
  • the imaging processing system further includes circuitry configured to receive at least two images of a biological sample and determine motion information for a plurality of regions of the at least two images.
  • the motion information corresponds to motion of the biological sample.
  • the circuitry is further configured to generate a graphical representation of at least two characteristic amounts.
  • the at least two characteristic amounts correspond to a region of the plurality of regions and one characteristic amount of the at least two characteristic amounts is indicative of the motion information.
  • a schematic diagram showing a configuration of an observation system according to an embodiment of the present technology A schematic diagram of a moving image acquired by a moving image acquiring unit of an information processing apparatus included in the observation system. A schematic diagram of an analysis range set by a range setting unit of the information processing apparatus. A schematic diagram of an analysis range set by a range setting unit of the information processing apparatus. A plot diagram generated by a plot generating unit of the information processing apparatus. A schematic diagram showing a gating mode in a gating unit of the information processing apparatus. An analysis image generated by an analysis image generating unit of the information processing apparatus. A schematic diagram showing a hardware configuration of the information processing apparatus. A flow chart showing an operation of the information processing apparatus.
  • a plot diagram generated by a plot generating unit of the information processing apparatus included in an observation system according to an alternative embodiment of the present technology A schematic diagram of an object to be observed in an application embodiment of the present technology.
  • a graph showing a time change extracted by a feature amount extracting unit of an information processing apparatus included in an observation system according to an application embodiment of the present technology A graph showing a time change of a feature amount extracted by the feature amount extracting unit of the information processing apparatus.
  • a plot diagram generated by a plot generating unit of the information processing apparatus. A moving image according to an embodiment of the present technology.
  • a plot diagram according to an embodiment of the present technology A moving image according to an embodiment of the present technology.
  • An analysis image according to an embodiment of the present technology. A moving image according to an embodiment of the present technology.
  • a plot diagram according to an embodiment of the present technology. A plot diagram according to an embodiment of the present technology.
  • An analysis image according to an embodiment of the present technology. An analysis image according to an embodiment of the present technology.
  • An image visualizing a feature amount according to an embodiment of the present technology. A plot diagram according to an embodiment of the present technology.
  • a plot diagram according to an embodiment of the present technology. A plot diagram according to an embodiment of the present technology.
  • An analysis image according to an embodiment of the present technology. A moving image according to an embodiment of the present technology.
  • a plot diagram according to an embodiment of the present technology. A schematic diagram showing a waveform pattern of the movement according to an embodiment of the present technology.
  • a moving image according to an embodiment of the present technology. A plot diagram according to an embodiment of the present technology.
  • Fig. 1 is a schematic diagram showing a configuration of an observation system 1 according to the embodiment.
  • the observation system 1 includes an image capturing apparatus 11 and an information processing apparatus.
  • Fig. 1 shows an object to be observed S.
  • the object to be observed S may be cells cultured in a culture vessel, for example, may be body tissues, and may not especially be limited.
  • the image capturing apparatus 11 may capture the object to be observed S over time, and generate a moving image.
  • the image capturing apparatus 11 may be a microscope including a microscope optical system and an image sensor, for example. Note that the image capturing apparatus 11 is not limited to the microscope, and may be any apparatus that can capture the object to be observed S over time.
  • the information processing apparatus 12 acquires and processes the moving image captured by the image capturing apparatus 11.
  • the information processing apparatus 12 may be a personal computer, for example. Note that the information processing apparatus 12 may be integrally configured of the image capturing apparatus 11, or may be independent from the image capturing apparatus 11.
  • the information processing apparatus 12 includes, as a functional configuration, a moving image acquiring unit 121, a range setting unit 122, a feature amount extracting unit 123, a plot generating unit 124, a gating unit 125, an analysis image generating unit 126 and an image outputting unit 127.
  • the moving image acquiring unit 121 acquires the moving image where the object to be observed S is captured over time.
  • Fig. 2 is a schematic diagram showing a moving image acquired by the moving image acquiring unit 121.
  • the moving image M is configured of a plurality of images (frames) F captured continuously.
  • the number of the images F configuring the moving image M is optional, and may be several tens per second of the moving image M, for example.
  • the moving image M is captured under the condition that a visual field to the object to be observed S is fixed, and may include a movement of the object to be observed S (a cell contractile movement, a substance transport, etc.).
  • the moving image acquiring unit 121 may acquire the moving image directly from the image capturing apparatus 11, or may acquire the moving image via a network or stored in a storage.
  • the range setting unit 122 sets an analysis range in the moving image.
  • Fig. 3 is a schematic diagram showing analysis ranges A set by the range setting unit 122. As shown in Fig. 3, the range setting unit 122 zones the moving image M into a plurality of areas having the same area to set the analysis ranges A.
  • the size of each analysis range A is set in advance, or is determined by a user's designation. The size of each analysis range A may be several pixels to several tens pixel square.
  • the range setting unit 122 may set the analysis ranges A in a specific area of the moving image.
  • Fig. 4 is a schematic diagram showing the analysis ranges A set in the specific area of the moving image M.
  • the range setting unit 122 may set the analysis ranges A by a user's designation.
  • the range setting unit 122 performs the image processing such as object detection processing on the moving image M, and may set the analysis ranges A based on the result.
  • the range setting unit 122 may be set the analysis ranges A such that cell bodies detected in the moving image M are included, for example.
  • the analysis ranges A set in the moving image M are also set to the same ranges in the respective images F.
  • the feature amount extracting unit 123 extracts the feature amount from each analysis range A.
  • the feature amount extracting unit 123 may extract the feature amount that changes by a time (a time of capturing the moving image)(hereinafter, a first feature amount) and the feature amount that are not changed by a time (hereinafter, a second feature amount).
  • the first feature amount can be extracted from two or more of images F captured at different times.
  • the feature amount extracting unit 123 can compare at least two images included in the images F configuring the moving image M, and perform the movement analysis.
  • the two images are compared by a block matching method or the like to extract a movement of a corresponding pixel group between pixels as a movement vector.
  • a magnitude of the movement vector represents a moving amount (i.e., a moving speed) between the images compared, and a direction of the movement vector represents a movement direction.
  • the feature amount representing the movement property is calculated.
  • the feature amount extracting unit 123 calculates an average, maximum and minimum values, a standard deviation, a rate of acceleration, a frequency component, an integrated value, etc. of a space or a time in any segment to the feature amount representing the movement property for each analysis range A, thereby providing the first feature amount. Also, the feature amount extracting unit 123 calculates a percentage of the area having a movement (moving area) above a certain level in each analysis range A, thereby providing the first feature amount. Furthermore, the characteristic amount extracting unit 123 calculates a similarity degree to a specific waveform pattern for each analysis range A, or the number of occurrences of the specific waveform pattern, thereby providing the first characteristic amount.
  • the second feature amount can be extracted from one or more of images F.
  • the feature amount extracting unit 123 calculates a brightness value of each pixel in the images F, and calculates an average, maximum and minimum values, a standard deviation, a rate of acceleration, a frequency component, an integrated value, etc. of a space or a time in any segment to the brightness value for each analysis range A, thereby providing the second feature amount.
  • the feature amount extracting unit 123 calculates a percentage of the area (analysis area) where a substance exists in each analysis range A, thereby providing the second feature amount.
  • the feature amount extracting unit 123 can extract one or more of the first feature amounts and one or more of the second feature amounts for each analysis range. Also, the feature amount extracting unit 123 may extract two or more first feature amounts for each analysis range.
  • the plot generating unit 124 plots the feature amount calculated in each analysis range, and generates a plot diagram.
  • Fig. 5 is an illustrative plot diagram generated by the plot generating unit 124. As shown in Fig. 5, the plot generating unit 124 plots the value in each analysis range with a movement average value (first feature amount) as an X axis and with a movement standard deviation (first feature amount) as an Y axis. Each plot in Fig. 5 is a plot for each analysis range.
  • the plot generating unit 124 can plot either of the first feature amount or the second feature amount, and any of the first feature amount and the second feature amount may be plotted on the X axis or the Y axis.
  • the plot generating unit 124 may plot any two of the first feature amounts as the X axis and the Y axis.
  • the plot by the plot generating unit 124 is not limited to a two-dimensional plot.
  • the plot generating unit 124 can generate a multidimensional plot using three or more feature amounts including at least one first feature amount.
  • the gating unit 125 performs gating in the plot diagram generated by the plot generating unit 124.
  • Fig. 6 is a schematic diagram showing a mode of the gating. As shown in Fig. 6, the gating unit 125 sets a gating range G in the plot diagram. The gating unit 125 can set the gating range G by a user's designation.
  • the gating range G can be specified by an upper limit and a lower limit of an X axis value and a Y axis value in the plot diagram shown in Fig. 6. Also, the gating range G may be specified by one or both of the upper limit and the lower limit of the X axis value, or one or both of the upper limit and the lower limit of the Y axis value. Furthermore, the gating range G may be the range having an optional shape in the plot diagram.
  • the gating unit 125 extracts (gates) only plots included in the gating range G, and does not extract the plots not included in the gating range G.
  • the gating unit 125 can calculate the number of the plot(s) gated (included in the gating range G).
  • Fig. 6 shows a calculation result (in Fig. 6, "plot numbers").
  • the gating unit 125 can calculate a percentage of the plot(s) gated for all plots.
  • the analysis image generating unit 126 visualizes a result of the gating, and generates the analysis image.
  • Fig. 7 is an example of the analysis image. As shown in Fig. 7, the analysis image generating unit 126 applies a masking (black area) to the analysis range corresponding to the plot excluded by the gating (not including in the gating range G) to superimpose with the moving image, thereby generating the analysis image. Also, the analysis image generating unit 126 may visualize the result of the gating in other ways.
  • the image outputting unit 127 outputs images to a display apparatus, and displays them thereon.
  • the image outputting unit 127 can display the plot diagram (see Fig. 6) generated by the plot generating unit 124. At this time, the image outputting unit 127 may display the number of the plot(s) calculated and gated by the gating unit 125 and its percentage together with the plot diagram (see Fig. 6).
  • the image outputting unit 127 can display the analysis image (see Fig. 7) generated by the analysis image generating unit 126.
  • Fig. 8 is a schematic diagram showing a hardware configuration of the information processing apparatus 12.
  • the information processing apparatus 12 includes a CPU 151, a GPU 152, a memory 153, a storage 154 and an input and output unit (I/O) 155, as a hardware configuration. These are mutually connected by a bus 156.
  • I/O input and output unit
  • a CPU (Central Processing Unit) 151 controls other components in accordance with a program stored in the memory 153, performs data processing in accordance with the program, and stores a processing result to the memory 153.
  • the CPU 151 may be a microprocessor.
  • a GPU (Graphic Processing Unit) 152 is controlled by the CPU 151, and executes the image processing.
  • the CPU 151 causes the GPU 152 to execute parallel arithmetic processing, and can extract the feature amount at a high speed.
  • the GPU 152 may be a microprocessor.
  • the memory 153 stores a program and data executed by the CPU 151.
  • the memory 153 may be a RAM (Random Access Memory).
  • the storage 154 stores a program and data.
  • the storage 154 may be a HDD (hard disk drive) and an SSD (Solid State Drive).
  • the input/output unit 155 receives an input to the information processing apparatus 12, and supplies an output from the information processing apparatus 12 to outside.
  • the input/output unit 155 includes an input device such as a keyboard and a mouse, an output device such as a display, and a connection interface such as a network.
  • the hardware configuration of the information processing apparatus 12 is not limited to one shown here, and may be the one that realizes the functional configuration of the information processing apparatus 12. Alternatively, a part or all of the hardware configuration may exist on the network.
  • FIG. 9 is a flow chart showing an operation of the information processing apparatus 12.
  • the moving image acquiring unit 121 acquires the moving image (St101).
  • the moving image is provided by capturing the object to be observed over time, as described above.
  • the moving image acquiring unit 121 supplies the moving image acquired to the range setting unit 122.
  • the range setting unit 122 sets the analysis range in the moving image.
  • the range setting unit 122 may set the analysis range by zoning the whole moving image into a plurality of areas (St102A), or may set the analysis range only at the specific area (St102B).
  • the range setting unit 122 supplies the moving image where the analysis range is set to the feature amount extracting unit 123.
  • the feature amount extracting unit 123 execute the analysis processing on the moving image, and extracts the feature amount for each analysis range (St103).
  • the feature amount includes at least one of the first feature amount and the second feature amount, or includes at least two first feature amounts.
  • the feature amount extracting unit 123 supplies the feature amount extracted to the plot generating unit 124.
  • the plot generating unit 124 plots the feature amount per analysis range, and generates the plot diagram (St104).
  • the plot generating unit 124 may plot the first feature amount and the second feature amount as the X axis and the Y axis. Also, the plot generating unit 124 may plot the two first feature amounts as the X axis and the Y axis.
  • the plot generating unit 124 supplies the plot diagram generated to the image outputting unit 127.
  • the image outputting unit 127 outputs the plot diagram to the display apparatus, and displays the plot diagram on the display apparatus (St105). A user can determine the gating range by referring to the plot diagram.
  • the gating unit 125 may supply the number of the plots and the percentage included in the gating range designated to the image outputting unit 127, and may display them thereon.
  • the gating unit 125 performs the gating by utilizing the gating range designated (St106).
  • the gating unit 125 supplies the analysis range corresponding to the gated plot (included in the gating range) to the analysis image generating unit 126.
  • the analysis image generating unit 126 visualizes the result of the gating, and generates the analysis image (St107).
  • the analysis image generating unit 126 supplies the analysis image to the image outputting unit 127.
  • the image outputting unit 127 outputs the analysis image to the display apparatus, and displays the analysis image on the display apparatus (St108).
  • the gating unit 125 performs the gating using a new gating range, and the analysis image generating unit 126 can update and display the analysis image.
  • the information processing apparatus 12 performs the above-described operations.
  • the movement of the object to be observed (such as movement amount and moving area) can be quantitatively evaluated, and the evaluation result can be visually perceived.
  • the information processing apparatus 12 can analyze two or more moving images by comparing them.
  • the moving images analyzed by the information processing apparatus 12 are defined as the first moving image and the second moving image.
  • the first moving image and the second moving image are provided by capturing the object to be observed under different observation conditions.
  • the object to be observed is captured for a long time (for example, about several hours) to provide the first moving image
  • the object to be observed is captured for a short time (for example, about several seconds) in the same visual field of first moving image to provide the second moving image.
  • the first moving image provided by capturing the object to be observed for a long time may be extracted for a certain period of time to provide the second moving image.
  • the first moving image and the second moving image may be provided by capturing the object to be observed under different culture environments.
  • the object to be observed is captured under a normal condition to provide the first moving image
  • the object to be observed is captured by changing the culture environment in the same visual field of first moving image to provide the second moving image.
  • the culture environment is changed by selecting administration of a medical agent or not, selecting a time from the administration of the medical agent, selecting a concentration of the medical agent, for example.
  • the information processing apparatus 12 can extract the feature amounts from the first moving image and the second moving image by the above-described method, and generate the plot diagram. Specifically, when the moving image acquiring unit 121 acquires the first moving image, the range setting unit 122 sets the analysis range in the first moving image. The feature amount extracting unit 123 extracts the feature amount from each analysis range.
  • the range setting unit 122 sets the analysis range in the second moving image. At this time, the range setting unit 122 sets the analysis range having the same arrangement as the first moving image.
  • the feature amount extracting unit 123 extracts the feature amount same as the feature amount extracted in the first moving image from each analysis range.
  • the plot generating unit 124 plots the feature amount extracted from the first moving image and the feature amount extracted from the second moving image to generate one plot diagram.
  • Fig. 10 shows the plot diagram. As shown in Fig. 10, the plot diagram includes a plot P1 of the feature amount extracted from the first moving image and a plot P2 of the feature amount extracted from the second moving image.
  • movement average values are compared, thereby providing a correlation between a movement of cells for several hours (cell migration etc.) and a movement of cells for several seconds (pseudopod movement, etc.).
  • the medical agent when the medical agent is administered or not on the first moving image and the second moving image, it is possible to evaluate an effect by the administration of the medical agent by comparing a maximum value of the movement.
  • the gating unit 125 may perform the gating, and the analysis image generating unit 128 may generate the analysis image, as described above.
  • the gating unit 125 may perform the gating on both of the plot P1 and the plot P2 in one gating range, or may perform the gating in different gating ranges for the plot P1 and the plot P2.
  • the analysis image generating unit 128 may generate the analysis image for the first moving image and the analysis image for the second moving image separately.
  • the information processing apparatus 12 can be used for evaluation of an intercellular transport.
  • Fig. 11 is a schematic diagram showing an intercellular transport, and shows that a cell C transports substances B.
  • the range setting unit 122 zones the moving image into a plurality of areas to set the analysis range.
  • analysis ranges A1 the analysis ranges where the substances B are passed are denoted as analysis ranges A1
  • analysis ranges A2 the analysis ranges where a transport of the substances B is retarded are denoted as analysis ranges A2.
  • the feature amount extracting unit 123 extracts the feature amount for each analysis range.
  • Fig. 12 shows a time change of the moving speed extracted in the analysis ranges A1.
  • Fig. 13 shows a time change of the moving area extracted in the analysis ranges A1. In Fig. 12 and Fig. 13, there are peaks derived from the pass of the substances B.
  • Fig. 14 shows a time change of the moving speed extracted in the analysis ranges A2.
  • Fig. 15 shows a time change of the moving area extracted in the analysis ranges A2. In Fig. 14 and Fig. 15, there are no peaks derived from the pass of the substances B.
  • the plot generating unit 124 plots the feature amount extracted by the feature amount extracting unit 123, and generates the plot diagram.
  • Fig. 16 is a plot diagram with the analysis area as the X axis and with a maximum value of the moving speed as the Y axis. As shown in Fig. 16, a coordinate (H1 in Fig. 16) and a coordinate (H2 in Fig. 16) are different. In the coordinate (H1 in Fig. 16), the plot of the analysis range (such as the analysis ranges A1) where the substance transfer is active is positioned. In the coordinate (H2 in Fig. 16), the plot of the analysis range (such as the analysis ranges A2) where the substance transfer is retarded is positioned.
  • the feature amounts such as the movement average value, the standard deviation and the moving area can be used other than the maximum value of the movement of the moving speed and the analysis area.
  • Fig. 17 shows a moving image to be analyzed.
  • the whole moving image shown in Fig. 17 was zoned to set the analysis range, and a variety of feature amounts were calculated for each analysis range.
  • the feature amounts calculated are an average value of a moving speed, a maximum value, a standard deviation, a rate of acceleration, a moving area and an analysis area for each analysis range.
  • Fig. 18 is a plot with the average value of the moving speed as the X axis, and with the moving area as the Y axis. As shown in Fig. 18, it shows that as the average value of the moving speed is increased, the moving area tends to be increased.
  • Fig. 19 is a moving image of lung cancer cells (H1975 cell). The whole moving image shown in Fig. 19 was zoned to set the analysis range, and calculated a variety of feature amounts for each analysis range.
  • Fig. 20 and Fig. 21 are plots with the analysis area as the X axis and with the average value of the moving speed as the Y axis.
  • Fig. 22 shows an analysis image generated by masking the analysis range other than the analysis range corresponding to the gated plot (included in gating range G1).
  • Fig. 23 shows the analysis image generated by masking the analysis range other than the analysis range corresponding to the gated plot (included in gating range G2).
  • Fig. 24 is a moving image of other of lung cancer cells (H358 cell). The whole moving image shown in Fig. 19 was zoned to set the analysis range, and calculated a variety of feature amounts for each analysis range.
  • Fig. 25 and Fig. 26 are plots with the analysis area as the X axis and with the average value of the moving speed as the Y axis.
  • Fig. 27 shows an analysis image generated by masking the analysis range other than the analysis range corresponding to the gated plot (included in gating range).
  • Fig. 28 shows the analysis image generated by masking the analysis range other than the analysis range corresponding to the gated plot.
  • Fig. 22 When Fig. 22 is compared with Fig. 23, the gated (not masked) areas are greatly different. In contrast, when Fig. 27 is compared with Fig. 28, a difference between the gated areas is small. It is therefore shown that the movement of the lung cancer cells shown in Fig. 19 is at low speed, and the movement of the lung cancer cells shown in Fig. 24 is at high speed.
  • moving image M 1 An moving image provided by capturing cultured cardiac muscle cells
  • moving image M 2 a moving image provided by capturing cultured cardiac muscle cells to which a low concentration of glycyrrhetinic acid is administered
  • moving image M 3 a moving image provided by capturing cultured cardiac muscle cells to which a high concentration of glycyrrhetinic acid is administered
  • Fig. 29 shows a time change of the moving speed for a specific analysis range in the moving image M1.
  • Fig. 30 shows a time change of the moving speed for a specific analysis range in the moving image M2.
  • Fig. 31 shows a time change of the moving speed for a specific analysis range in the moving image M3.
  • Fig. 32 is an image generated by adding shading to an average value of the moving speed per analysis range of the moving image M1.
  • Fig. 33 is an image generated by adding shading to an average value of the moving speed per analysis range of the moving image M2.
  • Fig. 34 an image generated by adding shading to an average value of the moving speed per analysis range of the moving image M3.
  • a propagation delay is generated in the cardiac muscle cells to which a low concentration of glycyrrhetinic acid is administered.
  • a propagation delay is generated in the cardiac muscle cells to which a high concentration of glycyrrhetinic acid is administered.
  • Fig. 35 is a plot diagram generated from the moving image M1.
  • Fig. 36 is a plot diagram generated from the moving image M2.
  • Fig. 37 is a plot diagram generated from the moving image M3.
  • Fig. 36 When Fig. 36 is compared with Fig. 35, it shows that, by administrating the cardiac muscle cells to a low concentration of glycyrrhetinic acid, a pulsation speed (the maximum value of the moving speed) is increased, and a pulsation number (the average value of the moving speed) is decreased. Furthermore, it shows that a deviation of the pulsation number is increased (a correlation between the average value and the maximum value of the moving speed is decreased).
  • Fig. 37 When Fig. 37 is compared with Fig. 35, it shows that, by administrating the cardiac muscle cells to a high concentration of glycyrrhetinic acid, a pulsation speed (the maximum value of the moving speed) is decreased, and a pulsation number (the average value of the moving speed) is increased. Furthermore, it shows that a deviation of the pulsation number is increased (a correlation between the average value and the maximum value of the moving speed is decreased).
  • Fig. 38 is a plot with the moving area as the X axis, and with the maximum value of the moving speed as the Y axis. As shown in Fig. 38, a range of X axis: 0 to 72%, Y axis: 1 to 23 ⁇ m/s (G5 in Fig. 38) was gated as the gating range.
  • Fig. 39 shows an example of an analysis image generated by masking the analysis range other than the analysis range corresponding to the gated plot.
  • a surface area of the moving area By plotting the moving area in the X axis, and the maximum value or the average value of the moving speed in the Y axis and selecting the area having the maximum value or the average value exceeding the predetermined value, a surface area of the moving area can be calculated.
  • the calculation of the surface area of the moving area allows evaluation of a state of the cells upon a medical assay and evaluation of muscle cells upon differentiation.
  • Fig. 40 is an moving image provided by capturing an electrode and cardiac muscle cells. The whole moving image was zoned to set the analysis range, and a variety of characteristic amounts were calculated for each analysis range.
  • Fig. 41 is a plot with the maximum value of the moving speed as the X axis and the average value of the moving speed as the Y axis extracted from the moving image.
  • Fig. 42 is a schematic diagram showing a waveform pattern of the movement.
  • a range of X axis: 20 to 70% (G6 in Fig. 41) was gated as the gating range.
  • the maximum values of the moving speed each having a certain height were taken out.
  • a plot of the maximum values of the moving speed exceeding 20% was 29.1%. It concludes that the parts functioning as the cardiac muscle occupy 30%.
  • Fig. 43 is an analysis image for visualizing a result of the gating, and shows the analysis image generated by masking the analysis range other than the analysis range corresponding to the gated plot.
  • the noise occurrence area often has a waveform pattern different from normal. Therefore, by specifying the areas having the predetermined value or more of the similarity degree to the specific waveform pattern, it is possible to specify the area other than the noise occurrence area.
  • the waveform patterns are different in the normal cells and the abnormal cells, it is possible to classify by calculating the similarity degree between the waveform patterns of the normal cells and the abnormal cells. This also applies to the characteristic amount such as the number of occurrences of the specific waveform pattern for each analysis range.
  • Fig. 44 is a moving image of sperms captured. The whole moving image was zoned to set the analysis range, and a variety of characteristic amounts were calculated for each analysis range.
  • Fig. 45 is a plot with the average value of the moving speed as the X axis and the moving area as the Y axis extracted from the moving image.
  • a range of Y axis: 30 to 40% was gated as the gating range.
  • Fig. 46 is an analysis image for visualizing a result of the gating, and shows the analysis image generated by masking the analysis range other than the analysis range corresponding to the gated plot.
  • sperms that move vigorously and specifying the moving area that is continuously large
  • the number may be counted.
  • a sperm mobility can be determined by calculating the number of the sperms that do not move in the object to be observed.
  • An image processing apparatus including: circuitry configured to: receive at least two images of a biological sample; determine motion information for a plurality of regions of the at least two images, wherein the motion information corresponds to motion of the biological sample; and generate a graphical representation of at least two characteristic amounts, wherein the at least two characteristic amounts correspond to a region of the plurality of regions and one characteristic amount of the at least two characteristic amounts is indicative of the motion information.
  • the graphical representation has a first axis corresponding to a first characteristic of the motion information and a second axis corresponding to a second characteristic.
  • the image processing apparatus of claim (2) wherein the graphical representation is a two-dimensional plot diagram of the at least two characteristic amounts on the first axis and the second axis.
  • the first characteristic is an average value, a maximum value, a standard deviation, a rate of acceleration, a frequency value, an integrated value, a percentage of area having movement above a predetermined threshold, a level of similarity with a waveform pattern, or a number of times when a level of similarity with a waveform pattern is above a predetermined threshold.
  • the image processing apparatus of claim (4) wherein the second characteristic is an average value, a maximum value, a standard deviation, a rate of acceleration, a frequency value, an integrated value, a motion area, a level of similarity with a waveform, or a number of times when showing specific waveform.
  • the second characteristic is an average value, a maximum value, a minimum value, a standard deviation, or an area percentage occupied by the biological sample.
  • the method further includes determining the at least two characteristic amounts by determining the first characteristic and the second characteristic for each region of the plurality of regions.
  • the image processing apparatus of claim (1) wherein the biological sample includes at least a portion of a cell, at least a portion of a tissue, or at least a portion of an animal.
  • the image processing apparatus of claim (1) wherein the method further includes receiving input indicating a portion of the at least two characteristic amounts.
  • the image processing apparatus of claim (9), wherein the method further includes presenting aggregate information for the portion of the at least two characteristic amounts, wherein the aggregate information indicates an amount of regions corresponding to the portion of the at least two characteristic amounts.
  • the method further includes generating an analysis image by superimposing on an image of the at least two images a mask at positions in the image excluded by regions corresponding to the portion of the at least two characteristic amounts.
  • analyzing the at least two images includes determining a feature of the biological sample and determining regions of the plurality of regions that include the feature.
  • analyzing the at least two images includes determining a motion vector corresponding to movement of a group of pixels between a first image of the at least two images and a second image of the at least two images.
  • An imaging processing method including: receiving at least two images of a biological sample; determining motion information for a plurality of regions of the at least two images, wherein the motion information corresponds to motion of the biological sample; and generating a graphical representation of at least two characteristic amounts, wherein the at least two characteristic amounts correspond to a region of the plurality of regions and one characteristic amount of the at least two characteristic amounts is indicative of the motion information.
  • An imaging processing system including: at least one image sensor configured to acquire at least two images of a biological sample; and circuitry configured to: receive the at least two images from the at least one image sensor; determine motion information for a plurality of regions of the at least two images, wherein the motion information corresponds to motion of the biological sample; and generate a graphical representation of at least two characteristic amounts, wherein the at least two characteristic amounts correspond to a region of the plurality of regions and one characteristic amount of the at least two characteristic amounts is indicative of the motion information.
  • the display is further configured to receive user input indicating a selection of an area of the graphical representation, wherein the area includes a plurality of data values of the graphical representation.
  • the display is further configured to display an analysis image based on the selected area in response to receiving the user input.
  • An information processing apparatus including: a characteristic amount extracting unit for extracting a first characteristic amount from at least two images captured at different times included in a moving image where an object to be observed is captured over time, and extracting a second characteristic amount from at least one image included in the moving image; and a plot generating unit for generating a plot diagram by plotting the first characteristic amount and the second characteristic amount.
  • a characteristic amount extracting unit for extracting a first characteristic amount from at least two images captured at different times included in a moving image where an object to be observed is captured over time, and extracting a second characteristic amount from at least one image included in the moving image
  • a plot generating unit for generating a plot diagram by plotting the first characteristic amount and the second characteristic amount.
  • the information processing apparatus according to (28) or (29) above, further including: a range setting unit for setting an analysis range in the moving image, in which the characteristic amount extracting unit extracts the first characteristic amount and the second characteristic amount per analysis range, and the plot generating unit generates a plot diagram by plotting the first characteristic amount and the second characteristic amount per analysis range.
  • a range setting unit for setting an analysis range in the moving image, in which the characteristic amount extracting unit extracts the first characteristic amount and the second characteristic amount per analysis range
  • the plot generating unit generates a plot diagram by plotting the first characteristic amount and the second characteristic amount per analysis range.
  • the range setting unit performs image processing on the moving image, and sets the analysis range based on a result of the image processing.
  • the moving image includes the first moving image and the second moving image where the object to be observed is captured under different conditions, and the plot generating unit generates a plot diagram by plotting a first characteristic amount and a second characteristic amount extracted from the first moving image and the first characteristic amount and the second characteristic amount extracted from the second moving image.
  • a program for functioning an information processing apparatus including: a characteristic amount extracting unit for extracting a first characteristic amount from at least two images captured at different times included in a moving image where an object to be observed is captured over time, and extracting a second characteristic amount from at least one image included in the moving image; and a plot generating unit for generating a plot diagram by plotting the first characteristic amount and the second characteristic amount.
  • An information processing method including: extracting a first characteristic amount from at least two images captured at different times included in a moving image where an object to be observed is captured over time; extracting a second characteristic amount from at least one image included in the moving image by a characteristic amount extracting unit; and generating a plot diagram by plotting the first characteristic amount and the second characteristic amount by a plot generating unit.
  • An observation system including: an image capturing apparatus for capturing an object to be observed over time, and generating a moving image; and an information processing apparatus including a characteristic amount extracting unit for extracting a first characteristic amount from at least two images captured at different times included in the moving image where an object to be observed is captured over time, and extracting a second characteristic amount from at least one image included in the moving image, and a plot generating unit for generating a plot diagram by plotting the first characteristic amount and the second characteristic amount.
  • observation system 11 image capturing apparatus 12 information processing apparatus 121 moving image acquiring unit 122 range setting unit 123 feature amount extracting unit 124 plot generating unit 125 gating unit 126 analysis image generating unit 127 image outputting unit 128 analysis image generating unit

Landscapes

  • Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Engineering & Computer Science (AREA)
  • Epidemiology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Primary Health Care (AREA)
  • Public Health (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Abstract

　L'invention concerne, selon certains aspects, un appareil de traitement d'image. L'appareil de traitement d'image comprend des circuits conçus pour recevoir au moins deux images d'un échantillon biologique et pour déterminer des informations de mouvement pour une pluralité de régions des deux images ou plus. Les informations de mouvement correspondent au mouvement de l'échantillon biologique. Les circuits sont en outre conçus pour générer une représentation graphique d'au moins deux quantités caractéristiques. Les deux quantités caractéristiques ou plus correspondent à une région parmi la pluralité de régions et une quantité caractéristique des deux quantités caractéristiques ou plus indique les informations de mouvement.
PCT/JP2016/001091 2015-03-04 2016-03-01 Appareil, procédé et système de traitement d'image Ceased WO2016139925A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/554,018 US10586333B2 (en) 2015-03-04 2016-03-01 Image processing apparatus, image processing method and image processing system
US16/719,583 US20200118271A1 (en) 2015-03-04 2019-12-18 Image processing apparatus, image processing method and image processing system

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
JP2015-042079 2015-03-04
JP2015042079 2015-03-04
JP2015206519A JP6759550B2 (ja) 2015-03-04 2015-10-20 情報処理装置、プログラム、情報処理方法及び観察システム
JP2015-206519 2015-10-20

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US15/554,018 A-371-Of-International US10586333B2 (en) 2015-03-04 2016-03-01 Image processing apparatus, image processing method and image processing system
US16/719,583 Continuation US20200118271A1 (en) 2015-03-04 2019-12-18 Image processing apparatus, image processing method and image processing system

Publications (1)

Publication Number Publication Date
WO2016139925A1 true WO2016139925A1 (fr) 2016-09-09

Family

ID=55646814

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2016/001091 Ceased WO2016139925A1 (fr) 2015-03-04 2016-03-01 Appareil, procédé et système de traitement d'image

Country Status (1)

Country Link
WO (1) WO2016139925A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0365928A2 (fr) * 1988-10-28 1990-05-02 Firma Carl Zeiss Procédé pour l'exploitation d'images de cellules
US5655028A (en) * 1991-12-30 1997-08-05 University Of Iowa Research Foundation Dynamic image analysis system
WO1998029732A1 (fr) * 1997-01-01 1998-07-09 Evotec Biosystems Gmbh Procede et dispositif pour detecter le mouvement d'objets microscopiques
US20060034545A1 (en) * 2001-03-08 2006-02-16 Universite Joseph Fourier Quantitative analysis, visualization and movement correction in dynamic processes
JP2014179061A (ja) 2013-02-14 2014-09-25 Sony Corp 分析システム、分析プログラム及び分析方法

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0365928A2 (fr) * 1988-10-28 1990-05-02 Firma Carl Zeiss Procédé pour l'exploitation d'images de cellules
US5031099A (en) * 1988-10-28 1991-07-09 Carl-Zeiss-Stiftung Process for the evaluation of cell pictures
US5655028A (en) * 1991-12-30 1997-08-05 University Of Iowa Research Foundation Dynamic image analysis system
WO1998029732A1 (fr) * 1997-01-01 1998-07-09 Evotec Biosystems Gmbh Procede et dispositif pour detecter le mouvement d'objets microscopiques
US20060034545A1 (en) * 2001-03-08 2006-02-16 Universite Joseph Fourier Quantitative analysis, visualization and movement correction in dynamic processes
JP2014179061A (ja) 2013-02-14 2014-09-25 Sony Corp 分析システム、分析プログラム及び分析方法

Similar Documents

Publication Publication Date Title
US20200118271A1 (en) Image processing apparatus, image processing method and image processing system
Stelzer et al. Statistical inference and multiple testing correction in classification-based multi-voxel pattern analysis (MVPA): random permutations and cluster size control
Cisler et al. A comparison of statistical methods for detecting context-modulated functional connectivity in fMRI
US10039501B2 (en) Computer-aided diagnosis (CAD) apparatus and method using consecutive medical images
US9662040B2 (en) Computer-aided diagnosis apparatus and method
US10872411B2 (en) Diagnostic imaging assistance apparatus and system, and diagnostic imaging assistance method
EP2842478A1 (fr) Dispositif de traitement d'image, procédé de traitement d'image, et programme de traitement d'image
US20250117655A1 (en) Weakly supervised learning with whole slide images
JP5442542B2 (ja) 病理診断支援装置、病理診断支援方法、病理診断支援のための制御プログラムおよび該制御プログラムを記録した記録媒体
US20140226069A1 (en) Analyzing system, analyzing program and analyzing method
Boussard et al. Three-dimensional spike localization and improved motion correction for Neuropixels recordings
Koukounis et al. A high performance hardware architecture for portable, low-power retinal vessel segmentation
US20150206064A1 (en) Method for supervised machine learning
WO2015040990A1 (fr) Dispositif d'analyse de maladie, procédé de commande et programme
Kumar et al. Computer-aided automated discrimination of Alzheimer's disease and its clinical progression in magnetic resonance images using hybrid clustering and game theory-based classification strategies
Zhang et al. Image enhancement algorithm using adaptive fractional differential mask technique.
Abdulraqeb et al. A novel segmentation algorithm for MRI brain tumor images
JP6746945B2 (ja) 情報処理装置、情報処理システム及び情報処理方法
US10803601B2 (en) Rapid assessment and visual reporting of local particle velocity
WO2016139925A1 (fr) Appareil, procédé et système de traitement d'image
WO2016174472A1 (fr) Procédé et appareil de détermination de comportement temporel d'un objet
JP2009175059A (ja) 環境負荷濃度変化判定装置及び方法
CN102525468A (zh) 一种基于脑功能成像的脑内激活趋势的分析方法
Lian et al. Generalized fMRI activation detection via Bayesian magnitude change point model
US20200020099A1 (en) Information processing device, information processing system, and information processing method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16713600

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15554018

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16713600

Country of ref document: EP

Kind code of ref document: A1