[go: up one dir, main page]

EP3136974B1 - Visualisation d'élastographie - Google Patents

Visualisation d'élastographie Download PDF

Info

Publication number
EP3136974B1
EP3136974B1 EP14736018.4A EP14736018A EP3136974B1 EP 3136974 B1 EP3136974 B1 EP 3136974B1 EP 14736018 A EP14736018 A EP 14736018A EP 3136974 B1 EP3136974 B1 EP 3136974B1
Authority
EP
European Patent Office
Prior art keywords
pixel
image
value
strain image
strain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP14736018.4A
Other languages
German (de)
English (en)
Other versions
EP3136974A1 (fr
Inventor
Reza AZAR
Kris Dickie
Michelle ALEXANDER
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BK Medical AS
Original Assignee
BK Medical AS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BK Medical AS filed Critical BK Medical AS
Publication of EP3136974A1 publication Critical patent/EP3136974A1/fr
Application granted granted Critical
Publication of EP3136974B1 publication Critical patent/EP3136974B1/fr
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Clinical applications
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
    • A61B8/4444—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device related to the probe
    • A61B8/4455—Features of the external shape of the probe, e.g. ergonomic aspects
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Clinical applications
    • A61B8/0833—Clinical applications involving detecting or locating foreign bodies or organic structures
    • A61B8/085—Clinical applications involving detecting or locating foreign bodies or organic structures for locating body or organic structures, e.g. tumours, calculi, blood vessels, nodules
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
    • A61B8/4483—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device characterised by features of the ultrasound transducer
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/46—Ultrasonic, sonic or infrasonic diagnostic devices with special arrangements for interfacing with the operator or the patient
    • A61B8/461—Displaying means of special interest
    • A61B8/463—Displaying means of special interest characterised by displaying multiple images or images and diagnostic data on one display
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/48—Diagnostic techniques
    • A61B8/485—Diagnostic techniques involving measuring strain or elastic properties
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5207—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of raw data to produce diagnostic data, e.g. for generating an image
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5238—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
    • A61B8/5246—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image combining images from the same or different imaging techniques, e.g. color Doppler and B-mode
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/30—Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/30—Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure
    • A61B2090/306—Devices for illuminating a surgical field, the devices having an interrelation with other surgical devices or with a surgical procedure using optical fibres

Definitions

  • the following generally relates to elastography visualization and more particularly to ultrasound elastography image visualization, and is described with particular application to an ultrasound imaging system.
  • An ultrasound imaging system has included at least an ultrasound probe and a console.
  • the ultrasound probe houses a transducer array of transducing elements, and the console includes a display monitor and a user interface.
  • the transducing elements transmit an ultrasound signal into a field of view and receive echoes produced in response to the signal interacting with structure therein.
  • the echoes are processed, producing a sequence of focused, coherent echo samples along focused scanlines of a scanplane.
  • the scanlines are scan converted into a format of a display monitor and visually presented as image via the display monitor.
  • ultrasound images are acquired from the tissue while the tissue undergoes compression or deformation.
  • the compression or deformation can be applied manually by the user (i.e. by slightly pressing the transducer against the tissue), induced by internal tissue motions (due to breathing or heart beat), or induced through focused beams of ultrasound energy to produce movement.
  • ultrasound signals are acquired and processed to generate the corresponding strain images.
  • the strain images have been displayed, for example, as a measure of tissue elasticity along-side the B-mode images.
  • Strain images generally, are a function of the applied compression. As a consequence, small compressions may not generate enough contrast and will lower the detectability of the tissue abnormalities, while large compressions may result in unreliable measurements and invalid images.
  • strain images are also a function of underlying tissue structure, as well as their ultrasonic signal to noise ratio. As such, some of the displayed pixels may not represent valid or useful information. Unfortunately, this may result in a false interpretation of the image.
  • EP-A-1889571 discloses an ultrasonic diagnostic apparatus in which allowance for modulus of elasticity is provided.
  • the apparatus comprise a computing section for determining a thickness change waveform which represents a variation in distance between two arbitrary measuring points on a subject based on received echo signals and a reference waveform, internal information of the subject being obtained by comparing the thickness change waveform and the reference waveform to one another.
  • Figure 1 illustrates an example imaging system 100, such as an ultrasound imaging system.
  • the ultrasound imaging system 100 includes a transducer array 102.
  • the transducer array 102 can include a one dimensional (1D) or two dimensional (2D) array of transducer elements 104.
  • the transducer elements 104 are configured to transmit ultrasound signals and receive echo signals.
  • Suitable arrays 102 include linear, curved, and/or otherwise shaped.
  • the transducer array 102 can be fully populated or sparse.
  • the ultrasound imaging system 100 includes transmit circuitry 106.
  • the transmit circuitry 106 generates a set of radio frequency (RF) pulses that are conveyed to the transducer array 102.
  • the set of pulses actuates a corresponding set of the transducer elements 104, causing the elements to transmit ultrasound signals into an examination or scan field of view.
  • RF radio frequency
  • the ultrasound imaging system 100 includes receive circuitry 108.
  • the receive circuitry 108 receives echoes (RF signals) generated in response to the transmitted ultrasound signals from the transducer array 102.
  • the echoes generally, are a result of the interaction between the emitted ultrasound signals and the structure (e.g., flowing blood cells, organ cells, etc.) in the scan field of view.
  • frames are acquired continuously at a rate of up to 30 frames per second or higher.
  • the ultrasound imaging system 100 includes a switch 110.
  • the switch 110 switches between the transmit circuitry 106 and the receive circuitry 108, depending on whether the transducer array 102 is operated in transmit or receive mode. In transmit mode, the switch 110 electrically connects the transmit circuitry 106 to the elements 104. In receive mode, the switch 110 electrically connects the receive circuitry 108 to the elements 104.
  • the ultrasound imaging system 100 includes a controller 112.
  • the controller 112 controls one or more of the transmit circuitry 106, the receive circuitry 108 or the switch 110. Such control can be based on available modes of operation. Examples of such modes of operation include one or more of B-mode, elastography mode, A-mode, velocity flow mode, Doppler mode, etc.
  • the ultrasound imaging system 100 includes a user interface (UI) 114.
  • the UI 114 may include one or more input devices (e.g., a button, a knob, a slider, a touch pad, etc.) and/or one or more output devices (e.g., a display screen, lights, a speaker, etc.).
  • the UI 114 can be used to select an imaging mode, activate scanning, etc.
  • the ultrasound imaging system 100 further includes an echo processor 116 that processes received echoes. Such processing may include applying time delays, weighting on the channels, summing, and/or otherwise beamforming received echoes.
  • the echo processor 116 produces a sequence of focused, coherent echo samples along focused scanlines of a scanplane. Other processing may lower speckle, improve specular reflector delineation, and/or includes FIR filtering, IIR filtering, etc.
  • the elastogram processor 118 processes the received signals and generates a strain image between at least two sequential frames, and a corresponding correlation image(s).
  • the elastogram processor 118 processes the B-mode images from the echo processor 116 and generates the strain image and the corresponding correlation image.
  • Figure 2 a non-limiting example of the elastogram processor 118 is schematically illustrated.
  • the elastogram processor 118 includes a motion estimator 202.
  • the motion estimator 202 estimates motion between sequences of received signals. In one non-limiting instance, this includes dividing the signals into windows of small overlapping windows and applying a motion tracking algorithm to each window. Elastograms with different resolutions can be generated by adjusting the size and overlap between these windows. Known and/or other motion algorithms can be employed.
  • the motion estimator 202 outputs a displacement image.
  • the motion estimator 202 also outputs a corresponding correlation image.
  • the displacement image represents the displacement between successive windows, and correlation image indicates a degree of match or similarity between the corresponding windows.
  • the normalized correlation image includes values between minus one (-1) and one (1), which indicates reliable sub-portions of the displacement image (e.g. sub-portions corresponding to higher correlation values) and less reliable and/or unreliable sub-portions of the displacement image (e.g. sub-portions corresponding to lower correlation values).
  • -1 means the windows are inverted version of each other, 0 means no similarity, and 1 means they are perfect match. It is understood that alternative measures of similarity such as non-normalized correlation image, sum of absolute differences, sum of absolute differences etc. as well as techniques that use phase shift estimation and complex correlation are contemplated herein.
  • the elastogram processor 118 further includes a spatial filter 204.
  • the spatial filter 204 applies a 2D spatial filtering to the displacement image and the correlation image. Suitable filters include a 2D medial filterer, a 2D mean filterer, and/or other filter.
  • the 2D medial filterer for example, removes outliers.
  • the 2D mean filterer for example, improves the signal to noise ratio (SNR).
  • the spatial filter 204 is omitted from the elastogram processor 118.
  • the elastogram processor 118 further includes a strain estimator 206.
  • the strain estimator 206 processes the displacement images and generates strain images.
  • the strain estimator 206 can employ known and/or other strain estimation algorithms.
  • An example of a known strain estimation algorithm includes a least-squares strain estimator.
  • the elastogram processor 118 further includes spatial and temporal filter 208.
  • the spatial and temporal filter 208 applies spatial filtering and temporal persistency to both the strain images and the correlation images. This, for example, improves the SNR of both the strain images and the correlation images.
  • the spatial and temporal filter 208 can employ known and/or other strain estimation algorithms. In a variation, the spatial and temporal filter 208 is omitted from the elastogram processor 118.
  • the elastogram processor 118 further includes a dynamic range adjuster 210.
  • the dynamic range adjuster 210 maps the strain values a predetermined scale such as the full scale other scale. In one instance, the mapping maintains the maximum dynamic range of the strain values. Examples of suitable algorithms include contrast stretching, histogram equalization, and/or other algorithms.
  • the ultrasound imaging system 100 further includes a scan converter 120.
  • the scan converter 120 scan converts the output of the echo processor 116 and generates data for display, for example, by converting the data to the coordinate system of a display.
  • the scan converter 120 also scan converts both the strain image and correlation images based on the geometry of the B-mode image. In a variation, a different scan converter is used for the B-mode image and the strain and correlation images.
  • the ultrasound imaging system 100 includes a rendering engine 122 and a display 124.
  • the rendering engine 122 visually presents the B-mode image overlaid with the strain image.
  • the rendering engine 122 overlays only the strain image (e.g., a color-coded or gray scale) over the B-mode image.
  • the strain image is a 1D map that maps the elasticity or strain values directly to the B-mode image.
  • the rendering engine 122 creates a 2D overlay image based on the strain image and the correlation image and/or the B-mode image.
  • the overlay image takes into account the reliability of each pixel of the strain image and gradually or abruptly visually suppresses (e.g., ignores, renders transparent, etc.) strain image pixels as a function of the reliability. This may include taking into account the signal used to create a B-mode image pixel, and visually suppressing the strain image pixel based on the reliability of the B-mode image pixel.
  • the observer of the combined image B-mode/strain image will be apprised of whether a strain image pixel corresponds to a valid or suspect measurement, which may facilitate mitigating a false interpretation of the B-mode image.
  • the displayed data can be used, for example, during the training phase when the clinicians are trying to improve their scanning techniques, during live scan to provide feedback to an end user such that the end user knows when a good sequence of elastograms have been acquired, during exam review when selecting individual frames where good images are generated, etc.
  • the clinician also allows, in one non-limiting instance, the clinician to better evaluate the generated images in real-time (as the data is acquired and images are generated) and/or off-line during the exam review, relative to just displaying the strain image alone over a B-mode image. It also allows the user to improve their scanning techniques and acquire better strain images. Generally, the processing of the rendering engine 122 improves visualization of strain images. It also provides for calculation of quality feedback to help clinicians acquire repeatable and more reliable strain images.
  • one or more of the echo processor 116, the elastogram processor 118, and the rendering engine 122 can be implemented via a processor (e.g., a microprocessor, central processing unit, etc.) executing one or more computer readable instructions encoded or embedded on computer readable storage medium, such as physical memory.
  • a processor e.g., a microprocessor, central processing unit, etc.
  • computer readable storage medium such as physical memory.
  • FIG. 3 an example of the rendering engine 122 is schematically illustrated.
  • the rendering engine 122 includes a graphics processor 302 and a visualization algorithm memory 304.
  • the graphics processor 302 receives, as input, a signal from the controller 112 ( Figure 1 ).
  • the signal indicates the visualization mode.
  • the visualization mode can be determined based on a predetermined default mode, a user input through the UI 114 ( Figure 1 ), and/or otherwise.
  • the graphics processor 302 also receives one or more of the B-mode image, the strain image and/or the correlation image.
  • the graphics processor 302 retrieves and/or invokes a suitable algorithm from the visualization algorithm memory 304.
  • the visualization algorithm memory 304 stores at least one of a soft blend 306 algorithm, a hard blend 308 algorithm or a B-mode priority 310 algorithm.
  • the graphics processor 302 displays the B-mode image with the strain image, unmodified, overlaid there over.
  • the strain image can be considered a 1D map in that it provides only strain information, which is mapped directly to the B-mode image.
  • the graphics processor 302 identifies a correlation value of a pixel from the correlation image that corresponds to the strain image pixel being processed.
  • the graphics processor 302 then identifies a transparency level for the correlation value. This can be through a look up table (LUT), a mathematical function (e.g., a polynomial), and/or otherwise.
  • the graphics processor 302 renders the pixel with the transparency level.
  • the transition in transparency from a correlation value of zero (or some other value) to a correlation value of one (or some other range) can be linear, non-linear, or have both linear and non-linear regions.
  • the transition in transparency can also be continuous, discrete or have both continuous and discrete regions.
  • strain image pixels with a corresponding correlation value less than a predetermined threshold are rendered transparent, and all other pixels are rendered opaque (or less transparent).
  • the graphics processor 302 identifies the correlation value of a pixel from the correlation image that corresponds to the strain image pixel being processed.
  • the graphics processor 302 the compares the correlation value with a predetermined threshold.
  • the graphics processor 302 then makes a binary decision as to whether to show the pixel or not (or display it transparent).
  • the graphics processor 302 renders the pixel accordingly.
  • This mode causes a sub-part of a strain image corresponding to low correlation to be completely removed or hidden.
  • the hard blend strain image like the soft blend strain image, can be considered a 2D image in that it provides strain information and strain reliably information.
  • the graphics processor 302 compares a B-mode image pixel value with a predetermined threshold. If the pixel value is less than the predetermined threshold, the strain image pixel corresponding to the B-mode image pixel is not displayed or is displayed transparent. Otherwise, the strain image pixel is displayed or is displayed, for example, based on the soft blend 306 algorithm, the hard blend 308 algorithm, and/or otherwise.
  • the B-mode priority strain image like the soft blend strain image and the hard blend strain image, can be considered a 2D image in that it provides strain information and strain reliably information.
  • Figure 4 show an example of a B-mode image.
  • Figure 5 shows an example of a strain image.
  • Figure 6 shows an example of the B-mode image of Figure 4 with the strain image of Figure 5 superimposed there over.
  • the strain image is superimposed on the B-mode image, and the correlation image is ignored.
  • Figure 6 invalid or poor strain measurements are displayed and shown to the end user. This may result in false interpretation of the image.
  • the strain image is shown in gray scale. However, it is to be understood that the strain image can be shown using a color map.
  • Figure 7 shows an example soft blend strain image.
  • the rendering engine 122 generates a soft blend strain image by applying a transparency map to the strain image.
  • the rendering engine 126 for a pixel in the strain image, identifies a corresponding pixel in the correlation image.
  • a correlation value is identified for the identified pixel.
  • a transparency level is then identified for the identified correlation value.
  • the transparency level is then applied to the pixel in the strain image. This is repeated for all the pixels in the strain image.
  • Figure 8 shows an example of the B-mode image of Figure 4 with the soft blend strain image of Figure 7 superimposed there over.
  • Figure 9 shows an example hard blend strain image.
  • the rendering engine 122 generates a hard blend image by thresholding the strain image.
  • the rendering engine 126 for a pixel in the strain image, identifies a corresponding pixel in the correlation image.
  • a correlation value is identified for the identified pixel.
  • the correlation value is compared against a predetermined threshold. If the correlation value is below the predetermined threshold, the strain image pixel value is set to completely transparent. Otherwise, the strain image pixel value is set with a transparency level from semi-transparent to opaque. This is repeated for all the pixels in the strain image.
  • Figure 10 shows an example of the B-mode image of Figure 4 with the hard blend strain image of Figure 9 superimposed there over.
  • Figure 11 shows an example B-mode priority image.
  • the rendering engine 122 identifies, for a pixel in the B-mode image, a pixel value.
  • the rendering engine 122 thresholds the B-mode image pixel value. For this, the rendering engine 126 compares the B-mode pixel value with a predetermined threshold. If the B-mode pixel value is below the predetermined threshold, the strain image pixel value is set to completely transparent or ignored. Otherwise, the strain image pixel value is set with a transparency level from semi-transparent to opaque.
  • Figure 11 shows an example of the B-mode image of Figure 4 with the only pixels of strain image of Figure 4 with values satisfying the threshold superimposed there over.
  • the pixels of strain image of Figure 4 with values that do not satisfy the threshold are not superimposed there over.
  • darker region 402 represent regions of low signal, and do not satisfy the predetermined threshold, and the strain image pixels corresponding to thereto are not shown.
  • Figure 12 illustrates a method
  • an array of ultrasound transducer elements is placed again a surface of a subject or object and activated to transmit an ultrasound signal into a field of view.
  • a first set of echoes is received.
  • pressure is applied to the subject or object. As discussed herein, this can be achieved by manually applying a pressure to the surface via the array, applying focused beams of ultrasound energy to the subject or object, etc.
  • a second set of echoes (compression echoes) is received.
  • the second set of echoes may include two or more sets of echoes.
  • the first set of echoes is processed, generating a B-mode image.
  • the second set of echoes is processed, generating a strain image and a correlation image.
  • the strain image and corresponding correlation image(s) may be generated from two or more sets of the second set of echoes.
  • the echoes are acquired continuously, and for each new acquisition, one B-mode image and one elastography image (e.g., by buffering the previous data) are generated.
  • an elastogram visualization algorithm is retrieved based on an elastogram mode of operation of interest.
  • the mode can be a default, user specified, changed, etc.
  • the soft blend 306 algorithm is applied. For this, for a pixel in the strain image, a corresponding pixel in the correlation image is identified. A correlation value is identified for the identified pixel. A transparency level is then identified for the identified correlation value. The transparency level is then applied to the pixel in the strain image. This is repeated for other pixels in the strain image.
  • the hard blend 308 algorithm is applied. For this, for a pixel in the strain image, a corresponding pixel in the correlation image is identified. A correlation value is identified for the identified pixel. The identified correlation pixel value is compared against a predetermined threshold. If the correlation pixel value is less than the threshold, the corresponding strain image pixel is set to completely transparent or ignored. Otherwise, the corresponding strain image pixel is set to semi-transparent or opaque. This is repeated for other pixels in the strain image.
  • the B-mode priority 310 algorithm is applied. For this, for a pixel in the B-mode image, a pixel is identified. The identified B-mode image pixel value is compared against a predetermined threshold. If the B-mode image pixel value is less than the threshold, the corresponding strain image pixel is set to completely transparent or ignored. Otherwise, the corresponding strain image pixel is set to semi-transparent or opaque. This is repeated for other pixels in the B-mode image.
  • the modified strain image is displayed, superimposed over the B-mode image.
  • At least a portion of the method discussed herein may be implemented by way of computer readable instructions, encoded or embedded on computer readable storage medium (which excludes transitory medium), which, when executed by a computer processor(s), causes the processor(s) to carry out the described acts. Additionally or alternatively, at least one of the computer readable instructions is carried by a signal, carrier wave or other transitory medium.
  • the embodiments described herein provide only valid and informative elastograms to the end user, for example, by making invalid regions of the strain image slightly or completely transparent, and/or blocking the elastography from displaying regions of the strain image that corresponds to regions of the B-mode image where not enough signal was acquired, for example, dark regions in the B-mode image.
  • the embodiments can be used, for example, during the training phase when the clinicians are trying to improve their scanning techniques, during live scan to provide feedback to an end user such that the end user knows when a good sequence of elastograms have been acquired, during exam review when selecting individual frames where good images are generated, etc.
  • Figures 13 and 14 illustrate non-limiting examples of the ultrasound imaging system 100.
  • at least the echo processor 116, the elastography processor 118, the scan converter 120, and the rendering engine 122 are part of console 1302 and 1402, and the display 124 and the console 1302 and 1402 are integrated in and part of respective mobile carts 1304 and 1404, which include movers 1306 and 1406 such as wheels, casters, etc.
  • the ultrasound imaging system 100 does not include movers and/or is not integrated into a cart, but instead rests on a table, desk, etc.
  • the ultrasound imaging system 100 is part of a hand-held ultrasound scanner.
  • An example of a hand-held scanner is described in US patent 7,699,776 , entitled “Intuitive Ultrasonic Imaging System and Related Method Thereof," and filed on March 6, 2003.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Gynecology & Obstetrics (AREA)
  • Vascular Medicine (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Claims (15)

  1. Procédé d'imagerie par ultrasons comprenant :
    l'émission (1202) d'un signal ultrasonore dans un champ de vision dans lequel un objet est disposé ;
    la réception (1204) d'un premier ensemble d'échos provenant du champ de vision ;
    la génération (1210) d'une image en mode B à partir du premier ensemble d'échos reçu ;
    l'application (1206) d'une compression à l'objet au sein du champ de vision ;
    la réception (1208) d'au moins un second ensemble d'échos provenant du champ de vision, chaque second ensemble d'échos comprenant des échos de compression ;
    la génération (1212) d'une image de déformation à partir d'une image de déplacement produite par l'estimation d'un mouvement entre des séquences des seconds ensembles d'échos ;
    la génération (1212) d'une image de corrélation indiquant un degré de similarité entre des séquences des seconds ensemble d'échos ;
    la modification (1216, 1218) de valeurs de pixel de l'image de déformation sur la base de l'un parmi : le degré de similarité entre des séquences des seconds ensembles d'échos et l'image en mode B générée pour générer une image de déformation modifiée ;
    l'affichage de l'image en mode B ; et
    la superposition (1222) de l'image de déformation modifiée sur l'image en mode B.
  2. Procédé selon la revendication 1, comprenant en outre :
    la modification des valeurs de pixel de l'image de déformation sur la base d'au moins l'un parmi : des valeurs de pixel de l'image de corrélation et des valeurs de pixel de l'image en mode B pour générer l'image de déformation modifiée.
  3. Procédé selon la revendication 1 ou 2, comprenant en outre :
    l'identification d'un pixel de l'image de déformation ;
    l'identification d'un pixel de l'image de corrélation correspondant au pixel de l'image de déformation ;
    l'identification d'une valeur de pixel du pixel identifié de l'image de corrélation, la valeur de pixel étant une valeur de corrélation au sein d'une plage prédéterminée ;
    la récupération d'une valeur de transparence prédéterminée pour la valeur de corrélation identifiée ;
    l'application de la valeur de transparence au pixel de l'image de déformation pour créer le pixel d'image de déformation modifié ; et
    la superposition du pixel d'image de déformation modifié sur le pixel d'image en mode B correspondant.
  4. Procédé selon la revendication 3, comprenant en outre :
    avant l'application de la valeur de transparence, la mise à l'échelle de la valeur de transparence en fonction de la valeur de corrélation du pixel de l'image de corrélation, et, l'étape d'application de la valeur de transparence au pixel de l'image de déformation inclut l'application de la valeur de transparence mise à l'échelle.
  5. Procédé selon la revendication 4, dans lequel la mise en échelle de la valeur de transparence comprend au moins l'un parmi : une transition continue de transparence et une transition discrète de transparence.
  6. Procédé selon la revendication 4 ou 5, dans lequel la mise à l'échelle de la valeur de transparence comprend au moins l'un parmi : une transition linéaire de transparence et une transition non linéaire de transparence.
  7. Procédé selon la revendication 3, dans lequel l'étape d'application de la valeur de transparence au pixel de l'image de déformation pour créer le pixel d'image de déformation modifié comprend l'application d'une valeur de transparence qui est soit transparente, soit dans une plage allant de semi transparent à opaque.
  8. Procédé selon l'une quelconque des revendications 3 à 7, comprenant en outre :
    la récupération de la valeur de transparence à partir d'une table de conversion établissant une correspondance entre les valeurs de transparence et les valeurs de corrélation.
  9. Procédé selon la revendication 1, comprenant en outre :
    l'identification d'un pixel de l'image de déformation ;
    l'identification d'un pixel de l'image de corrélation correspondant au pixel de l'image de déformation ;
    l'identification d'une valeur de pixel du pixel identifié de l'image de corrélation, la valeur de pixel étant une valeur de corrélation au sein d'une plage prédéterminée ;
    la comparaison de la valeur de pixel identifiée à un seuil prédéterminé ;
    le fait de rendre le pixel de l'image de déformation complètement transparent en réponse à la valeur de pixel identifiée ne satisfaisant pas le seuil prédéterminé pour créer le pixel d'image de déformation modifié ;
    le fait de rendre le pixel de l'image de déformation semi-transparent ou opaque en réponse à la valeur de pixel identifiée satisfaisant le seuil prédéterminé pour créer le pixel d'image de déformation modifié ; et
    la superposition du pixel d'image de déformation modifié sur le pixel d'image en mode B correspondant.
  10. Procédé selon la revendication 9, comprenant en outre :
    la réception d'un signal indiquant un changement par un utilisateur du seuil prédéterminé pour créer un seuil actualisé ;
    la comparaison de la valeur de pixel identifiée au seuil actualisé ;
    le fait de rendre le pixel de l'image de déformation complètement transparent en réponse à la valeur de pixel identifiée ne satisfaisant pas le seuil actualisé pour créer le pixel d'image de déformation modifié ;
    le fait de rendre le pixel de l'image de déformation semi-transparent ou opaque en réponse à la valeur de pixel identifiée satisfaisant le seuil actualisé pour créer le pixel d'image de déformation modifié ; et
    la superposition du pixel d'image de déformation modifié sur le pixel d'image en mode B correspondant.
  11. Procédé selon la revendication 1, comprenant en outre :
    l'identification d'un pixel de l'image en mode B ;
    l'identification d'une valeur de pixel pour le pixel identifié de l'image en mode B ;
    la comparaison de la valeur de pixel identifiée à un seuil prédéterminé ;
    le fait de rendre un pixel de l'image de déformation correspondant au pixel de l'image en mode B complètement transparent en réponse à la valeur de pixel identifiée ne satisfaisant pas le seuil prédéterminé pour créer le pixel d'image de déformation modifié ;
    le fait de rendre le pixel de l'image de déformation correspondant au pixel de l'image en mode B semi-transparent ou opaque en réponse à la valeur de pixel identifiée satisfaisant le seuil prédéterminé pour créer le pixel d'image de déformation modifié ; et
    la superposition du pixel d'image de déformation modifié sur le pixel d'image en mode B correspondant.
  12. Système d'imagerie par ultrasons (100) comprenant :
    un réseau de transducteurs (102, 104) configuré pour émettre des signaux ultrasonores dans un champ de vision et pour recevoir des échos provenant du champ de vision, les échos reçus incluant un premier ensemble d'échos et au moins un second ensemble d'échos, chaque second ensemble d'échos comprenant des échos de compression ;
    un circuit d'émission (106) configuré pour générer les signaux ultrasonores à des fins d'émission par le réseau de transducteurs (102, 104) ;
    un circuit de réception (108) configuré pour recevoir les échos à partir du réseau de transducteurs (102, 104) ;
    un dispositif de commande (112) configuré pour commander le circuit d'émission (106) et le circuit de réception (108) ;
    un processeur d'écho (116) configuré pour recevoir le premier ensemble d'échos à partir du circuit de réception (108) et pour traiter le premier ensemble d'échos pour générer une image en mode B ;
    un processeur d'élastogramme (118) configuré pour recevoir l'au moins un second ensemble d'échos à partir du circuit de réception (108) et pour traiter l'au moins un second ensemble d'échos pour :
    estimer un mouvement entre des séquences des seconds ensembles d'échos ;
    générer une image de déplacement à partir du mouvement estimé ;
    générer une image de corrélation indiquant un degré de similarité entre des séquences des seconds ensembles d'échos ;
    générer une image de déformation à partir d'une image de déplacement produite par l'estimation d'un mouvement ; et
    modifier des valeurs de pixel de l'image de déformation sur la base de l'un parmi : le degré de similarité entre des séquences des seconds ensembles d'échos et l'image en mode B générée pour générer une image de déformation modifiée ; et
    un moteur de rendu (122) configuré pour recevoir des données pour affichage à partir du processeur d'écho (116) et du processeur d'élastogramme (118) et pour traiter les données reçues pour afficher l'image en mode B et pour superposer l'image de déformation modifiée sur l'image en mode B.
  13. Système selon la revendication 12, dans lequel le moteur de rendu (122) est configuré en outre pour :
    identifier un pixel de l'image de déformation ;
    identifier un pixel de l'image de corrélation correspondant au pixel de l'image de déformation ;
    identifier une valeur de pixel du pixel identifié de l'image de corrélation, la valeur de pixel étant une valeur de corrélation au sein d'une plage prédéterminée ;
    récupérer une valeur de transparence prédéterminée pour la valeur de corrélation identifiée ;
    appliquer la valeur de transparence au pixel de l'image de déformation pour créer le pixel d'image de déformation modifié ; et
    superposer le pixel d'image de déformation modifié sur le pixel d'image en mode B correspondant.
  14. Système selon la revendication 12, dans lequel le moteur de rendu (122) est configuré en outre pour :
    identifier un pixel de l'image de déformation ;
    identifier un pixel de l'image de corrélation correspondant au pixel de l'image de déformation ;
    identifier une valeur de pixel du pixel identifié de l'image de corrélation, la valeur de pixel étant une valeur de corrélation au sein d'une plage prédéterminée ;
    comparer la valeur de pixel identifiée à un seuil prédéterminé ;
    rendre le pixel de l'image de déformation complètement transparent en réponse à la valeur de pixel identifiée ne satisfaisant pas le seuil prédéterminé pour créer le pixel d'image de déformation modifié ;
    rendre le pixel de l'image de déformation semi-transparent ou opaque en réponse à la valeur de pixel identifiée satisfaisant le seuil prédéterminé pour créer le pixel d'image de déformation modifié ; et
    superposer le pixel d'image de déformation modifié sur le pixel d'image en mode B correspondant.
  15. Système selon la revendication 12, dans lequel le moteur de rendu (122) est configuré en outre pour :
    identifier un pixel de l'image de déformation ;
    identifier un pixel de l'image de corrélation correspondant au pixel de l'image de déformation ;
    identifier une valeur de pixel du pixel identifié de l'image de corrélation, la valeur de pixel étant une valeur de corrélation au sein d'une plage prédéterminée ;
    comparer la valeur de pixel identifiée à un seuil prédéterminé ;
    rendre le pixel de l'image de déformation complètement transparent en réponse à la valeur de pixel identifiée ne satisfaisant pas le seuil prédéterminé pour créer le pixel d'image de déformation modifié ;
    rendre le pixel de l'image de déformation semi-transparent ou opaque en réponse à la valeur de pixel identifiée satisfaisant le seuil prédéterminé pour créer le pixel d'image de déformation modifié ; et
    superposer le pixel d'image de déformation modifié sur le pixel d'image en mode B correspondant.
EP14736018.4A 2014-05-02 2014-05-02 Visualisation d'élastographie Not-in-force EP3136974B1 (fr)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IB2014/061165 WO2015166311A1 (fr) 2014-05-02 2014-05-02 Visualisation d'élastographie

Publications (2)

Publication Number Publication Date
EP3136974A1 EP3136974A1 (fr) 2017-03-08
EP3136974B1 true EP3136974B1 (fr) 2018-12-19

Family

ID=51134146

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14736018.4A Not-in-force EP3136974B1 (fr) 2014-05-02 2014-05-02 Visualisation d'élastographie

Country Status (4)

Country Link
US (1) US20170049416A1 (fr)
EP (1) EP3136974B1 (fr)
CN (1) CN106456121B (fr)
WO (1) WO2015166311A1 (fr)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107205653B (zh) * 2015-01-09 2020-11-20 马克斯-普朗克科学促进学会 用于表征可激励以变形的介质的空间-时空动态的方法和设备
US20170347992A1 (en) * 2016-06-02 2017-12-07 Carestream Health, Inc. Automated region of interest placement
CN110477948B (zh) * 2019-08-21 2022-05-10 东软医疗系统股份有限公司 弹性成像方法及装置、成像设备、存储介质
US11250564B2 (en) * 2019-12-19 2022-02-15 GE Precision Healthcare LLC Methods and systems for automatic measurement of strains and strain-ratio calculation for sonoelastography
US20210321988A1 (en) * 2020-04-20 2021-10-21 Shenzhen Mindray Bio-Medical Electronics Co., Ltd. System and method for contrast enhanced ultrasound quantification imaging

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1549221B1 (fr) 2002-03-08 2010-09-15 University Of Virginia Patent Foundation Systeme d'imagerie ultrasonore intuitif et procede associe
US8298143B2 (en) * 2005-05-30 2012-10-30 Panasonic Corporation Ultrasonograph that determines tissue properties utilizing a reference waveform
US8081806B2 (en) * 2006-05-05 2011-12-20 General Electric Company User interface and method for displaying information in an ultrasound system
GB0708358D0 (en) * 2007-05-01 2007-06-06 Cambridge Entpr Ltd Strain image display systems
JP2013503680A (ja) * 2009-09-04 2013-02-04 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ フェード制御を用いた超音波弾性イメージング歪イメージング

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
WO2015166311A1 (fr) 2015-11-05
EP3136974A1 (fr) 2017-03-08
CN106456121A (zh) 2017-02-22
CN106456121B (zh) 2019-11-05
US20170049416A1 (en) 2017-02-23

Similar Documents

Publication Publication Date Title
US12133768B2 (en) Ultrasonic diagnostic apparatus, medical imaging apparatus, training device, ultrasonic image display method, and storage medium
US6558324B1 (en) System and method for strain image display
US8416301B2 (en) Strain image display systems
US8948485B2 (en) Ultrasonic diagnostic apparatus, ultrasonic image processing apparatus, ultrasonic image processing program, and ultrasonic image generation method
JP4627366B2 (ja) パケット・データ獲得を用いた超音波フロー撮像における運動の可視化のための方法および装置
CN111095428A (zh) 具有用于图像伪影识别和移除的深度学习网络的超声系统
CN101427932A (zh) 超声波诊断装置
EP3136974B1 (fr) Visualisation d'élastographie
US20050215904A1 (en) Ultrasound breathing waveform detection system and method
US20090177086A1 (en) Method and apparatus for selectively enhancing ultrasound image data
KR20150089835A (ko) 초음파 영상 표시 방법 및 이를 위한 초음파 장치
WO2005023098A2 (fr) Etablissement de moyenne de trame adapte au mouvement pour imagerie couleur doppler ultrasonore
CN106691502B (zh) 用于生成弹性图像的超声系统和方法
CN101467893A (zh) 超声波诊断装置、超声波图像处理装置及方法
JP2011120901A (ja) 超音波空間合成映像を提供する超音波システムおよび方法
US11375981B2 (en) Ultrasound color flow imaging
EP2486421B1 (fr) Imagerie ultrasonore anéchoïque
JP5473527B2 (ja) 超音波診断装置
JP5356140B2 (ja) 超音波診断装置及びその制御プログラム
JP2007518512A (ja) 心筋灌流を表示するための画像分割
US20170296146A1 (en) Ultrasonic diagnostic device and method of generating discrimination information
US20070255138A1 (en) Method and apparatus for 3D visualization of flow jets
CA2943666C (fr) Procede et appareil de demodulation adaptative pour une image ultrasonore
US9842427B2 (en) Methods and systems for visualization of flow jets
JP2013240721A (ja) 超音波診断装置

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20161202

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

DAX Request for extension of the european patent (deleted)
GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20180924

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602014038199

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1077812

Country of ref document: AT

Kind code of ref document: T

Effective date: 20190115

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20181219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190319

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190319

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1077812

Country of ref document: AT

Kind code of ref document: T

Effective date: 20181219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190320

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190419

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20190419

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602014038199

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

26N No opposition filed

Effective date: 20190920

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20190502

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190531

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190531

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20190531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190502

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190502

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190502

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20190531

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20140502

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20210510

Year of fee payment: 8

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20181219

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602014038199

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20221201