[go: up one dir, main page]

EP2873371A1 - Procédé et système permettant d'enregistrer des images intravasculaires - Google Patents

Procédé et système permettant d'enregistrer des images intravasculaires Download PDF

Info

Publication number
EP2873371A1
EP2873371A1 EP20130192749 EP13192749A EP2873371A1 EP 2873371 A1 EP2873371 A1 EP 2873371A1 EP 20130192749 EP20130192749 EP 20130192749 EP 13192749 A EP13192749 A EP 13192749A EP 2873371 A1 EP2873371 A1 EP 2873371A1
Authority
EP
European Patent Office
Prior art keywords
intravascular
images
angiography
vessel
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP20130192749
Other languages
German (de)
English (en)
Other versions
EP2873371B1 (fr
Inventor
Jean-Paul Aben
Boudewijn Verstraelen
Tristan Slots
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Pie Medical Imaging BV
Original Assignee
Pie Medical Imaging BV
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Pie Medical Imaging BV filed Critical Pie Medical Imaging BV
Priority to EP13192749.3A priority Critical patent/EP2873371B1/fr
Priority to JP2014226394A priority patent/JP6573447B2/ja
Priority to US14/537,425 priority patent/US9811939B2/en
Publication of EP2873371A1 publication Critical patent/EP2873371A1/fr
Application granted granted Critical
Publication of EP2873371B1 publication Critical patent/EP2873371B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/12—Arrangements for detecting or locating foreign bodies
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/48—Diagnostic techniques
    • A61B6/486—Diagnostic techniques involving generating temporal series of image data
    • A61B6/487—Diagnostic techniques involving generating temporal series of image data involving fluoroscopy
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/50—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications
    • A61B6/504—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment specially adapted for specific body parts; specially adapted for specific clinical applications for diagnosis of blood vessels, e.g. by angiography
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/52—Devices using data or image processing specially adapted for radiation diagnosis
    • A61B6/5211—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data
    • A61B6/5229—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image
    • A61B6/5247—Devices using data or image processing specially adapted for radiation diagnosis involving processing of medical diagnostic data combining image data of a patient, e.g. combining a functional image with an anatomical image combining images from an ionising-radiation diagnostic technique and a non-ionising radiation diagnostic technique, e.g. X-ray and ultrasound
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Clinical applications
    • A61B8/0891—Clinical applications for diagnosis of blood vessels
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/12—Diagnosis using ultrasonic, sonic or infrasonic waves in body cavities or body tracts, e.g. by using catheters
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/5215—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data
    • A61B8/5238—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image
    • A61B8/5261—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of medical diagnostic data for combining image data of patient, e.g. merging several images from different acquisition modes into one image combining images from different diagnostic modalities, e.g. ultrasound and X-ray
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/32—Determination of transform parameters for the alignment of images, i.e. image registration using correlation-based methods
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/44—Constructional features of apparatus for radiation diagnosis
    • A61B6/4429—Constructional features of apparatus for radiation diagnosis related to the mounting of source units and detector units
    • A61B6/4435—Constructional features of apparatus for radiation diagnosis related to the mounting of source units and detector units the source unit and the detector unit being coupled by a rigid structure
    • A61B6/4441—Constructional features of apparatus for radiation diagnosis related to the mounting of source units and detector units the source unit and the detector unit being coupled by a rigid structure the rigid structure being a C-arm or U-arm
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/48—Diagnostic techniques
    • A61B6/481—Diagnostic techniques involving the use of contrast agents
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/54—Control of apparatus or devices for radiation diagnosis
    • A61B6/541—Control of apparatus or devices for radiation diagnosis involving acquisition triggered by a physiological signal
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/08—Clinical applications
    • A61B8/0883—Clinical applications for diagnosis of the heart
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10116—X-ray image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10132—Ultrasound image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30101—Blood vessel; Artery; Vein; Vascular
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Definitions

  • the invention relates to the technical field of medical imaging, particularly in percutaneous interventions.
  • X-ray angiographic imaging is used to guide the intervention procedure.
  • Detailed information of the diseased vessel for example atherosclerotic plaque, is assessed by intravascular imaging modalities, such as intravascular ultrasound (IVUS) or optical coherence tomography (OCT), during the intervention procedure.
  • IVUS intravascular ultrasound
  • OCT optical coherence tomography
  • Another example of an intravascular imaging modality is intravascular fractional flow reserve (FFR) in which the pressure within the vessel is measured during pullback.
  • FFR intravascular fractional flow reserve
  • Current invention relates to all intravascular imaging devices having a controlled pullback speed.
  • Intravascular imaging is performed during pullback of an intravascular device through a vessel and produces a stack of images showing vessel cross sections.
  • US patent n. 7,729,746 discloses a method in which the co-registration between X-ray and intravascular images is performed by generation of a 3D reconstruction of the vessel based on two X-ray angiographic projections.
  • two additional X-ray fluoroscopic images are required in which the user identifies in each fluoroscopic image the tip of the intravascular transducer. This results in a 3D point in space which is used to perform the co-registration.
  • This approach goes in the right direction since the 3D reconstruction of the vessel of interest eliminates foreshortening and out-of-plane magnification errors, although the incorrect assumption is made that the centerline of the 3D model corresponds with the intravascular path in the vessel during pullback.
  • any errors in the 2D segmentation of the vessel within the X-ray angiographic images will negatively influence the co-registration since they directly affect the 3D reconstruction of the vessel and its resulting centerline.
  • Another disadvantage is that four additional X-ray images are required for this method resulting in additional exposure to the patient.
  • a further disadvantage of the prior art is that the true imaging plane within the space occupied by the intravascular electronics is not taken into consideration.
  • the invention reaches the aim with a method for co-registration of angiography and intravascular images, which intravascular images are in the form of a sequence of images obtained from an intravascular imaging device which is pulled back through a vessel, the method comprising the following steps:
  • the co-registration can be performed after the intravascular pullback or just before the intravascular pullback.
  • the latter allows real time feedback of position of intravascular image in the X-ray images during the intravascular pullback.
  • the angiography images are typically bi-dimensional X-ray images obtained with or without contrast agents. Particularly, as there's no need to reconstruct the vessel wall as well as the centerlines as in the prior art, such images can be obtained without contrast agents, i.e. they can be so-called fluoroscopic images.
  • the trajectory of the intravascular device can be, in fact, advantageously reconstructed by following the position of the wire of the catheter associated with the intravascular device starting from the tip once in the start of examination position, hence no contrast enhancement is necessary.
  • one of the at least two angiography images is preferably the same angiographic image (i.e. obtained with a contrast agent) that is usually acquired during common catheterization laboratory workflow of a percutaneous intervention procedure. This allows reducing X-ray exposure to the patient.
  • the second image obtained from a different perspective, does not require contrast agent and thus can advantageously be a fluoroscopic image.
  • the idea at the base of the invention is to perform co-registration by using the 3D reconstruction of the trajectory of the intravascular device, and not the 3D reconstruction of the vessel, obtained from two angiography images in which the start of the 3D trajectory of the intravascular device is the intravascular device itself and this will correspond with the first intravascular image.
  • an accurate match with the true position of the intravascular imaging plane with respect to its location as visible on fluoroscopy and/or angiography can be obtained by using the technical specifications of the intravascular imaging device used during the procedure.
  • the element of the device whose first position is determined within the 3D reconstruction of the trajectory to correlate the position of the device in the reconstructed trajectory during pull back, can advantageously be a marker or any position on proximal side or distal side of the device having a known distance to the imaging plane of the intravascular device.
  • co-registration can be automatically performed between angiography imaging and intravascular imaging in a very easy and effective way for immediate use during normal catheterization procedures. Furthermore, the co-registration is based on the true 3D trajectory of the intravascular device resulting in a perfect length assessment allowing accurate length measurement to support the clinician in appropriate interventional treatment.
  • a so called longitudinal image wherein the intravascular images acquired during pull back are stacked on each other along a longitudinal line, the position of each intravascular image within the vessel being identified by a marker on the line and a corresponding marker in the angiography image.
  • the degree of perpendicularity of the acquired intravascular images with reference to the vessel can be determined by calculating planes perpendicular to the reconstructed trajectory and back projecting such planes on the angiography image.
  • Such degree of perpendicularity can be evaluated, for example, by comparing the orientation of the planes perpendicular to the reconstructed trajectory with corresponding planes perpendicular to the lumen boundary of the vessel on the angiography image.
  • the orientation of the planes perpendicular to the trajectory and to the lumen boundary can be shown as markers in the form of segments on the angiography image.
  • the degree of perpendicularity can also be advantageously quantified using a colour-coded representation on the angiography image. This allows to immediately correct possible overestimated areas within the vessel.
  • the invention also relates to a computer product directly loadable into the memory of a computer and comprising software code portions for performing the method as disclosed above when the product is run on a computer.
  • the invention relates to a system for registering angiography and intravascular images of a vessel having processing means configured to perform the method as disclosed above to register intravascular images with at least one angiographic image.
  • the system comprises:
  • the system can be provided in combination with an X-ray apparatus for acquiring two or more bi-dimensional angiography images of a vessel of a patient from different perspectives.
  • the X-ray apparatus can be a dual arm X-ray apparatus to acquire biplane angiography images or a single arm X-ray apparatus to acquire single plane fluoroscopic and/or angiographic images.
  • Intravascular images can be obtained with any kind of intravascular imaging modality during the intervention procedure such as intravascular ultrasound (IVUS), optical coherence tomography (OCT) or fractional flow reserve (FFR) which can have a controlled pullback speed. FFR will result in a number of values representing the local pressure at each position during the pullback.
  • IVUS intravascular ultrasound
  • OCT optical coherence tomography
  • FFR fractional flow reserve
  • the co-registration can be performed after the intravascular pullback or just before the intravascular pullback.
  • the latter allows real time feedback of position of intravascular image in the X-ray images during the intravascular pullback.
  • the true pullback length during intravascular imaging is required from the angiography images. Since angiography produces 2D images, length measurements are prone to errors due to foreshortening and out-of-plane magnification effects.
  • the true pullback length of intravascular device is assessed by a 3D reconstruction of the catheter wire of the intravascular device which is visible in two angiography images. By using the 3D reconstruction of the intravascular catheter-path instead of the 3D centerline from the 3D reconstruction of the vascular lumen, co-registration errors are avoided in tortuous vessel which is common anatomy in coronary vessels.
  • Figure 1 is a schematic representation of an angiography image of a vessel with an inserted intravascular device: due to tortuousity, the centerline of the vessel follows a different route with reference to the position of the catheter wire.
  • Step 1 Acquire two angiography images
  • the angiography images are typically bi-dimensional X-ray images obtained, with or without contrast agents, when the intravascular device is in its start of examination position, i.e. it has been placed in the vessel of interest to perform intravascular imaging during pull back of the same.
  • Any image device capable of providing 2D images can be used for the purpose.
  • a biplane or single plane angiographic system can be used such as those manufactured, for example, by Siemens (Artis zee Biplane) or Philips (Allura Xper FD).
  • Siemens Article zee Biplane
  • Philips Allura Xper FD
  • the second X-ray image is a fluoroscopic image of the intravascular device inserted in the vessel to avoid the use of additional contrast agent.
  • an additional fluoroscopic X-ray image is not necessary and the biplane images are directly used to perform the 3D reconstruction of the catheter path.
  • Step 2 Indicate catheter path in the two angiography images
  • the catheter path is indicated starting at the intravascular transducer.
  • the corresponding frame of the second X-ray image can be automatically selected by matching the corresponding frame based on the ECG signal within the first X-ray image. This is achieved by detection of the length of the cardiac cycle, for example as disclosed in " A novel method for detecting R-peaks in electrocardiogram (ECG) signal” M. Sabarimalai, K.P. Soman, Biomedical Signal Processing and Control (2011 ), doi:10.1016/j.bspc.2011.03.004.
  • a percentage is calculated within the cardiac cycle corresponding to the frame of the first X-ray image.
  • a frame is selected with the same percentage within the cardiac cycle based on ECG signal belonging to the second X-ray image as shown in Fig. 4 .
  • the indication of the intravascular catheter in the X-ray images can be performed quickly, for example, by using the parametric Catmull-Rom 2D spline fitting through several user indicated points.
  • the Catmull-Rom 2D spline is disclosed, for example, in " A Class of Local Interpolating Splines", E. Catmull and A. Rom, Computer Aided Geometrical Design, 1974, pages 317-326 . See also the article by Robert Dunlop that can be found on the Internet at http://www.mvps.org/directx/articles/catmull. An automatic path recognition algorithm can be equally employed.
  • Fig. 5 and 6 the catheter wire of the intravascular device is outlined on both 2D angiography images.
  • Step 3 3D reconstruction of trajectory of the intravascular transducer
  • the 3D trajectory (hereinafter referred also as path) of the intravascular device is reconstructed by a 3D reconstruction of the intravascular catheter wire, and starting at the tip of the intravascular device.
  • the 3D reconstruction of the path can be performed by using epipolar 3D reconstruction techniques, for example as disclosed in "A novel dedicated 3-dimensional quantitative coronary analysis methodology for bifurcation lesions", Yoshinobu Onuma, Chrysafios Girasis, Jean-Paul Aben, Giovanna Sarno, Nicolo Piazza, Coen Lokkerbol, Marie-Angel Morel, Patrick W. Serruys, Eurolntervention 2011; 6:1-00 .
  • an addition point within one of the X-ray images is required to define the location of the intravascular device within the 3D reconstruction of the intravascular catheter wire.
  • Step 4 Incorporate technical data of intravascular device
  • This step is completely optional and aims at increasing accuracy.
  • Intravascular imaging is typically performed with a catheter having on the tip an imaging device, such as a transducer in case of IVUS, which absorbers X-ray radiation and therefore is visible both on fluoroscopy and angiography.
  • an imaging device such as a transducer in case of IVUS, which absorbers X-ray radiation and therefore is visible both on fluoroscopy and angiography.
  • the device is normally manufactured as small as possible, it still uses physical space. For example, for an IVUS device this can be up to 10 mm. Within this space, the true imaging plane is somewhere located. By using the technical specification provided by the intravascular device manufacturer, the position of such plane can be determined and thus used to increase the accuracy of the co-registration.
  • the start of the 3D model is repositioned taking into account the distance of the imaging plane.
  • the user identifies the radiopaque marker, for example by the second node which is used for fitting the Catmull-row spline, or by indicating this radiopaque marker by right mouse click instead of left mouse click in which the remaining nodes are positioned.
  • An automatic marker detection algorithm can be equally employed as, for instance, as disclosed in US 8,411,927 . With this information, the start of the 3D model can be repositioned by using the known distance from the technical description of the intravascular device.
  • An example of an IVUS device which is accompanied by such description is the one manufactured by Volcano Corporation with the commercial name Eagle Eye@ Platinum RX Digital IVUS Catheter.
  • Step 5 Perform co-registration of X-ray imaging and intravascular imaging
  • the co-registration can be performed after the intravascular pullback or just before the intravascular pullback.
  • the latter allows real time feedback of position of intravascular image in the X-ray images during the intravascular pullback.
  • Step 6 Perform intravascular pullback
  • the pullback method consists of initially positioning the intravascular transducer, or catheter tip, distal in a vessel of interest and pull the catheter tip with a controlled speed to the proximal part of the vessel of interest. During this pullback the intravascular transducer acquires intravascular images.
  • each intravascular image To match the spatial position of each intravascular image with respect to the X-ray image information on the pull back speed during its path from distal to proximal is required.
  • the registration is performed by computing the frame speed and matching each intravascular frame to the length of the 3D intravascular catheter path.
  • the motion of the intravascular device can be recorded to obtain information of the pullback speed during its path from distal to proximal. This can be accomplished, for example, by measuring the longitudinal motion of the intravascular catheter by means of a motion measurement system. Registration between X-ray and intravascular imaging can be performed by matching each intravascular frame, by using the recorded longitudinal motion i.e. the inter-frame distance, to the length of the 3D intravascular catheter path starting from the distal position of the 3D intravascular path.
  • Figure 9 shows an example of co-registration between IVUS and angiography.
  • Bottom figure illustrates a so called longitudinal image. This is a view in which all the IVUS frames are stacked on each other and a cross section is made longitudinal. Corresponding position of IVUS frame in longitudinal view and angiographic view are visualized by markers. Arrow identifies the start of first IVUS frame and its position in the angiographic image. The same can also be performed on the fluoroscopic image.
  • the location of the intravascular transducer can be indicated real time on the X-ray image.
  • the physician can accurately define length measurements in both the X-ray angiographic image as well as in a longitudinal view of the intravascular image data. Furthermore, detailed vessel information from the intravascular image is accurately correlated its location in the X-ray angiographic image.
  • the invention also provides information on the perpendicularity of the acquired intravascular images with respect to the vessel. Since the 3D trajectory of the intravascular transducer is known, a plane perpendicular at a position within this 3D trajectory can be back-projected on the X-ray angiographic image allowing the physician to view the perpendicularity of the intravascular image with respect to the vessel. The severity of mismatch in the 2D X-ray imaging viewing direction of perpendicularity can be visualized and/or quantified, for instance, by means of colour coding the back-projected line. In case the intravascular image is not perpendicular to the vessel, the cross sectional area of the vessel in IVUS and/or OCT images will be overestimated.
  • Figure 10 shows an example of a visualization of a non perpendicular intravascular imaging plane
  • Figure 11 shows an example of a visualization of a non perpendicular intravascular imaging plane in case of mismatch in the 2D X-ray imaging viewing directions.
  • the invention has been mainly disclosed with reference to co-registration of intravascular images and X-ray angiographic images.
  • This teaching can be equally extended to cover co-registration of images made with any imaging or measuring device that travels through any tubular object with images taken from outside the object with any type of imaging modality including X-ray, MRI, SPECT, Ultrasound or the like.
  • any imaging or measuring device that travels through any tubular object with images taken from outside the object with any type of imaging modality including X-ray, MRI, SPECT, Ultrasound or the like.
  • the same apparatus can be used to reconstruct both types of images thus providing a very compact system.
  • the same system could also provide the actuating commands to perform pullback at a known speed starting at a known instant of time thus increasing manoeuvrability and repeatability.
  • tubular objects may be the oesophagus, the intestine or the bronchitis in the medical field, and any kind of pipe in general in the field of non-destructive testing.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Physics & Mathematics (AREA)
  • Radiology & Medical Imaging (AREA)
  • General Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Surgery (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Pathology (AREA)
  • Optics & Photonics (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Vascular Medicine (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Dentistry (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Computer Graphics (AREA)
  • Apparatus For Radiation Diagnosis (AREA)
  • Endoscopes (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Quality & Reliability (AREA)
EP13192749.3A 2013-11-13 2013-11-13 Procédé et système permettant d'enregistrer des images intravasculaires Active EP2873371B1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP13192749.3A EP2873371B1 (fr) 2013-11-13 2013-11-13 Procédé et système permettant d'enregistrer des images intravasculaires
JP2014226394A JP6573447B2 (ja) 2013-11-13 2014-11-06 血管内画像を登録するためのシステム
US14/537,425 US9811939B2 (en) 2013-11-13 2014-11-10 Method and system for registering intravascular images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP13192749.3A EP2873371B1 (fr) 2013-11-13 2013-11-13 Procédé et système permettant d'enregistrer des images intravasculaires

Publications (2)

Publication Number Publication Date
EP2873371A1 true EP2873371A1 (fr) 2015-05-20
EP2873371B1 EP2873371B1 (fr) 2022-12-21

Family

ID=49679305

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13192749.3A Active EP2873371B1 (fr) 2013-11-13 2013-11-13 Procédé et système permettant d'enregistrer des images intravasculaires

Country Status (3)

Country Link
US (1) US9811939B2 (fr)
EP (1) EP2873371B1 (fr)
JP (1) JP6573447B2 (fr)

Cited By (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3199107A1 (fr) * 2016-01-26 2017-08-02 Terumo Kabushiki Kaisha Afficheur d'image, procédé de commande associé et procédé de détection de marqueur radio-opaque
CN107126227A (zh) * 2016-02-29 2017-09-05 通用电气公司 X射线/血管内成像共置方法和系统
EP3884868A1 (fr) * 2020-03-26 2021-09-29 Pie Medical Imaging BV Procédé et système pour enregistrer des données intra-objets avec des données extra-objets
EP4035600A1 (fr) * 2021-02-01 2022-08-03 Koninklijke Philips N.V. Enregistrement d'images intraluminales et extraluminales
US11660059B2 (en) 2014-04-15 2023-05-30 4DMedical Limited Apparatus and method of imaging
US11707196B2 (en) 2012-10-24 2023-07-25 Cathworks Ltd. Automated measurement system and method for coronary artery disease scoring
US11723617B2 (en) 2016-02-03 2023-08-15 4DMedical Limited Method and system for imaging
US11816837B2 (en) 2013-10-24 2023-11-14 Cathworks Ltd. Vascular characteristic determination with correspondence modeling of a vascular tree
US11937963B2 (en) 2016-05-16 2024-03-26 Cathworks Ltd. Vascular selection from images
US12039685B2 (en) 2019-09-23 2024-07-16 Cathworks Ltd. Methods, apparatus, and system for synchronization between a three-dimensional vascular model and an imaging device
US12079994B2 (en) 2019-04-01 2024-09-03 Cathworks Ltd. Methods and apparatus for angiographic image selection
US12102414B2 (en) 2017-02-28 2024-10-01 4DMedical Limited Method of scanning and assessing lung and vascular health
US12138027B2 (en) 2016-05-16 2024-11-12 Cath Works Ltd. System for vascular assessment
US12217872B2 (en) 2012-10-24 2025-02-04 Cathworks Ltd. Diagnostically useful results in real time
US12315076B1 (en) 2021-09-22 2025-05-27 Cathworks Ltd. Four-dimensional motion analysis of a patient's coronary arteries and myocardial wall
US12387325B2 (en) 2022-02-10 2025-08-12 Cath Works Ltd. System and method for machine-learning based sensor analysis and vascular tree segmentation
US12446965B2 (en) 2023-08-09 2025-10-21 Cathworks Ltd. Enhanced user interface and crosstalk analysis for vascular index measurement

Families Citing this family (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104582572B (zh) * 2012-08-16 2018-04-13 东芝医疗系统株式会社 图像处理装置、医用图像诊断装置以及血压监视器
US9208559B1 (en) * 2014-07-25 2015-12-08 Siemens Aktiengesellschaft Method and apparatus for gastric artery chemical embolization
KR101797042B1 (ko) * 2015-05-15 2017-11-13 삼성전자주식회사 의료 영상 합성 방법 및 장치
CN107203988B (zh) * 2016-03-18 2019-07-19 北京大学 一种由二维x光图像重建三维体图像的方法及其应用
WO2017221159A1 (fr) * 2016-06-22 2017-12-28 Sync-Rx, Ltd. Mise à jour d'une indication d'emplacement de lumière
DE102016215971A1 (de) * 2016-08-25 2018-03-01 Siemens Healthcare Gmbh Segmentierung einer Angiographie unter Verwendung einer bestehenden dreidimensionalen Rekonstruktion
US10842589B2 (en) * 2017-03-21 2020-11-24 Canon U.S.A., Inc. Method for displaying an anatomical image of a coronary artery on a graphical user interface
JP7054411B2 (ja) 2017-07-26 2022-04-13 キヤノン ユーエスエイ，インコーポレイテッド 血管造影画像を用いて心臓運動を評価するための方法
EP3453326A1 (fr) * 2017-09-08 2019-03-13 Koninklijke Philips N.V. Enregistrement et comparaison de courbes de retrait intracoronaire simulées et mesurées
US10709399B2 (en) * 2017-11-30 2020-07-14 Shenzhen Keya Medical Technology Corporation Methods and devices for performing three-dimensional blood vessel reconstruction using angiographic images
CN108245249A (zh) * 2017-12-26 2018-07-06 成都真实维度科技有限公司 一种基于虚拟成像技术的血管支架用球囊选择方法
EP3628235A1 (fr) 2018-09-26 2020-04-01 Koninklijke Philips N.V. Imagerie intravasculaire par ultrasons
US11918423B2 (en) 2018-10-30 2024-03-05 Corindus, Inc. System and method for navigating a device through a path to a target location
WO2020159984A1 (fr) 2019-01-30 2020-08-06 Canon U.S.A., Inc. Appareils, systèmes, procédés et supports d'informations pour la réalisation d'un co-enregistrement
US10861157B2 (en) 2019-04-04 2020-12-08 Medtronic Vascular, Inc. System and methods for determining modified fractional flow reserve values
US11357573B2 (en) * 2019-04-25 2022-06-14 International Business Machines Corporation Optimum treatment planning during coronary intervention by simultaneous simulation of a continuum of outcomes
US12211208B2 (en) * 2019-09-09 2025-01-28 Medhub Ltd Automated analysis of image data to determine fractional flow reserve
US12109056B2 (en) 2019-09-17 2024-10-08 Canon U.S.A., Inc. Constructing or reconstructing 3D structure(s)
US11580694B1 (en) * 2019-10-28 2023-02-14 Massachusetts Institute Of Technology Passive wide-area three-dimensional imaging
CN113679402B (zh) * 2020-05-18 2024-05-24 西门子(深圳)磁共振有限公司 介入治疗中的图像呈现方法及系统、成像系统和存储介质
JP2021194268A (ja) * 2020-06-15 2021-12-27 パナソニックｉ−ＰＲＯセンシングソリューションズ株式会社 血管観察システムおよび血管観察方法
EP4042924A1 (fr) * 2021-02-12 2022-08-17 Koninklijke Philips N.V. Estimation de la position d'un dispositif d'intervention
WO2022202201A1 (fr) * 2021-03-25 2022-09-29 テルモ株式会社 Dispositif de traitement d'images, système de traitement d'images, procédé d'affichage d'image et programme de traitement d'images
CN113616160B (zh) * 2021-09-14 2024-02-06 苏州博动戎影医疗科技有限公司 基于多模态医学影像的ffr确定方法、装置、设备及介质
WO2023100838A1 (fr) * 2021-11-30 2023-06-08 テルモ株式会社 Programme informatique, dispositif de traitement d'informations, procédé de traitement d'informations et procédé de génération de modèle d'apprentissage
CN115546174B (zh) * 2022-10-20 2023-09-08 数坤(北京)网络科技股份有限公司 图像处理方法、装置、计算设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7729746B2 (en) 2005-11-04 2010-06-01 Siemens Aktiengesellschaft Three-dimensional co-registration between intravascular and angiographic data
US20110034801A1 (en) * 2009-08-06 2011-02-10 Siemens Medical Solutions Usa, Inc. System for Processing Angiography and Ultrasound Image Data
US20120059253A1 (en) 2010-06-29 2012-03-08 Siemens Aktiengesellschaft Method and System for Image Based Device Tracking for Co-registration of Angiography and Intravascular Ultrasound Images
US8411927B2 (en) 2009-10-29 2013-04-02 Siemens Aktiengesellschaft Marker detection in X-ray images

Family Cites Families (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060036167A1 (en) * 2004-07-03 2006-02-16 Shina Systems Ltd. Vascular image processing
EP2712553A3 (fr) * 2005-01-11 2014-09-17 Volcano Corporation Co-enregistrement d'images vasculaires
WO2007002685A2 (fr) * 2005-06-24 2007-01-04 Volcano Corporation Co-registration de donnees d'images graphiques representant des elements vasculaires tridimensionnels
US7940974B2 (en) * 2006-11-21 2011-05-10 General Electric Company Method and system for adjusting 3D CT vessel segmentation
BRPI0719032A8 (pt) * 2006-11-22 2015-10-13 Koninklijke Philips Electronics Nv Sistemas, aparelho de formação de imagem de raio x e método para estimar uma posição em uma imagem de projeção de raio x correspondendo a uma posição de sonda projetada de uma sonda intravascular na hora de adquirir dados da sonda intravascular
US8073221B2 (en) * 2008-05-12 2011-12-06 Markus Kukuk System for three-dimensional medical instrument navigation
US11064903B2 (en) * 2008-11-18 2021-07-20 Sync-Rx, Ltd Apparatus and methods for mapping a sequence of images to a roadmap image
US9101286B2 (en) * 2008-11-18 2015-08-11 Sync-Rx, Ltd. Apparatus and methods for determining a dimension of a portion of a stack of endoluminal data points
EP2348982B1 (fr) * 2008-12-03 2020-03-25 St. Jude Medical, Atrial Fibrillation Division, Inc. Système pour déterminer la position de la pointe d'un cathéter médical à l'intérieur du corps d'un patient
US9195905B2 (en) * 2010-03-10 2015-11-24 Siemens Aktiengesellschaft Method and system for graph based interactive detection of curve structures in 2D fluoroscopy
FR2960332B1 (fr) * 2010-05-21 2013-07-05 Gen Electric Procede de traitement d'images radiologiques pour determiner une position 3d d'une aiguille.
US8892186B2 (en) * 2010-09-20 2014-11-18 Siemens Aktiengesellschaft Method and system for detection and tracking of coronary sinus catheter electrodes in fluoroscopic images
US9220467B2 (en) * 2011-09-19 2015-12-29 Siemens Aktiengesellschaft Method and system for tracking catheters in 2D X-ray fluoroscopy using a graphics processing unit
US9002436B2 (en) * 2011-09-19 2015-04-07 Siemens Aktiengesellschaft Method and system for ablation catheter and circumferential mapping catheter tracking in fluoroscopic images
US9292918B2 (en) * 2012-10-05 2016-03-22 Volcano Corporation Methods and systems for transforming luminal images
US9286673B2 (en) * 2012-10-05 2016-03-15 Volcano Corporation Systems for correcting distortions in a medical image and methods of use thereof
JP2016507304A (ja) * 2013-02-11 2016-03-10 アンジオメトリックス コーポレーション 物体を検出及び追跡するとともに重ね合わせるシステム
US9351698B2 (en) * 2013-03-12 2016-05-31 Lightlab Imaging, Inc. Vascular data processing and image registration systems, methods, and apparatuses
US20140275996A1 (en) * 2013-03-12 2014-09-18 Volcano Corporation Systems and methods for constructing an image of a body structure

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7729746B2 (en) 2005-11-04 2010-06-01 Siemens Aktiengesellschaft Three-dimensional co-registration between intravascular and angiographic data
US20110034801A1 (en) * 2009-08-06 2011-02-10 Siemens Medical Solutions Usa, Inc. System for Processing Angiography and Ultrasound Image Data
US8411927B2 (en) 2009-10-29 2013-04-02 Siemens Aktiengesellschaft Marker detection in X-ray images
US20120059253A1 (en) 2010-06-29 2012-03-08 Siemens Aktiengesellschaft Method and System for Image Based Device Tracking for Co-registration of Angiography and Intravascular Ultrasound Images

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
E. CATMULL; A. ROM: "A Class of Local Interpolating Splines", COMPUTER AIDED GEOMETRICAL DESIGN, 1974, pages 317 - 326
SHENGXIAN TU ET AL: "Fusion of 3D QCA and IVUS/OCT", THE INTERNATIONAL JOURNAL OF CARDIOVASCULAR IMAGING, vol. 27, no. 2, 2 February 2011 (2011-02-02), pages 197 - 207, XP055091651, ISSN: 1569-5794, DOI: 10.1007/s10554-011-9809-2 *
YOSHINOBU ONUMA; CHRYSAFIOS GIRASIS; JEAN-PAUL ABEN; GIOVANNA SARNO; NICOLO PIAZZA; COEN LOKKERBOL; MARIE-ANGEL MOREL; PATRICK W.: "A novel dedicated 3- dimensional quantitative coronary analysis methodology for bifurcation lesions", EUROLNTERVENTION, vol. 6, 2011, pages 1 - 00

Cited By (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11707196B2 (en) 2012-10-24 2023-07-25 Cathworks Ltd. Automated measurement system and method for coronary artery disease scoring
US12217872B2 (en) 2012-10-24 2025-02-04 Cathworks Ltd. Diagnostically useful results in real time
US12354755B2 (en) 2012-10-24 2025-07-08 Cathworks Ltd Creating a vascular tree model
US12343119B2 (en) 2012-10-24 2025-07-01 Cathworks Ltd. Automated measurement system and method for coronary artery disease scoring
US12236600B2 (en) 2013-10-24 2025-02-25 CathWorks, LTD. Vascular characteristic determination based on multiple images identifying a common vascular segment with correspondence modeling of a vascular tree
US11816837B2 (en) 2013-10-24 2023-11-14 Cathworks Ltd. Vascular characteristic determination with correspondence modeling of a vascular tree
US11660059B2 (en) 2014-04-15 2023-05-30 4DMedical Limited Apparatus and method of imaging
EP3199107A1 (fr) * 2016-01-26 2017-08-02 Terumo Kabushiki Kaisha Afficheur d'image, procédé de commande associé et procédé de détection de marqueur radio-opaque
US10733753B2 (en) 2016-01-26 2020-08-04 Terumo Kabushiki Kaisha Image display device and its control method, x-ray opaque marker detection method
US11723617B2 (en) 2016-02-03 2023-08-15 4DMedical Limited Method and system for imaging
CN107126227A (zh) * 2016-02-29 2017-09-05 通用电气公司 X射线/血管内成像共置方法和系统
US12408885B2 (en) 2016-05-16 2025-09-09 Cathworks Ltd. Vascular selection from images
US11937963B2 (en) 2016-05-16 2024-03-26 Cathworks Ltd. Vascular selection from images
US12138027B2 (en) 2016-05-16 2024-11-12 Cath Works Ltd. System for vascular assessment
US12102414B2 (en) 2017-02-28 2024-10-01 4DMedical Limited Method of scanning and assessing lung and vascular health
US12079994B2 (en) 2019-04-01 2024-09-03 Cathworks Ltd. Methods and apparatus for angiographic image selection
US12039685B2 (en) 2019-09-23 2024-07-16 Cathworks Ltd. Methods, apparatus, and system for synchronization between a three-dimensional vascular model and an imaging device
US11911203B2 (en) 2020-03-26 2024-02-27 Pie Medical Imaging B.V. Method and system for registering intra-object data with extra-object data
EP3884868A1 (fr) * 2020-03-26 2021-09-29 Pie Medical Imaging BV Procédé et système pour enregistrer des données intra-objets avec des données extra-objets
WO2022161790A1 (fr) * 2021-02-01 2022-08-04 Koninklijke Philips N.V. Enregistrement d'images intracavitaires et extracavitaires
EP4035600A1 (fr) * 2021-02-01 2022-08-03 Koninklijke Philips N.V. Enregistrement d'images intraluminales et extraluminales
US12315076B1 (en) 2021-09-22 2025-05-27 Cathworks Ltd. Four-dimensional motion analysis of a patient's coronary arteries and myocardial wall
US12387325B2 (en) 2022-02-10 2025-08-12 Cath Works Ltd. System and method for machine-learning based sensor analysis and vascular tree segmentation
US12423813B2 (en) 2022-02-10 2025-09-23 Cathworks Ltd. System and method for machine-learning based sensor analysis and vascular tree segmentation
US12446965B2 (en) 2023-08-09 2025-10-21 Cathworks Ltd. Enhanced user interface and crosstalk analysis for vascular index measurement

Also Published As

Publication number Publication date
US20150131886A1 (en) 2015-05-14
EP2873371B1 (fr) 2022-12-21
JP6573447B2 (ja) 2019-09-11
JP2015109968A (ja) 2015-06-18
US9811939B2 (en) 2017-11-07

Similar Documents

Publication Publication Date Title
EP2873371B1 (fr) Procédé et système permettant d'enregistrer des images intravasculaires
JP5394930B2 (ja) Ｘ線の経脈管的に収集されたデータとの結合
JP6581598B2 (ja) カテーテルの特定の位置を決定するための装置
JP7717870B2 (ja) プロセッサ装置の作動方法
US10687777B2 (en) Vascular data processing and image registration systems, methods, and apparatuses
AU2013387679B2 (en) Vascular data processing and image registration systems, methods, and apparatuses
CN102458554B (zh) 介入期间的设备尺寸测定支持
US20190076196A1 (en) Vessel geometry and additional boundary conditions for hemodynamic ffr/ifr simulations from intravascular imaging
JP2018192287A (ja) プロセッサ装置の作動方法
JP6835959B2 (ja) 管状構造内の器具の位置を決定するための位置決定装置
WO2015071343A1 (fr) Détection d'angle de rotation d'un dispositif d'intervention
JP2024510575A (ja) 血管画像のコレジストレーションのためのシステムおよび方法
CN116916828A (zh) 管腔内和管腔外图像配准
JP2025502799A (ja) 血管画像コレジストレーションのためのシステム及び方法
EP3628238A1 (fr) Correction de torsion pour images intravasculaires
CN118714977A (zh) 用于血管图像共配准的系统和方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20131113

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

R17P Request for examination filed (corrected)

Effective date: 20151026

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: EXAMINATION IS IN PROGRESS

17Q First examination report despatched

Effective date: 20191127

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

RIC1 Information provided on ipc code assigned before grant

Ipc: G06T 7/32 20170101ALI20220622BHEP

Ipc: G06T 7/00 20060101ALI20220622BHEP

Ipc: A61B 8/12 20060101ALI20220622BHEP

Ipc: A61B 8/08 20060101ALI20220622BHEP

Ipc: A61B 6/00 20060101ALI20220622BHEP

Ipc: A61B 6/12 20060101AFI20220622BHEP

INTG Intention to grant announced

Effective date: 20220712

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013083097

Country of ref document: DE

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 1538550

Country of ref document: AT

Kind code of ref document: T

Effective date: 20230115

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: FP

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG9D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230321

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1538550

Country of ref document: AT

Kind code of ref document: T

Effective date: 20221221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230322

P01 Opt-out of the competence of the unified patent court (upc) registered

Effective date: 20230309

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230421

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20230421

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013083097

Country of ref document: DE

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

26N No opposition filed

Effective date: 20230922

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231130

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20231113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231113

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231130

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20231130

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231113

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231113

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20231130

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20241029

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20241023

Year of fee payment: 12

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20131113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20131113

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20221221

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: NL

Payment date: 20251112

Year of fee payment: 13