US20160180519A1 - Seam elimination and motion compensation in imaging data - Google Patents
Seam elimination and motion compensation in imaging data Download PDFInfo
- Publication number
- US20160180519A1 US20160180519A1 US14/964,315 US201514964315A US2016180519A1 US 20160180519 A1 US20160180519 A1 US 20160180519A1 US 201514964315 A US201514964315 A US 201514964315A US 2016180519 A1 US2016180519 A1 US 2016180519A1
- Authority
- US
- United States
- Prior art keywords
- signal
- point
- pixel
- probability
- weighted average
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5269—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving detection or reduction of artifacts
- A61B8/5276—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving detection or reduction of artifacts due to motion
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/13—Tomography
- A61B8/14—Echo-tomography
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0062—Arrangements for scanning
- A61B5/0066—Optical coherence imaging
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0082—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes
- A61B5/0084—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence adapted for particular medical purposes for introduction into the body, e.g. by catheters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7203—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal
- A61B5/7207—Signal processing specially adapted for physiological signals or for diagnostic purposes for noise prevention, reduction or removal of noise induced by motion artifacts
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/08—Clinical applications
- A61B8/0891—Clinical applications for diagnosis of blood vessels
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/12—Diagnosis using ultrasonic, sonic or infrasonic waves in body cavities or body tracts, e.g. by using catheters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
- A61B8/4444—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device related to the probe
- A61B8/445—Details of catheter construction
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
- A61B8/4483—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device characterised by features of the ultrasound transducer
- A61B8/4488—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device characterised by features of the ultrasound transducer the transducer being a phased array
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
- A61B8/4483—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device characterised by features of the ultrasound transducer
- A61B8/4494—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device characterised by features of the ultrasound transducer characterised by the arrangement of the transducer elements
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5207—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving processing of raw data to produce diagnostic data, e.g. for generating an image
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/52—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/5269—Devices using data or image processing specially adapted for diagnosis using ultrasonic, sonic or infrasonic waves involving detection or reduction of artifacts
-
- G06T5/002—
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0033—Features or image-related aspects of imaging apparatus, e.g. for MRI, optical tomography or impedance tomography apparatus; Arrangements of imaging apparatus in a room
- A61B5/0037—Performing a preliminary scan, e.g. a prescan for identifying a region of interest
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/06—Measuring blood flow
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
- A61B8/44—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device
- A61B8/4416—Constructional features of the ultrasonic, sonic or infrasonic diagnostic device related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10101—Optical tomography; Optical coherence tomography [OCT]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10132—Ultrasound image
Definitions
- the medical sensing system includes an intravascular portion having an elongate member with one or more sensors such as ultrasound transducers and/or optical coherence tomography sensors oriented to collect data radially as the elongate member is drawn longitudinally through a vessel.
- the medical sensing system assembles the radial data into cross-sectional slices of the vessel. Due to the longitudinal motion, the radial data is actually a helical representation of the vessel and according, the medical sensing system flattens the helical data to produce a cross-sectional view.
- the system is suitable for use in a variety of applications including intravascular ultrasound.
- some embodiments of the present disclosure provide an IVUS medical sensing system particularly suited to imaging a human blood vessel.
- IVUS imaging is widely used in interventional cardiology as a diagnostic tool for assessing a diseased vessel, such as an artery, within the human body to determine the need for treatment, to guide the intervention, and/or to assess its effectiveness.
- IVUS devices There are two general types of IVUS devices in use today: rotational and solid-state (also known as synthetic aperture phased array).
- rotational and solid-state also known as synthetic aperture phased array.
- a typical rotational IVUS device a single ultrasound transducer element is located at the tip of a flexible driveshaft that spins inside a plastic sheath inserted into the vessel of interest.
- the transducer element is oriented such that the ultrasound beam propagates generally perpendicular to the longitudinal axis of the device.
- the transducer element In forward-looking rotational devices, the transducer element is pitched towards the distal tip so that the ultrasound beam propagates more towards the tip (in some devices, being emitted parallel to the longitudinal centerline).
- the fluid-filled sheath protects the vessel tissue from the spinning transducer and driveshaft while permitting ultrasound signals to propagate from the transducer into the tissue and back.
- the transducer As the driveshaft rotates, the transducer is periodically excited with a high voltage pulse to emit a short burst of ultrasound. The same transducer then listens for the returning echoes reflected from various tissue structures.
- the IVUS medical sensing system assembles a two dimensional display of the tissue, vessel, heart structure, etc. from a sequence of pulse/acquisition cycles occurring during a single revolution of the transducer. In order to image a length of a vessel, the transducer element is drawn through the vessel as it spins.
- solid-state IVUS devices utilize a scanner assembly that includes an array of ultrasound transducers connected to a set of transducer controllers.
- the transducers are distributed around the circumference of the device.
- the transducers are a linear array arranged at the distal tip and pitched so that the ultrasound beam propagates closer to parallel with the longitudinal centerline.
- the transducer controllers select transducer sets for transmitting an ultrasound pulse and for receiving the echo signal. By stepping through a sequence of transmit-receive sets, the solid-state IVUS system can synthesize the effect of a mechanically scanned transducer element but without moving parts.
- the transducer array can be placed in direct contact with the blood and vessel tissue with minimal risk of vessel trauma. Furthermore, because there is no rotating element, the interface is simplified.
- the solid-state scanner can be wired directly to the medical sensing system with a simple electrical cable and a standard detachable electrical connector. While the transducers of the scanner assembly do not spin, operation is similar to that of a rotational system in that, in order to image a length of a vessel, the scanner assembly is drawn through the vessel while stepping through the transmit-receive sets to produce a series of radial scans.
- Rotational and solid-state state IVUS are merely some examples of imaging modalities that sample a narrow region of the environment and assemble a two- or three-dimensional image from the results.
- Other examples include optical coherence tomography (OCT).
- OCT optical coherence tomography
- One of the key challenges in this type of imaging is inferring information in areas of the environment that are not directly imaged.
- the radial data is only coplanar if there is no movement during a rotation.
- each radially imaged region will have a different longitudinal orientation. In other words, the scanner assembly images a helical region rather than a set of flat circular regions.
- a discontinuity or seam may occur between the first and last radial scans of the image.
- the seam also indicates that the scanning data is being incorrectly displayed.
- improper flattening may hide the curvature of the vessel. Effects due to improper flattening become more pronounced as the longitudinal motion becomes greater relative to the rotational speed. For example, OCT may have a much higher relative pullback speed than IVUS, resulting in more longitudinal distortion and may experience an even greater benefit from proper correction.
- a more accurate technique for reconstructing a planar cross-section from non-planar data would produce a more accurate image that is closer to what a clinician would expect and may thus improve diagnostic accuracy.
- it may reduce or eliminate distortions and discontinuities (e.g., seams).
- Embodiments of the present disclosure provide an imaging component that assembles a cross-sectional representation from a helical set of imaging data and removes artifacts caused by conventional flattening processes.
- the imaging component may be used in applications such as an intravascular ultrasound medical sensing system.
- a medical sensing system comprising a de-seaming engine operable to receive a first signal at a first point and a second signal at a second point.
- the first point and the second point may have similar relative positions while being in different frames.
- the first and second points may have the same radial and azimuthal orientation but occupy different positions longitudinally.
- the de-seaming engine determines an intensity at a location between the first and second points using the first signal and the second signal.
- the de-seaming engine includes an averaging element operable to determine a weighted average of the first signal and the second signal.
- the weighted average is used to determine an interpolated signal value at the location based on the signal values measured at the first and second points.
- the de-seaming engine also includes a first comparator operable to determine a first probability based on the first signal and the weighted average and a second comparator operable to determine a second probability based on the second signal and the weighted average.
- the first and second probabilities are provided to a third comparator of the de-seaming engine, which is operable to select one of the first and second signal intensities as an intensity at the location based on the first and second probabilities and to provide the selected intensity as a pixel intensity for forming an image.
- a method in some embodiments, includes receiving a first signal value associated with a first point located a first longitudinal distance from a pixel and a second signal value associated with a second point located a second longitudinal distance from the pixel.
- a first probability associated with the first point is determined based on the first longitudinal distance
- a second probability associated with the second point is determined based on the second longitudinal distance.
- a signal value for the pixel is selected based on the first probability and the second probability, and the selected signal value for the pixel is provided for forming an image.
- the method also includes determining a first weighting factor for the first point based on the first longitudinal distance and determining a second weighting factor for the second point based on the second longitudinal distance.
- a weighted average is determined based on the first signal value, the first weighting factor, the second signal value, and the second weighting factor.
- the determining of the first probability includes comparing the first signal value to the weighted average, and the determining of the second probability includes comparing the second signal value to the weighted average.
- a method of diagnostic imaging includes identifying a pixel located between a first point and a second point in a longitudinal direction.
- a first imaging signal for the first point and a second imaging signal for the second point are received, and a weighted average of the first imaging signal and the second imaging signal is determined.
- the first signal is compared to the weighted average to determine a first probability
- the second signal is compared to the weighted average to determine a second probability.
- a value for the pixel is selected based on the first probability and the second probability. The selected value is provided for use in forming a diagnostic image.
- the method also includes determining a first weighting factor for the first imaging signal based on a longitudinal distance between the first point and the pixel and determining a second weighting factor for the second imaging signal based on a longitudinal distance between the second point and the pixel.
- the determining of the weighted average includes applying the first weighting factor to the first imaging signal and applying the second weighting factor to the second imaging signal.
- the first weighting factor is determined based on a count of radial measurements between the first point and the pixel
- the second weighting factor is determined based on a count of radial measurements between the second point and the pixel.
- the techniques of the present disclosure interpolate a value at the pixel using the adjacent first and second points by accounting for longitudinal movement of a scanner element.
- the technique removes discontinuities (e.g., seams) and/or other distortions from the resulting image.
- the de-seamed image provides for more accurate measurements of the surrounding tissue.
- the resulting image is also free of feature doubling, the tendency of a single feature to be present at more than one location in the image.
- a medical sensing system is able to perform this interpolation without loss of resolution, blurring, or unintended filtering of features.
- the de-seaming technique may be performed on a variety of sensing data types including ultrasound echo data and optical coherence tomography data and does not require prior conversion of the data to any particular representation (e.g., baseband or time-domain).
- any particular representation e.g., baseband or time-domain.
- FIGS. 1A and 1B are diagrammatic schematic views of a medical sensing system according to some embodiments of the present disclosure.
- FIG. 2 is a flow diagram of a method of performing a diagnostic procedure using the medical sensing system according to embodiments of the present disclosure.
- FIG. 3 is a perspective diagram of a path of an elongate member during data acquisition according to embodiments of the present disclosure.
- FIG. 4 is a representation of an image produced by a medical sensing system using sensing data acquired by the elongate member according to embodiments of the present disclosure.
- FIGS. 5A and 5B are flow diagrams of a method of motion compensation and seam elimination according to embodiments of the present disclosure.
- FIG. 6 is a perspective diagram of a path of an elongate member during data acquisition according to embodiments of the present disclosure.
- FIG. 7 is a schematic diagram of a de-seaming engine of a medical sensing system according to embodiments of the present disclosure.
- FIG. 8 is a representation of a de-seamed image produced by the medical sensing system according to embodiments of the present disclosure.
- FIG. 1A is a diagrammatic schematic view of a medical sensing system 100 according to some embodiments of the present disclosure.
- the medical sensing system 100 includes an elongate member 102 (such as a catheter, guide wire, or guide catheter) of the medical sensing system 100 .
- elongate member or “flexible elongate member” includes at least any thin, long, flexible structure that can be inserted into the vasculature of a patient.
- flexible elongate members 102 include, for example, guide wires, catheters, and guide catheters.
- a catheter may or may not include a lumen extending along its length for receiving and/or guiding other instruments. If the catheter includes a lumen, the lumen may be centered or offset with respect to the cross-sectional profile of the device.
- the medical sensing system 100 may be utilized in a variety of applications and can be used to assess vessels and structures within a living body. To do so, the elongate member 102 is advanced into a vessel 104 .
- Vessel 104 represents fluid filled or surrounded structures, both natural and man-made, within a living body that may be imaged and can include for example, but without limitation, structures such as: organs including the liver, heart, kidneys, as well as valves within the blood or other systems of the body.
- the images may also include man-made structures such as, but without limitation, heart valves, stents, shunts, filters and other devices positioned within the body.
- the elongate member 102 includes sensors 106 disposed along the length of the member 102 to collect diagnostic data regarding the vessel 104 .
- the sensors 106 correspond to sensing modalities such as flow, optical flow, IVUS, photoacoustic IVUS, FL-IVUS, pressure, optical pressure, fractional flow reserve (FFR) determination, coronary flow reserve (CFR) determination, OCT, transesophageal echocardiography, image-guided therapy, other suitable modalities, and/or combinations thereof.
- the elongate member 102 includes a solid-state IVUS device
- the sensors 106 include an array of IVUS ultrasound transceivers and associated control circuitry incorporated into a scanner assembly 108 .
- the sensors 106 may be arranged around the circumference of the elongate member 102 and positioned to emit ultrasound energy radially 110 in order to obtain a cross-sectional representation of the vessel 104 and the surrounding anatomy.
- the control circuitry selects some of the IVUS transceivers to transmit an ultrasound pulse that is reflected by the vessel 104 and the surrounding structures.
- the control circuitry also selects some transceivers to receive the echo signal.
- FIG. 1B is a schematic view of a system that includes an alternative elongate member 102 according to some embodiments of the present disclosure.
- the elongate member 102 of FIG. 1B is typical of a rotational device such as a rotational IVUS ultrasound system and the sensor 106 includes one or more IVUS transducers arranged to emit ultrasound energy in a radial direction 110 .
- the sensor(s) 106 may be mechanically rotated around a longitudinal axis of the elongate member 102 to obtain a cross-sectional representation of the vessel 104 .
- sensor 106 includes an OCT transceiver.
- OCT transceiver Other embodiments incorporate other combinations of sensors, and no particular sensor or combination of sensors is required for any particular embodiment.
- PIM Patient Interface Module
- the PIM 114 is an isolation device as, in various surgical settings, patient safety requirements mandate physical and electrical isolation of the patient. Thus, if complete electrical isolation is required, the PIM 114 and the console 114 may be communicatively coupled by an optical, RF, or other non-conductive link. In less stringent environments, conductive communication links and/or power couplings may extend between the two. Moreover, in some embodiments, the PIM 114 and console 114 are collocated and/or part of the same system, unit, chassis, or module.
- the PIM 114 and console 114 assemble, process, and render the sensor data for display as an image on a monitor 118 .
- the PIM 114 and/or the console 114 generates control signals to configure the sensor 106 , generates signals to activate the sensor 106 , performs amplification, filtering, and/or aggregating of sensor data, and formats the sensor data as an image for display.
- the allocation of these tasks and others between the PIM 114 and the console 114 is merely arbitrary.
- the elongate member 102 may include a guide wire exit port 120 as shown in FIG. 1A .
- the guide wire exit port 120 allows a guide wire 122 to be inserted towards the distal end in order to direct the member 102 through a vascular structure (i.e., the vessel) 104 .
- the elongate member 102 is a rapid-exchange catheter.
- the elongate member 102 is advanced through the vessel 104 inside a guide catheter 124 as shown in FIG. 1B .
- the elongate member 102 includes an inflatable balloon portion 126 near the distal tip.
- the balloon portion 126 is open to a lumen that travels along the length of the IVUS device and ends in an inflation port (not shown). The balloon 126 may be selectively inflated and deflated via the inflation port.
- FIG. 2 is a flow diagram of a method 200 of performing a diagnostic procedure using the medical sensing system 100 according to embodiments of the present disclosure. It is understood that additional steps can be provided before, during, and after the steps of method 200 , and that some of the steps described can be replaced or eliminated for other embodiments of the method.
- a surgeon places a guide wire 122 in the vessel 104 .
- the guide wire 122 is threaded through at least a portion of the distal end of the elongate member 102 before, during, or after placement of the guide wire 122 .
- the elongate member 102 is advanced over the guide wire.
- a guide catheter 124 is advanced in the vessel 104 in block 202 and the elongate member 102 is advanced within the guide catheter in block 204 .
- the sensor 106 is activated. Imaging is described in more detail with reference to FIGS. 3-8 , but at a high level, signals sent from the PIM 114 to the sensor 106 via the cable 112 cause the sensor to obtain diagnostic information.
- transducers within the sensor 106 emit a specified ultrasonic waveform. The ultrasonic waveform is reflected by the vessel 104 and the surrounding anatomy. The reflections are received by the transducers and are amplified for transmission via the cable 112 . The echo data is placed on the cable 112 and sent to the PIM 114 .
- an optical emitter within the sensor 106 produces light that is reflected by structures in the surrounding environment. The reflected light is compared to a reference signal by the elongate member 102 or the PIM 114 to obtain intensity and distance measurements.
- the PIM 114 may perform any suitable signal processing or enhancement before retransmitting the sensor data to the console 114 in block 208 .
- the console 114 aggregates and assembles the received sensor data to create an image of the vessel 104 for display on the monitor 118 .
- the elongate member 102 is advanced beyond the area of the vessel 104 to be imaged and pulled back as the scanner assembly 108 is operating, thereby exposing and imaging a longitudinal portion of the vessel 104 .
- a pullback mechanism is used in some instances.
- a typical withdraw velocity is 0.5 cm/s.
- the member 102 includes an inflatable balloon portion 126 .
- the device may be positioned adjacent to a stenosis (narrow segment) or an obstructing plaque within the vessel 104 and inflated in an attempt to widen the restricted area of the vessel 104 .
- FIG. 3 is a perspective diagram of a path 300 of an elongate member 102 during data acquisition according to embodiments of the present disclosure.
- FIG. 4 is a representation of an image 400 produced by the medical sensing system 100 using sensing data acquired by the elongate member 102 according to embodiments of the present disclosure.
- FIG. 4 represents an image 400 generated from IVUS echo data although the principles of the present disclosure apply equally to other sensing modalities.
- FIG. 3 the path 300 of an elongate member 102 is shown.
- the elongate member 102 travels longitudinally as indicated by cylinder 302 , and as it does so, one or more sensors 106 of the elongate member 102 acquire imaging data in a radial direction as indicated by the radial lines 304 .
- Helical line 306 represents the imaged area due to the longitudinal motion and the radial scanning.
- the medical sensing system 100 presents the imaging data on a monitor 118 as a two-dimensional cross section of the surrounding environment.
- plane 308 of FIG. 3 represents one possible cross-sectional slice.
- interpolation is performed using data outside the plane 308 .
- FIG. 4 in a simple example, data collected from one revolution of the helix (i.e., one frame of data indicated by reference 310 in FIG. 3 ) centered at the cross-sectional plane is displayed without any further processing.
- the seam 402 arises because data collected on opposite sides of the seam has different longitudinal positions that are not properly accounted for.
- the flattening process produces other distortions that are less obvious and may be more troublesome. For example, because the flattened helix is not a true cross-section, it may hide curvature of the vessel 104 and may skew feature measurements.
- FIGS. 5A and 5B are flow diagrams of the method 500 of motion compensation and seam elimination according to embodiments of the present disclosure. It is understood that additional steps can be provided before, during, and after the steps of method 500 , and that some of the steps described can be replaced or eliminated for other embodiments of the method.
- FIG. 6 is a perspective diagram of a path 600 of an elongate member 102 during data acquisition according to embodiments of the present disclosure.
- FIG. 7 is a schematic diagram of a de-seaming engine 700 of a medical sensing system 100 according to embodiments of the present disclosure.
- the de-seaming engine 700 may be incorporated into the PIM 114 , console 114 , and/or any other processing element of the medical sensing system 100 and may operate in conjunction with any other data acquisition, image processing, and/or image analysis elements of the medical sensing system 100 .
- FIG. 8 is a representation of a de-seamed image 800 produced by the medical sensing system 100 according to embodiments of the present disclosure.
- FIG. 8 represents an image 800 generated from IVUS echo data, although the principles of the present disclosure apply equally to other sensing modalities.
- the medical sensing system 100 determines a desired location for a cross-sectional slice in a longitudinal direction.
- the medical sensing system 100 may produce cross-sectional images at various intervals determined by position and/or time.
- the slice/image location need not correspond to a radial beam, and such a medical sensing system 100 supports cross-sectional slices located between the radial beams.
- plane 308 represents an exemplary longitudinal location for the cross-sectional slice.
- the medical sensing system 100 identifies at least one revolution or frame (e.g., leading frame 602 ) ahead of the cross-sectional plane 308 in the longitudinal direction and at least one revolution or frame (e.g., lagging frame 604 ) behind the cross-sectional plane 308 in the longitudinal direction.
- the medical sensing system 100 uses these two frames to interpolate imaging data that lies in the cross-sectional plane 308 rather than just flattening the acquired data.
- the medical sensing system identifies a pixel 606 within the cross-sectional plane 308 .
- the pixel 606 is a region of arbitrary size and may be identified by its cylindrical coordinates, a combination of the pixel's radial orientation (r), azimuthal orientation ( ⁇ ), and longitudinal orientation (z).
- the exemplary radial pixel 606 is selected, and the cylindrical coordinates (r, ⁇ , z) are indicated.
- imaging data values at the pixel 606 may be determined from acquired data values at corresponding radial and azimuthal orientations by compensating for the difference in longitudinal orientation.
- the medical sensing system 100 acquires a signal intensity at a first point 608 in the first frame at a radial and azimuthal orientation that corresponds to the pixel 606 (e.g., the leading frame 602 ).
- the first point 608 may be beyond the imaging plane 308 and thus offset from the pixel in a longitudinal direction (e.g., leading or lagging).
- the signal intensity may correspond to ultrasound echo data, OCT data, and/or any other type of sensing data and may be in any representation including baseband, radio-frequency/time domain, and/or other suitable representation.
- the medical sensing system 100 determines the amount of longitudinal offset between the first point 608 and the pixel 606 .
- the longitudinal offset may be determined based on the number of radial measurements taken between the plane 308 of the pixel 606 and the first point 608 .
- the longitudinal offset may be determined by the difference of the two values.
- the longitudinal offset may be expressed as a number of radial measurements, but may also be expressed as a distance value or time value.
- the longitudinal offset may be normalized so that the span of each frame is a fixed value, such as 1 or the number of radial measurements per frame.
- a weighting element 702 of the medical sensing system 100 determines a weighting factor for the first point 608 based on the longitudinal offset of block 510 .
- the weighting factor reflects the certainty that a signal intensity at the pixel 606 corresponds to the signal intensity at the first point 608 and thus, is inversely proportional to the offset.
- the weighting factor has a linear relationship and/or a higher-order polynomial relationship with the longitudinal offset.
- the weighting factor may depend on the square of the longitudinal offset in order to place greater emphasis on values obtained near to the pixel 606 .
- the medical sensing system 100 acquires a signal intensity at a second point 610 in the second frame (e.g., lagging frame 604 ) at a radial and azimuthal orientation that corresponds to the pixel 606 .
- the second point 610 may be beyond the imaging plane 308 and offset from the pixel 606 in the opposite longitudinal direction from the first point 608 . Accordingly, the second point 610 may be the first point 608 delayed by a frame (or vice versa).
- the signal intensity may correspond to ultrasound echo data, OCT data, and/or any other type of sensing data and may be in any representation including baseband, radio-frequency/time domain, and/or other suitable representation.
- the medical sensing system 100 determines the amount of longitudinal offset between the second point 610 and the pixel 606 substantially as described in block 510 .
- a second weighting element 702 of the medical sensing system 100 determines a second weighting factor to the second point 610 based on the determined offset of block 516 .
- the second weighting factor reflects the certainty that a signal intensity at the pixel 606 corresponds to the signal intensity at the second point 610 and thus, is inversely proportional to the offset.
- the second weighting factor has a linear relationship and/or a higher-order polynomial relationship with the longitudinal offset.
- an averaging element 704 of the medical sensing system 100 averages the weighted signal intensity at the first point 608 and the weighted signal intensity at the second point 610 .
- This average signal intensity is an interpolated value and has at least some correlation to the signal intensity at the pixel 606 . Accordingly, in some embodiments, the average signal intensity is used as the interpolated signal value at the pixel 606 without further processing.
- An image formed using average signal intensities may eliminate the seam found in other techniques and may reduce other distortion including curvature distortion.
- the average signal intensity has a number of other artifacts, such as feature doubling, that can affect diagnosis.
- Feature doubling occurs when a strong reflective feature is located at a first position in a first frame and a second position in a second frame.
- the strong signal intensity dominates the weighted average so that the feature appears in both positions in the average. This may give the appearance of two separate and distinct features.
- Feature doubling may be caused by a stationary feature that extends along the longitudinal path but is somewhat skewed relative to the pullback path. It may also be caused by tissue moving faster than the frame rate and the under-sampling and the resultant aliasing.
- averaging is, in effect, a type of low pass filtering, and weak signal intensities are often obscured. As a result, the average signal intensity may exhibit resolution blurring and reduced sensitivity.
- the medical sensing system 100 uses the average signal intensity to determine a probability that the value at the pixel 606 closer to the signal intensity at the first point 608 than the second point 610 (or vice-versa).
- a first comparator 706 of the medical sensing system 100 determines the difference between the average signal intensity and the signal intensity at the first point 608 to determine a first probability indicator.
- the magnitudes (i.e., absolute values) of the average signal intensity and the signal intensity at the first point are compared and the probability indicator is expressed as a magnitude. Referring to block 524 of FIG. 5B and to FIG.
- a second comparator 708 of the medical sensing system determines the difference between the average signal intensity and the signal intensity at the second point 610 to determine a second probability indicator.
- the magnitudes of the average signal intensity and the signal intensity at the second point are compared and the second probability indicator is expressed as a magnitude.
- a third comparator 710 of the medical sensing system 100 compares the first probability indicator to the second probability indicator, and the intensity at the pixel 606 is selected based on the comparison.
- a multiplexer selects the first signal intensity or the second signal intensity based on the result.
- the intensity at the first point 608 is selected to represent the intensity at the pixel 606 when the first probability indicator is greater
- the intensity at the second point 610 is selected to represent the intensity at the pixel 606 when the second probability indicator is greater.
- the selected signal intensity is provided for further signal processing, such as image filtering, envelope filtering, log compression, gamma curve adjustment, etc., in order to generate a diagnostic image.
- the processes of blocks 506 - 528 may be repeated for each pixel in the diagnostic image.
- FIG. 8 is a representation of a de-seamed image 800 produced by the medical sensing system 100 according to embodiments of the present disclosure.
- the method 500 produces pixel intensities such that the resulting image does not have a seam.
- the resultant images may also depict vascular curvature more accurately.
- the pixel values selected by the method 500 do not exhibit feature doubling.
- the resultant image suffers no loss of resolution, blurring, or low-intensity filtering, which may occur from other seam-elimination techniques.
- the method 500 may be applied to any representation of the signal intensities and does not require prior conditioning to convert the data to a format such as baseband, radio-frequency (RF)/time domain, or any other representation.
- RF radio-frequency
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Signal Processing (AREA)
- Gynecology & Obstetrics (AREA)
- Artificial Intelligence (AREA)
- Physiology (AREA)
- Psychiatry (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Vascular Medicine (AREA)
- Quality & Reliability (AREA)
- Ultra Sonic Daignosis Equipment (AREA)
Abstract
Description
- The present application claims priority to and the benefit of the U.S. Provisional Patent Application No. 62/094,303, filed Dec. 19, 2014, which is hereby incorporated by reference in its entirety.
- The present disclosure relates generally to imaging and, in particular, to correcting for longitudinal movement of an imaging device during data acquisition. In various embodiments, the medical sensing system includes an intravascular portion having an elongate member with one or more sensors such as ultrasound transducers and/or optical coherence tomography sensors oriented to collect data radially as the elongate member is drawn longitudinally through a vessel. The medical sensing system assembles the radial data into cross-sectional slices of the vessel. Due to the longitudinal motion, the radial data is actually a helical representation of the vessel and according, the medical sensing system flattens the helical data to produce a cross-sectional view. The system is suitable for use in a variety of applications including intravascular ultrasound. For example, some embodiments of the present disclosure provide an IVUS medical sensing system particularly suited to imaging a human blood vessel.
- Innovations in diagnosing and verifying the level of success of treatment of disease have migrated from external imaging processes to internal diagnostic processes. In particular, diagnostic equipment and processes have been developed for diagnosing vasculature blockages and other vasculature disease by means of ultra-miniature sensors placed upon the distal end of a flexible elongate member such as a catheter, or a guide wire used for catheterization procedures. For example, known medical sensing techniques include angiography, intravascular ultrasound (IVUS), forward looking IVUS (FL-IVUS), fractional flow reserve (FFR) determination, a coronary flow reserve (CFR) determination, optical coherence tomography (OCT), transesophageal echocardiography, and image-guided therapy.
- For example, intravascular ultrasound (IVUS) imaging is widely used in interventional cardiology as a diagnostic tool for assessing a diseased vessel, such as an artery, within the human body to determine the need for treatment, to guide the intervention, and/or to assess its effectiveness. There are two general types of IVUS devices in use today: rotational and solid-state (also known as synthetic aperture phased array). For a typical rotational IVUS device, a single ultrasound transducer element is located at the tip of a flexible driveshaft that spins inside a plastic sheath inserted into the vessel of interest. In side-looking rotational devices, the transducer element is oriented such that the ultrasound beam propagates generally perpendicular to the longitudinal axis of the device. In forward-looking rotational devices, the transducer element is pitched towards the distal tip so that the ultrasound beam propagates more towards the tip (in some devices, being emitted parallel to the longitudinal centerline). The fluid-filled sheath protects the vessel tissue from the spinning transducer and driveshaft while permitting ultrasound signals to propagate from the transducer into the tissue and back. As the driveshaft rotates, the transducer is periodically excited with a high voltage pulse to emit a short burst of ultrasound. The same transducer then listens for the returning echoes reflected from various tissue structures. The IVUS medical sensing system assembles a two dimensional display of the tissue, vessel, heart structure, etc. from a sequence of pulse/acquisition cycles occurring during a single revolution of the transducer. In order to image a length of a vessel, the transducer element is drawn through the vessel as it spins.
- In contrast, solid-state IVUS devices utilize a scanner assembly that includes an array of ultrasound transducers connected to a set of transducer controllers. In side-looking and some forward-looking IVUS devices, the transducers are distributed around the circumference of the device. In other forward-looking IVUS devices, the transducers are a linear array arranged at the distal tip and pitched so that the ultrasound beam propagates closer to parallel with the longitudinal centerline. The transducer controllers select transducer sets for transmitting an ultrasound pulse and for receiving the echo signal. By stepping through a sequence of transmit-receive sets, the solid-state IVUS system can synthesize the effect of a mechanically scanned transducer element but without moving parts. Since there is no rotating mechanical element, the transducer array can be placed in direct contact with the blood and vessel tissue with minimal risk of vessel trauma. Furthermore, because there is no rotating element, the interface is simplified. The solid-state scanner can be wired directly to the medical sensing system with a simple electrical cable and a standard detachable electrical connector. While the transducers of the scanner assembly do not spin, operation is similar to that of a rotational system in that, in order to image a length of a vessel, the scanner assembly is drawn through the vessel while stepping through the transmit-receive sets to produce a series of radial scans.
- Rotational and solid-state state IVUS are merely some examples of imaging modalities that sample a narrow region of the environment and assemble a two- or three-dimensional image from the results. Other examples include optical coherence tomography (OCT). One of the key challenges in this type of imaging is inferring information in areas of the environment that are not directly imaged. In the preceding examples where a narrow beam rotates around the axis, the radial data is only coplanar if there is no movement during a rotation. In more common applications where the scanning element is drawn through the vessel during imaging, each radially imaged region will have a different longitudinal orientation. In other words, the scanner assembly images a helical region rather than a set of flat circular regions. If the helical data is simply flattened, a discontinuity or seam may occur between the first and last radial scans of the image. In addition to being visually unappealing, the seam also indicates that the scanning data is being incorrectly displayed. For example, improper flattening may hide the curvature of the vessel. Effects due to improper flattening become more pronounced as the longitudinal motion becomes greater relative to the rotational speed. For example, OCT may have a much higher relative pullback speed than IVUS, resulting in more longitudinal distortion and may experience an even greater benefit from proper correction.
- Accordingly, a more accurate technique for reconstructing a planar cross-section from non-planar data would produce a more accurate image that is closer to what a clinician would expect and may thus improve diagnostic accuracy. As an additional benefit, it may reduce or eliminate distortions and discontinuities (e.g., seams). For these reasons and others, the need exists for improved systems and techniques that account for longitudinal motion of a scanning element and for compensating when interpolating data.
- Embodiments of the present disclosure provide an imaging component that assembles a cross-sectional representation from a helical set of imaging data and removes artifacts caused by conventional flattening processes. The imaging component may be used in applications such as an intravascular ultrasound medical sensing system.
- In some embodiments, a medical sensing system is provided. The system comprises a de-seaming engine operable to receive a first signal at a first point and a second signal at a second point. The first point and the second point may have similar relative positions while being in different frames. In other words, the first and second points may have the same radial and azimuthal orientation but occupy different positions longitudinally. The de-seaming engine determines an intensity at a location between the first and second points using the first signal and the second signal. To achieve this, the de-seaming engine includes an averaging element operable to determine a weighted average of the first signal and the second signal. The weighted average is used to determine an interpolated signal value at the location based on the signal values measured at the first and second points. Accordingly, the de-seaming engine also includes a first comparator operable to determine a first probability based on the first signal and the weighted average and a second comparator operable to determine a second probability based on the second signal and the weighted average. The first and second probabilities are provided to a third comparator of the de-seaming engine, which is operable to select one of the first and second signal intensities as an intensity at the location based on the first and second probabilities and to provide the selected intensity as a pixel intensity for forming an image.
- In some embodiments, a method is provided. The method includes receiving a first signal value associated with a first point located a first longitudinal distance from a pixel and a second signal value associated with a second point located a second longitudinal distance from the pixel. A first probability associated with the first point is determined based on the first longitudinal distance, and a second probability associated with the second point is determined based on the second longitudinal distance. A signal value for the pixel is selected based on the first probability and the second probability, and the selected signal value for the pixel is provided for forming an image. In some such embodiments the method also includes determining a first weighting factor for the first point based on the first longitudinal distance and determining a second weighting factor for the second point based on the second longitudinal distance. A weighted average is determined based on the first signal value, the first weighting factor, the second signal value, and the second weighting factor. The determining of the first probability includes comparing the first signal value to the weighted average, and the determining of the second probability includes comparing the second signal value to the weighted average.
- In further embodiments, a method of diagnostic imaging is provided. The method includes identifying a pixel located between a first point and a second point in a longitudinal direction. A first imaging signal for the first point and a second imaging signal for the second point are received, and a weighted average of the first imaging signal and the second imaging signal is determined. The first signal is compared to the weighted average to determine a first probability, and the second signal is compared to the weighted average to determine a second probability. A value for the pixel is selected based on the first probability and the second probability. The selected value is provided for use in forming a diagnostic image. In some such embodiments, the method also includes determining a first weighting factor for the first imaging signal based on a longitudinal distance between the first point and the pixel and determining a second weighting factor for the second imaging signal based on a longitudinal distance between the second point and the pixel. The determining of the weighted average includes applying the first weighting factor to the first imaging signal and applying the second weighting factor to the second imaging signal. In some such embodiments, the first weighting factor is determined based on a count of radial measurements between the first point and the pixel, and the second weighting factor is determined based on a count of radial measurements between the second point and the pixel.
- In this way, the techniques of the present disclosure interpolate a value at the pixel using the adjacent first and second points by accounting for longitudinal movement of a scanner element. By so doing, the technique removes discontinuities (e.g., seams) and/or other distortions from the resulting image. In addition to being more visually appealing, the de-seamed image provides for more accurate measurements of the surrounding tissue. In many embodiments, by selecting from the first and second signal intensities rather than averaging the intensities, the resulting image is also free of feature doubling, the tendency of a single feature to be present at more than one location in the image. Compared to other techniques, a medical sensing system is able to perform this interpolation without loss of resolution, blurring, or unintended filtering of features. The de-seaming technique may be performed on a variety of sensing data types including ultrasound echo data and optical coherence tomography data and does not require prior conversion of the data to any particular representation (e.g., baseband or time-domain). Of course, these advantages are merely exemplary, and no single advantage is characteristic of or required for any particular embodiment.
- Additional aspects, features, and advantages of the present disclosure will become apparent from the following detailed description.
- Illustrative embodiments of the present disclosure will be described with reference to the accompanying drawings, of which:
-
FIGS. 1A and 1B are diagrammatic schematic views of a medical sensing system according to some embodiments of the present disclosure. -
FIG. 2 is a flow diagram of a method of performing a diagnostic procedure using the medical sensing system according to embodiments of the present disclosure. -
FIG. 3 is a perspective diagram of a path of an elongate member during data acquisition according to embodiments of the present disclosure. -
FIG. 4 is a representation of an image produced by a medical sensing system using sensing data acquired by the elongate member according to embodiments of the present disclosure. -
FIGS. 5A and 5B are flow diagrams of a method of motion compensation and seam elimination according to embodiments of the present disclosure. -
FIG. 6 is a perspective diagram of a path of an elongate member during data acquisition according to embodiments of the present disclosure. -
FIG. 7 is a schematic diagram of a de-seaming engine of a medical sensing system according to embodiments of the present disclosure. -
FIG. 8 is a representation of a de-seamed image produced by the medical sensing system according to embodiments of the present disclosure. - For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the embodiments illustrated in the drawings, and specific language will be used to describe the same. It is nevertheless understood that no limitation to the scope of the disclosure is intended. Any alterations and further modifications to the described devices, systems, and methods, and any further application of the principles of the present disclosure are fully contemplated and included within the present disclosure as would normally occur to one skilled in the art to which the disclosure relates. For example, while the intravascular sensing system is described in terms of cardiovascular imaging, it is understood that it is not intended to be limited to this application. The system is equally well suited to any application requiring imaging within a confined cavity. In particular, it is fully contemplated that the features, components, and/or steps described with respect to one embodiment may be combined with the features, components, and/or steps described with respect to other embodiments of the present disclosure. For the sake of brevity, however, the numerous iterations of these combinations will not be described separately.
-
FIG. 1A is a diagrammatic schematic view of amedical sensing system 100 according to some embodiments of the present disclosure. Themedical sensing system 100 includes an elongate member 102 (such as a catheter, guide wire, or guide catheter) of themedical sensing system 100. As used herein, “elongate member” or “flexible elongate member” includes at least any thin, long, flexible structure that can be inserted into the vasculature of a patient. While the illustrated embodiments of the “elongate members” of the present disclosure have a cylindrical profile with a circular cross-sectional profile that defines an outer diameter of the flexible elongate member, in other instances, all or a portion of the flexible elongate members may have other geometric cross-sectional profiles (e.g., oval, rectangular, square, elliptical, etc.) or non-geometric cross-sectional profiles. Flexibleelongate members 102 include, for example, guide wires, catheters, and guide catheters. In that regard, a catheter may or may not include a lumen extending along its length for receiving and/or guiding other instruments. If the catheter includes a lumen, the lumen may be centered or offset with respect to the cross-sectional profile of the device. - The
medical sensing system 100 may be utilized in a variety of applications and can be used to assess vessels and structures within a living body. To do so, theelongate member 102 is advanced into avessel 104.Vessel 104 represents fluid filled or surrounded structures, both natural and man-made, within a living body that may be imaged and can include for example, but without limitation, structures such as: organs including the liver, heart, kidneys, as well as valves within the blood or other systems of the body. In addition to imaging natural structures, the images may also include man-made structures such as, but without limitation, heart valves, stents, shunts, filters and other devices positioned within the body. Theelongate member 102 includessensors 106 disposed along the length of themember 102 to collect diagnostic data regarding thevessel 104. In various embodiments, thesensors 106 correspond to sensing modalities such as flow, optical flow, IVUS, photoacoustic IVUS, FL-IVUS, pressure, optical pressure, fractional flow reserve (FFR) determination, coronary flow reserve (CFR) determination, OCT, transesophageal echocardiography, image-guided therapy, other suitable modalities, and/or combinations thereof. - In the exemplary embodiment of
FIG. 1A , theelongate member 102 includes a solid-state IVUS device, and thesensors 106 include an array of IVUS ultrasound transceivers and associated control circuitry incorporated into ascanner assembly 108. Thesensors 106 may be arranged around the circumference of theelongate member 102 and positioned to emit ultrasound energy radially 110 in order to obtain a cross-sectional representation of thevessel 104 and the surrounding anatomy. When thescanner assembly 108 is positioned near the area to be imaged, the control circuitry selects some of the IVUS transceivers to transmit an ultrasound pulse that is reflected by thevessel 104 and the surrounding structures. The control circuitry also selects some transceivers to receive the echo signal. By stepping through sequences of transmit-receive sets, themedical sensing system 100 system can synthesize the effect of a mechanically scanned transducer element without moving parts. -
FIG. 1B is a schematic view of a system that includes an alternativeelongate member 102 according to some embodiments of the present disclosure. Theelongate member 102 ofFIG. 1B is typical of a rotational device such as a rotational IVUS ultrasound system and thesensor 106 includes one or more IVUS transducers arranged to emit ultrasound energy in aradial direction 110. In such an embodiment, the sensor(s) 106 may be mechanically rotated around a longitudinal axis of theelongate member 102 to obtain a cross-sectional representation of thevessel 104. - In a further embodiment,
sensor 106 includes an OCT transceiver. Other embodiments incorporate other combinations of sensors, and no particular sensor or combination of sensors is required for any particular embodiment. - Data from the sensor(s) 106 is transmitted via a
cable 112 to a Patient Interface Module (PIM) 114 and/or aconsole 114. ThePIM 114 is an isolation device as, in various surgical settings, patient safety requirements mandate physical and electrical isolation of the patient. Thus, if complete electrical isolation is required, thePIM 114 and theconsole 114 may be communicatively coupled by an optical, RF, or other non-conductive link. In less stringent environments, conductive communication links and/or power couplings may extend between the two. Moreover, in some embodiments, thePIM 114 andconsole 114 are collocated and/or part of the same system, unit, chassis, or module. Together thePIM 114 andconsole 114 assemble, process, and render the sensor data for display as an image on amonitor 118. For example, in various embodiments, thePIM 114 and/or theconsole 114 generates control signals to configure thesensor 106, generates signals to activate thesensor 106, performs amplification, filtering, and/or aggregating of sensor data, and formats the sensor data as an image for display. The allocation of these tasks and others between thePIM 114 and theconsole 114 is merely arbitrary. - In addition to
various sensors 106, theelongate member 102 may include a guidewire exit port 120 as shown inFIG. 1A . The guidewire exit port 120 allows aguide wire 122 to be inserted towards the distal end in order to direct themember 102 through a vascular structure (i.e., the vessel) 104. Accordingly, in some instances theelongate member 102 is a rapid-exchange catheter. Additionally or in the alternative, theelongate member 102 is advanced through thevessel 104 inside aguide catheter 124 as shown inFIG. 1B . In an embodiment, theelongate member 102 includes aninflatable balloon portion 126 near the distal tip. Theballoon portion 126 is open to a lumen that travels along the length of the IVUS device and ends in an inflation port (not shown). Theballoon 126 may be selectively inflated and deflated via the inflation port. -
FIG. 2 is a flow diagram of amethod 200 of performing a diagnostic procedure using themedical sensing system 100 according to embodiments of the present disclosure. It is understood that additional steps can be provided before, during, and after the steps ofmethod 200, and that some of the steps described can be replaced or eliminated for other embodiments of the method. - Referring
block 202 ofFIG. 2 and referring still toFIGS. 1A and 1B , in an illustrative example of a typical environment and application of the system, a surgeon places aguide wire 122 in thevessel 104. Theguide wire 122 is threaded through at least a portion of the distal end of theelongate member 102 before, during, or after placement of theguide wire 122. Referring to block 204 ofFIG. 2 , once theguide wire 122 is in place, theelongate member 102 is advanced over the guide wire. Additionally or in the alternative, aguide catheter 124 is advanced in thevessel 104 inblock 202 and theelongate member 102 is advanced within the guide catheter inblock 204. - Referring to block 206, once positioned, the
sensor 106 is activated. Imaging is described in more detail with reference toFIGS. 3-8 , but at a high level, signals sent from thePIM 114 to thesensor 106 via thecable 112 cause the sensor to obtain diagnostic information. In the example of an IVUS application, transducers within thesensor 106 emit a specified ultrasonic waveform. The ultrasonic waveform is reflected by thevessel 104 and the surrounding anatomy. The reflections are received by the transducers and are amplified for transmission via thecable 112. The echo data is placed on thecable 112 and sent to thePIM 114. In the example of an OCT application, an optical emitter within thesensor 106 produces light that is reflected by structures in the surrounding environment. The reflected light is compared to a reference signal by theelongate member 102 or thePIM 114 to obtain intensity and distance measurements. - In these examples and others, the
PIM 114 may perform any suitable signal processing or enhancement before retransmitting the sensor data to theconsole 114 inblock 208. Referring to block 210, theconsole 114 aggregates and assembles the received sensor data to create an image of thevessel 104 for display on themonitor 118. In some exemplary applications, theelongate member 102 is advanced beyond the area of thevessel 104 to be imaged and pulled back as thescanner assembly 108 is operating, thereby exposing and imaging a longitudinal portion of thevessel 104. To ensure a constant velocity, a pullback mechanism is used in some instances. A typical withdraw velocity is 0.5 cm/s. In some embodiments, themember 102 includes aninflatable balloon portion 126. As part of a treatment procedure, the device may be positioned adjacent to a stenosis (narrow segment) or an obstructing plaque within thevessel 104 and inflated in an attempt to widen the restricted area of thevessel 104. - Because the sensor(s) 106 collect data as the
elongate member 102 is withdrawn longitudinally through thevessel 104, each data point has a longitudinal component in addition to the radial component. The effect of this longitudinal motion on images produced by themedical sensing system 100 is described with reference toFIGS. 3 and 4 .FIG. 3 is a perspective diagram of apath 300 of anelongate member 102 during data acquisition according to embodiments of the present disclosure.FIG. 4 is a representation of animage 400 produced by themedical sensing system 100 using sensing data acquired by theelongate member 102 according to embodiments of the present disclosure.FIG. 4 represents animage 400 generated from IVUS echo data although the principles of the present disclosure apply equally to other sensing modalities. - Referring first to
FIG. 3 , thepath 300 of anelongate member 102 is shown. Theelongate member 102 travels longitudinally as indicated bycylinder 302, and as it does so, one ormore sensors 106 of theelongate member 102 acquire imaging data in a radial direction as indicated by the radial lines 304.Helical line 306 represents the imaged area due to the longitudinal motion and the radial scanning. - In many embodiments, the
medical sensing system 100 presents the imaging data on amonitor 118 as a two-dimensional cross section of the surrounding environment. For example,plane 308 ofFIG. 3 represents one possible cross-sectional slice. However, when a complete revolution of data is not available in the cross-sectional plane, such as may occur when theelongate member 102 is being pulled back, interpolation is performed using data outside theplane 308. Referring toFIG. 4 , in a simple example, data collected from one revolution of the helix (i.e., one frame of data indicated byreference 310 inFIG. 3 ) centered at the cross-sectional plane is displayed without any further processing. However, this may result in an obvious discontinuity or seam as data from the beginning and end of the revolution are positioned adjacent one another. In the example ofFIG. 4 , the seam is indicated byreference marker 402. Theseam 402 arises because data collected on opposite sides of the seam has different longitudinal positions that are not properly accounted for. The flattening process produces other distortions that are less obvious and may be more troublesome. For example, because the flattened helix is not a true cross-section, it may hide curvature of thevessel 104 and may skew feature measurements. - Accordingly, a method is provided that corrects for longitudinal motion during data acquisition in order to produce a more accurate cross-sectional image. Embodiments of the method are described with reference to
FIGS. 5A-8 .FIGS. 5A and 5B are flow diagrams of themethod 500 of motion compensation and seam elimination according to embodiments of the present disclosure. It is understood that additional steps can be provided before, during, and after the steps ofmethod 500, and that some of the steps described can be replaced or eliminated for other embodiments of the method.FIG. 6 is a perspective diagram of apath 600 of anelongate member 102 during data acquisition according to embodiments of the present disclosure.FIG. 7 is a schematic diagram of ade-seaming engine 700 of amedical sensing system 100 according to embodiments of the present disclosure. Thede-seaming engine 700 may be incorporated into thePIM 114,console 114, and/or any other processing element of themedical sensing system 100 and may operate in conjunction with any other data acquisition, image processing, and/or image analysis elements of themedical sensing system 100.FIG. 8 is a representation of ade-seamed image 800 produced by themedical sensing system 100 according to embodiments of the present disclosure.FIG. 8 represents animage 800 generated from IVUS echo data, although the principles of the present disclosure apply equally to other sensing modalities. - Referring to block 502 of
FIG. 5A and referring toFIG. 6 , themedical sensing system 100 determines a desired location for a cross-sectional slice in a longitudinal direction. For example, themedical sensing system 100 may produce cross-sectional images at various intervals determined by position and/or time. In some embodiments, the slice/image location need not correspond to a radial beam, and such amedical sensing system 100 supports cross-sectional slices located between the radial beams. In the embodiments ofFIG. 6 ,plane 308 represents an exemplary longitudinal location for the cross-sectional slice. Referring to block 504 ofFIG. 4 and referring still toFIG. 6 , themedical sensing system 100 identifies at least one revolution or frame (e.g., leading frame 602) ahead of thecross-sectional plane 308 in the longitudinal direction and at least one revolution or frame (e.g., lagging frame 604) behind thecross-sectional plane 308 in the longitudinal direction. In the procedure that follows, themedical sensing system 100 uses these two frames to interpolate imaging data that lies in thecross-sectional plane 308 rather than just flattening the acquired data. - Referring to block 506 of
FIG. 5A and referring still toFIG. 6 , the medical sensing system identifies apixel 606 within thecross-sectional plane 308. Thepixel 606 is a region of arbitrary size and may be identified by its cylindrical coordinates, a combination of the pixel's radial orientation (r), azimuthal orientation (θ), and longitudinal orientation (z). In the example ofFIG. 6 , the exemplaryradial pixel 606 is selected, and the cylindrical coordinates (r, θ, z) are indicated. As explained below, imaging data values at thepixel 606 may be determined from acquired data values at corresponding radial and azimuthal orientations by compensating for the difference in longitudinal orientation. - Referring to block 508 of
FIG. 5A and toFIGS. 6 and 7 , themedical sensing system 100 acquires a signal intensity at afirst point 608 in the first frame at a radial and azimuthal orientation that corresponds to the pixel 606 (e.g., the leading frame 602). Thefirst point 608 may be beyond theimaging plane 308 and thus offset from the pixel in a longitudinal direction (e.g., leading or lagging). The signal intensity may correspond to ultrasound echo data, OCT data, and/or any other type of sensing data and may be in any representation including baseband, radio-frequency/time domain, and/or other suitable representation. - Referring to block 510 of
FIG. 5A , themedical sensing system 100 determines the amount of longitudinal offset between thefirst point 608 and thepixel 606. In an exemplary embodiment with a known number of radial measurements per frame/revolution and a relatively constant pullback velocity, the longitudinal offset may be determined based on the number of radial measurements taken between theplane 308 of thepixel 606 and thefirst point 608. For example, if theplane 308 is known to coincide with radial measurement 0 and thefirst point 608 is acquired during radial measurement 128, the longitudinal offset may be determined by the difference of the two values. Accordingly, in such embodiments, the longitudinal offset may be expressed as a number of radial measurements, but may also be expressed as a distance value or time value. The longitudinal offset may be normalized so that the span of each frame is a fixed value, such as 1 or the number of radial measurements per frame. - Referring to block 512 of
FIG. 5A and toFIG. 7 , aweighting element 702 of themedical sensing system 100 determines a weighting factor for thefirst point 608 based on the longitudinal offset ofblock 510. The weighting factor reflects the certainty that a signal intensity at thepixel 606 corresponds to the signal intensity at thefirst point 608 and thus, is inversely proportional to the offset. In various embodiments, the weighting factor has a linear relationship and/or a higher-order polynomial relationship with the longitudinal offset. For example, the weighting factor may depend on the square of the longitudinal offset in order to place greater emphasis on values obtained near to thepixel 606. - Referring to block 514 of
FIG. 5A and toFIGS. 6 and 7 , themedical sensing system 100 acquires a signal intensity at asecond point 610 in the second frame (e.g., lagging frame 604) at a radial and azimuthal orientation that corresponds to thepixel 606. Thesecond point 610 may be beyond theimaging plane 308 and offset from thepixel 606 in the opposite longitudinal direction from thefirst point 608. Accordingly, thesecond point 610 may be thefirst point 608 delayed by a frame (or vice versa). The signal intensity may correspond to ultrasound echo data, OCT data, and/or any other type of sensing data and may be in any representation including baseband, radio-frequency/time domain, and/or other suitable representation. Referring to block 516 ofFIG. 5A , themedical sensing system 100 determines the amount of longitudinal offset between thesecond point 610 and thepixel 606 substantially as described inblock 510. Referring to block 518 ofFIG. 5A and toFIG. 7 , asecond weighting element 702 of themedical sensing system 100 determines a second weighting factor to thesecond point 610 based on the determined offset ofblock 516. The second weighting factor reflects the certainty that a signal intensity at thepixel 606 corresponds to the signal intensity at thesecond point 610 and thus, is inversely proportional to the offset. In various embodiments, the second weighting factor has a linear relationship and/or a higher-order polynomial relationship with the longitudinal offset. - Referring to block 520 of
FIG. 5B and toFIG. 7 , an averagingelement 704 of themedical sensing system 100 averages the weighted signal intensity at thefirst point 608 and the weighted signal intensity at thesecond point 610. This average signal intensity is an interpolated value and has at least some correlation to the signal intensity at thepixel 606. Accordingly, in some embodiments, the average signal intensity is used as the interpolated signal value at thepixel 606 without further processing. An image formed using average signal intensities may eliminate the seam found in other techniques and may reduce other distortion including curvature distortion. - However, the average signal intensity has a number of other artifacts, such as feature doubling, that can affect diagnosis. Feature doubling occurs when a strong reflective feature is located at a first position in a first frame and a second position in a second frame. The strong signal intensity dominates the weighted average so that the feature appears in both positions in the average. This may give the appearance of two separate and distinct features. Feature doubling may be caused by a stationary feature that extends along the longitudinal path but is somewhat skewed relative to the pullback path. It may also be caused by tissue moving faster than the frame rate and the under-sampling and the resultant aliasing. Furthermore, averaging is, in effect, a type of low pass filtering, and weak signal intensities are often obscured. As a result, the average signal intensity may exhibit resolution blurring and reduced sensitivity. These effects can complicate feature recognition, tissue characterization, tissue measurement, and other diagnostic efforts.
- As an alternative, in some embodiments, the
medical sensing system 100 uses the average signal intensity to determine a probability that the value at thepixel 606 closer to the signal intensity at thefirst point 608 than the second point 610 (or vice-versa). Referring to block 522 ofFIG. 5B and toFIG. 7 , afirst comparator 706 of themedical sensing system 100 determines the difference between the average signal intensity and the signal intensity at thefirst point 608 to determine a first probability indicator. In some embodiments, the magnitudes (i.e., absolute values) of the average signal intensity and the signal intensity at the first point are compared and the probability indicator is expressed as a magnitude. Referring to block 524 ofFIG. 5B and toFIG. 7 , asecond comparator 708 of the medical sensing system determines the difference between the average signal intensity and the signal intensity at thesecond point 610 to determine a second probability indicator. Here as well, in some embodiments, the magnitudes of the average signal intensity and the signal intensity at the second point are compared and the second probability indicator is expressed as a magnitude. - Referring to block 526 of
FIG. 5B and toFIG. 7 , athird comparator 710 of themedical sensing system 100 compares the first probability indicator to the second probability indicator, and the intensity at thepixel 606 is selected based on the comparison. In the embodiments ofFIG. 7 , a multiplexer selects the first signal intensity or the second signal intensity based on the result. In an example, the intensity at thefirst point 608 is selected to represent the intensity at thepixel 606 when the first probability indicator is greater, and the intensity at thesecond point 610 is selected to represent the intensity at thepixel 606 when the second probability indicator is greater. Referring to block 528, the selected signal intensity is provided for further signal processing, such as image filtering, envelope filtering, log compression, gamma curve adjustment, etc., in order to generate a diagnostic image. The processes of blocks 506-528 may be repeated for each pixel in the diagnostic image. -
FIG. 8 is a representation of ade-seamed image 800 produced by themedical sensing system 100 according to embodiments of the present disclosure. As evident inFIG. 8 , in many embodiments, themethod 500 produces pixel intensities such that the resulting image does not have a seam. The resultant images may also depict vascular curvature more accurately. Furthermore, unlike average signal intensities, the pixel values selected by themethod 500 do not exhibit feature doubling. Additionally, in many embodiments, the resultant image suffers no loss of resolution, blurring, or low-intensity filtering, which may occur from other seam-elimination techniques. As an additional benefit, themethod 500 may be applied to any representation of the signal intensities and does not require prior conditioning to convert the data to a format such as baseband, radio-frequency (RF)/time domain, or any other representation. - Persons skilled in the art will recognize that the apparatus, systems, and methods described above can be modified in various ways. Accordingly, persons of ordinary skill in the art will appreciate that the embodiments encompassed by the present disclosure are not limited to the particular exemplary embodiments described above. In that regard, although illustrative embodiments have been shown and described, a wide range of modification, change, and substitution is contemplated in the foregoing disclosure. It is understood that such variations may be made to the foregoing without departing from the scope of the present disclosure. Accordingly, it is appropriate that the appended claims be construed broadly and in a manner consistent with the present disclosure.
Claims (20)
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US14/964,315 US9629553B2 (en) | 2014-12-19 | 2015-12-09 | Seam elimination and motion compensation in imaging data |
| US15/495,684 US9918699B2 (en) | 2014-12-19 | 2017-04-24 | Seam elimination and motion compensation in imaging data |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US201462094303P | 2014-12-19 | 2014-12-19 | |
| US14/964,315 US9629553B2 (en) | 2014-12-19 | 2015-12-09 | Seam elimination and motion compensation in imaging data |
Related Child Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US15/495,684 Continuation US9918699B2 (en) | 2014-12-19 | 2017-04-24 | Seam elimination and motion compensation in imaging data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| US20160180519A1 true US20160180519A1 (en) | 2016-06-23 |
| US9629553B2 US9629553B2 (en) | 2017-04-25 |
Family
ID=56130020
Family Applications (2)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US14/964,315 Active US9629553B2 (en) | 2014-12-19 | 2015-12-09 | Seam elimination and motion compensation in imaging data |
| US15/495,684 Active US9918699B2 (en) | 2014-12-19 | 2017-04-24 | Seam elimination and motion compensation in imaging data |
Family Applications After (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| US15/495,684 Active US9918699B2 (en) | 2014-12-19 | 2017-04-24 | Seam elimination and motion compensation in imaging data |
Country Status (1)
| Country | Link |
|---|---|
| US (2) | US9629553B2 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109964246A (en) * | 2016-11-16 | 2019-07-02 | 皇家飞利浦有限公司 | Adaptive Halo Reduction for Coronary and Peripheral Intravascular Ultrasound (IVUS) |
| CN114066889A (en) * | 2022-01-12 | 2022-02-18 | 广州永士达医疗科技有限责任公司 | Imaging quality detection method and device of OCT (optical coherence tomography) host |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6148095A (en) * | 1997-09-08 | 2000-11-14 | University Of Iowa Research Foundation | Apparatus and method for determining three-dimensional representations of tortuous vessels |
| US20080287795A1 (en) * | 2002-08-26 | 2008-11-20 | Volcano Corporation | System And Method For Identifying A Vascular Border |
| US20120059253A1 (en) * | 2010-06-29 | 2012-03-08 | Siemens Aktiengesellschaft | Method and System for Image Based Device Tracking for Co-registration of Angiography and Intravascular Ultrasound Images |
| US20140039308A1 (en) * | 2012-08-02 | 2014-02-06 | Volcano Corporation | Method for Seam Elimination and Reconstruction of Coplanar Images from Intravascular Ultrasonic Data |
| US8727990B2 (en) * | 2009-11-19 | 2014-05-20 | Samsung Medison Co., Ltd. | Providing an ultrasound spatial compound image in an ultrasound system |
| US9480403B2 (en) * | 2013-12-16 | 2016-11-01 | General Electric Company | Medical imaging system and method for generating a blended cine loop |
-
2015
- 2015-12-09 US US14/964,315 patent/US9629553B2/en active Active
-
2017
- 2017-04-24 US US15/495,684 patent/US9918699B2/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6148095A (en) * | 1997-09-08 | 2000-11-14 | University Of Iowa Research Foundation | Apparatus and method for determining three-dimensional representations of tortuous vessels |
| US20080287795A1 (en) * | 2002-08-26 | 2008-11-20 | Volcano Corporation | System And Method For Identifying A Vascular Border |
| US8727990B2 (en) * | 2009-11-19 | 2014-05-20 | Samsung Medison Co., Ltd. | Providing an ultrasound spatial compound image in an ultrasound system |
| US20120059253A1 (en) * | 2010-06-29 | 2012-03-08 | Siemens Aktiengesellschaft | Method and System for Image Based Device Tracking for Co-registration of Angiography and Intravascular Ultrasound Images |
| US20140039308A1 (en) * | 2012-08-02 | 2014-02-06 | Volcano Corporation | Method for Seam Elimination and Reconstruction of Coplanar Images from Intravascular Ultrasonic Data |
| US9480403B2 (en) * | 2013-12-16 | 2016-11-01 | General Electric Company | Medical imaging system and method for generating a blended cine loop |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109964246A (en) * | 2016-11-16 | 2019-07-02 | 皇家飞利浦有限公司 | Adaptive Halo Reduction for Coronary and Peripheral Intravascular Ultrasound (IVUS) |
| CN114066889A (en) * | 2022-01-12 | 2022-02-18 | 广州永士达医疗科技有限责任公司 | Imaging quality detection method and device of OCT (optical coherence tomography) host |
Also Published As
| Publication number | Publication date |
|---|---|
| US9918699B2 (en) | 2018-03-20 |
| US20170224313A1 (en) | 2017-08-10 |
| US9629553B2 (en) | 2017-04-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10269096B2 (en) | Clutter suppression for synthetic aperture ultrasound | |
| US20230293149A1 (en) | Phased array intravascular devices, systems, and methods utilizing photoacoustic and ultrasound techniques` | |
| US9693754B2 (en) | Imaging processing systems and methods | |
| US12201474B2 (en) | Tissue and vascular pathway mapping using synchronized photoacoustic and ultrasound pullback techniques | |
| US12178640B2 (en) | Visualization of reflectors in intraluminal ultrasound images and associated systems, methods, and devices | |
| US20210022714A1 (en) | Non-uniform rotation distortion (nurd) reduction in ultrasound imaging devices, systems, and methods | |
| US11559207B2 (en) | Rotational intravascular devices, systems, and methods utilizing photoacoustic and ultrasound imaging techniques | |
| US9918699B2 (en) | Seam elimination and motion compensation in imaging data | |
| EP3860466B1 (en) | Fluid flow detection for ultrasound imaging devices, systems, and methods | |
| US10159467B2 (en) | Method for seam elimination and reconstruction of coplanar images from intravascular ultrasonic data | |
| EP2082369B1 (en) | Systems and methods for restoring a medical image affected by nonuniform rotational distortion | |
| CN113196094B (en) | Grating lobe artifact minimization and associated devices, systems, and methods for ultrasound images | |
| EP3435876B1 (en) | Tissue and vascular pathway mapping utilizing photoacoustic and ultrasound techniques | |
| KR101060386B1 (en) | Ultrasound system and method for forming elastic images |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| AS | Assignment |
Owner name: VOLCANO CORPORATION, CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:HANCOCK, ANDREW;REEL/FRAME:038025/0117 Effective date: 20160316 |
|
| STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
| MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |
|
| MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |