[go: up one dir, main page]

EP3447525B1 - Procédé de reconstruction d'image basée sur un modèle - Google Patents

Procédé de reconstruction d'image basée sur un modèle Download PDF

Info

Publication number
EP3447525B1
EP3447525B1 EP17187412.6A EP17187412A EP3447525B1 EP 3447525 B1 EP3447525 B1 EP 3447525B1 EP 17187412 A EP17187412 A EP 17187412A EP 3447525 B1 EP3447525 B1 EP 3447525B1
Authority
EP
European Patent Office
Prior art keywords
measurement
reflectivity
hypersurface
values
grid
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP17187412.6A
Other languages
German (de)
English (en)
Other versions
EP3447525A1 (fr
Inventor
Adrien BESSON
Dimitris Perdios
Marcel Arditi
Jean-Philippe Thiran
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ecole Polytechnique Federale de Lausanne EPFL
Original Assignee
Ecole Polytechnique Federale de Lausanne EPFL
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to ES17187412T priority Critical patent/ES2812649T3/es
Application filed by Ecole Polytechnique Federale de Lausanne EPFL filed Critical Ecole Polytechnique Federale de Lausanne EPFL
Priority to EP17187412.6A priority patent/EP3447525B1/fr
Priority to US16/641,234 priority patent/US11175392B2/en
Priority to JP2020511196A priority patent/JP6924530B2/ja
Priority to KR1020207008376A priority patent/KR102326149B1/ko
Priority to PCT/EP2018/072579 priority patent/WO2019038296A1/fr
Priority to CN201880066558.6A priority patent/CN111213066B/zh
Publication of EP3447525A1 publication Critical patent/EP3447525A1/fr
Priority to IL272819A priority patent/IL272819B/en
Application granted granted Critical
Publication of EP3447525B1 publication Critical patent/EP3447525B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/52—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00
    • G01S7/52017—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00 particularly adapted to short-range imaging
    • G01S7/52023—Details of receivers
    • G01S7/52025—Details of receivers for pulse systems
    • G01S7/52026—Extracting wanted echo signals
    • G01S7/52028—Extracting wanted echo signals using digital techniques
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/88—Sonar systems specially adapted for specific applications
    • G01S15/89—Sonar systems specially adapted for specific applications for mapping or imaging
    • G01S15/8906—Short-range imaging systems; Acoustic microscope systems using pulse-echo techniques
    • G01S15/8909—Short-range imaging systems; Acoustic microscope systems using pulse-echo techniques using a static transducer configuration
    • G01S15/8915—Short-range imaging systems; Acoustic microscope systems using pulse-echo techniques using a static transducer configuration using a transducer array
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
    • G01S15/88—Sonar systems specially adapted for specific applications
    • G01S15/89—Sonar systems specially adapted for specific applications for mapping or imaging
    • G01S15/8906—Short-range imaging systems; Acoustic microscope systems using pulse-echo techniques
    • G01S15/8977—Short-range imaging systems; Acoustic microscope systems using pulse-echo techniques using special techniques for image reconstruction, e.g. FFT, geometrical transformations, spatial deconvolution, time deconvolution
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/02—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S13/00
    • G01S7/28—Details of pulse systems
    • G01S7/285—Receivers
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/4802—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00 using analysis of echo signal for target characterisation; Target signature; Target cross-section
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
    • G01S7/483—Details of pulse systems
    • G01S7/486—Receivers
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
    • G01S7/52—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00
    • G01S7/52017—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S15/00 particularly adapted to short-range imaging
    • G01S7/52085—Details related to the ultrasound signal acquisition, e.g. scan sequences

Definitions

  • the present invention relates to a model-based image reconstruction method suitable to be used for example in ultrasound imaging.
  • the invention also relates to an imaging apparatus for carrying out the method.
  • Pulse wave imaging applications such as ultrasound (US) imaging applications, may be used in various technical fields. In medical applications, it is used to see internal body structures such as tendons, muscles, joints, vessels and internal organs. However, pulse wave imaging techniques are also widely used outside the medical field.
  • NDT non-destructive testing
  • An example of an NDT would be in oil pipes.
  • ultrasound waves are used to propagate down the pipe, and in this manner, it is possible to detect if there is a crack or defect as a reflection of the pulse. With knowledge of the speed of the wave in the pipe and the time interval between the reflection from the crack and sending the waves, the position of the defect can be estimated.
  • the same theory used for crack detection in pipes can be used to detect cracks in aircraft structures/wing panels etc and to detect the build-up of ice on the surface of the wing for example.
  • pulse wave imaging has gained popularity, because of its safety, portability and real-time capability.
  • the proposed method has the advantage that the reconstructed image has a very high quality and that it can be reconstructed quickly. In other words, compared to prior art solutions, image contrast and resolution can be significantly improved.
  • an imaging apparatus configured to carry out the method according to the first aspect of the invention as recited in claim 15.
  • the proposed image formation method relies on two main pillars: a fast and matrix-free measurement model H and an image reconstruction method which permits to retrieve an estimate of the object under scrutiny ⁇ given the measurements m.
  • the object may be an internal part or element of a larger structure.
  • the proposed method can be carried out by a standard US system comprising a US probe, an image formation module and a post-processing and display modules, as described in Figure 1 .
  • FIG. 1 is simplified block diagram showing some elements of the imaging system or apparatus 1, which are useful for understanding the teachings of the present invention.
  • the system 1 comprises an ultrasound probe 3, which in this example comprises a linear array of piezoelectric elements or sensors, also referred to as transducers.
  • Such a probe is suitable for the purpose of imaging in 2D, and may include a cylindrical focusing scheme to concentrate the acoustic energy, in elevation, in the vicinity of the image plane.
  • this kind of probe is merely an example.
  • the proposed method could be easily adapted to different probe geometries (such as convex probes, phased arrays etc) or technologies (including polyvinylidene fluoride (PVDF) copolymers and capacitive micromachined ultrasonic transducers (CMUT) etc).
  • the proposed method can be extended to cases using 2D matrix probes, designed to transmit acoustic beams and collect echoes from the insonified volume in order to provide volumetric imaging, whereby the imaging process produces a 3D representation of the volume under scrutiny.
  • the US probe 3 is connected to an image formation module or apparatus 5, which is configured to carry out the image formation.
  • a measurement model estimated in a measurement model unit 7 is used in the image formation process.
  • the proposed measurement model as explained later in detail is based on a pulse-echo spatial impulse response model.
  • the present invention introduces matrix-free formulations of the measurement model and its adjoint, the adjoint of an operator being defined as a continuous extension of the transpose of a square matrix in the case of a continuous operator, which represent a novel image formation method in pulse wave imaging.
  • the description of the measurement model may be achieved through two main steps:
  • the model may be derived for example for a PW, DW and SA imaging. It is compatible with radio frequency and in-phase and quadrature (IQ) data. Finally, a matrix-free formulation of the adjoint of the proposed measurement model is derived. It will be demonstrated that it can also be recast as an integral over hypersurfaces, defined as 1D manifolds in the case of 2D imaging and as 2D manifolds in the case of 3D imaging, that are parametrised and that the same discretisation scheme as proposed for the measurement model may be used.
  • the image formation apparatus 5 also comprises an image reconstruction unit 9 for running an image reconstruction process or method.
  • the image reconstruction method aims to retrieve an estimate ⁇ of the image under scrutiny.
  • Two different image reconstruction methods are presented in the present description:
  • the post-processing step covers a range of applications, such as B-mode imaging, colour Doppler imaging, vector Doppler imaging and elastography.
  • an envelope detection is applied onto the reconstructed data.
  • the envelope detection can, for instance, be achieved by means of the Hilbert transform followed by a magnitude detection and optional low-pass filtering. It can also be achieved by squaring and low-pass filtering the signal. If B-mode imaging is used with IQ data, the magnitude of the signal is extracted.
  • the envelope detection step is followed by a normalisation and dynamics compression steps. For Doppler imaging and elastography, the reconstructed RF or IQ data are directly used without post-processing.
  • the proposed method defines a first grid and a second grid which both lie in the 3D space.
  • the first grid is referred to as a reflectivity grid, which comprises reflectivity grid points at intersections of two crossing grid lines, which in this example form an angle of substantially 90 degrees with respect to each other. Every reflectivity grid point of the reflectivity grid belongs to the medium ⁇ .
  • the second grid is referred to as a measurement grid, which comprises measurement grid points at intersections of two crossing grid lines, which in this example form an angle of substantially 90 degrees with respect to each other.
  • the measurement grid points coincide with the locations of the receiving sensors, in one dimension, and with the sampling time instants of each sensor in the other dimension.
  • the measurement model used in this disclosure is a pulse-echo spatial impulse response model first introduced by G. E. Tupholme, "Generation of acoustic pulses by baffled plane pistons", Mathematika, vol. 16, p. 209, 1969 , and P. R. Stepanishen, "The Time-Dependent Force and Radiation Impedance on a Piston in a Rigid Infinite Planar Baffle", J. Acoust. Soc. Am., vol. 49, p. 76, 1971 . According to this model, one can write the below equations which account for the raw data sample recorded at position p and time t.
  • ⁇ ( p,q ,t ) defines a conic when r lies in the 2D plane (parabola or ellipse) and a quadric when r lies in a 3D volume.
  • Figure 3 schematically illustrates the measurement configuration in continuous domain showing also the reflectivity hypersurface.
  • the reflectivity hypersurface ⁇ ( p,q ,t ) can now be parametrised in order to reduce the integration dimension.
  • parametric equations define a set of quantities as functions of one or more independent variables (ie parameters).
  • Parametric equations are commonly used to express the coordinates of the points that make up a geometric object such as a surface or curve, in which case the equations are collectively called a parametric representation or parameterisation of the object.
  • the equation on the right of the equivalence defines the reflectivity parametric equations of r since r now depends on a parameter ⁇ ⁇ A, where A c A ⁇ R 2 if r ⁇ R 2 and A ⁇ R when r ⁇ R 2 . It is to be noted if there is one set of reflectivity parametric equations per measurement value or sample. Furthermore, the hypersurface can be obtained by drawing the set of reflectivity parametric equations.
  • a continuous measurement model for a configuration of one transmitting finite length element of arbitrary shape and one receiving point source sensor is explained next.
  • the transmitting element has a finite length and an arbitrary shape. It is characterised by a continuous set of point source locations X j .
  • m p t ⁇ ⁇ ⁇ R , q ⁇ X j , ⁇ ⁇ A o d r ⁇ p q ⁇ , p , q ⁇ r ⁇ p q ⁇ J ⁇ p q ⁇ v pe t ⁇ ⁇ d ⁇ d ⁇ q d ⁇ .
  • a continuous measurement model for a configuration of one transmitting finite length element of arbitrary shape and one receiving finite length sensor is explained next.
  • the receiving sensor has a finite length and arbitrary shape. It is defined by the set of point source locations ⁇ i .
  • the coordinate of the resulting signal is denoted by m ( ⁇ i ,t ) , where ⁇ i ⁇ ⁇ i . It can be for example the coordinate of the point in the centre of ⁇ i .
  • m ( ⁇ i ,t ) ⁇ p ⁇ i m ( p ,t ) d ⁇ ( p ) .
  • the resulting measurements are obtained by integrating along the transducer surface defined by ⁇ i .
  • m ⁇ i t ⁇ p ⁇ ⁇ i , q ⁇ X j , r ⁇ ⁇ o d r p q ⁇ r v pe t ⁇ t Tx r q ⁇ t Rx r p d ⁇ r ⁇ d ⁇ q d ⁇ p .
  • m ⁇ i t ⁇ ⁇ ⁇ R , p ⁇ ⁇ i , q ⁇ X j , ⁇ ⁇ A o d r ⁇ p q ⁇ , p , q ⁇ r ⁇ p q ⁇ ⁇ p q ⁇ v pe t ⁇ ⁇ d ⁇ d ⁇ q d ⁇ p d ⁇ .
  • a continuous measurement model for a configuration of multiple transmitting finite length elements of arbitrary shape and one receiving finite length sensor of arbitrary shape is explained next.
  • the continuous parametric formulation of the measurement model is next discretised.
  • This step consists in discretising the parametric formulation of the measurement model described in connection with the configuration of multiple transmitting finite length elements of arbitrary shape and one receiving finite length sensor of arbitrary shape.
  • any of the models described above could be discretised instead.
  • the above equation comprises a set of reflectivity values ⁇ r ⁇ p p n q k t l ⁇ p ⁇ A d j , referred to as hypersurface reflectivity samples or values, which are estimated on the points of the reflectivity hypersurface.
  • the hypersurface reflectivity values are next interpolated to substantially coincide with the reflectivity grid points.
  • the hypersurface reflectivity values are usually unknown since they do not lie on the reflectivity grid points. Thus, they need to be approximated from the reflectivity samples defined on the reflectivity grid by means of an interpolation kernel as introduced by P. Thévenaz, T. Blu and M. Republic, "Interpolation revisited", IEEE Transactions on Medical Imaging, vol. 19, p. 739-758, 2000 .
  • the above interpolation equation relates the hypersurface reflectivity values ⁇ ( r( ⁇ p , p n , q k ,t l )) (which are typically not on the reflectivity grid points) to the reflectivity samples ⁇ s .
  • a summation of the discretised and interpolated hypersurface reflectivity samples or values is carried out next.
  • the output of the summation is eventually convolved with the pulse shape, as described in the previous equations in order to obtain an estimate of the measurement value, also referred to as a measurement value estimate, at the considered measurement grid point.
  • the above described process is carried out for every measurement value.
  • the outcome of the estimation or processing of the measurement model is the estimate of the measurement value at the respective measurement grid point.
  • the estimation, evaluation, computation or implementation of the adjoint operator of the measurement model at each reflectivity grid point is carried out next.
  • a continuous parametric formulation of the adjoint operator of the measurement model is obtained.
  • the continuous formulation of the adjoint operator can be derived from the measurement model, ie from example from any of the continuous measurement models given above, using functional analysis tools.
  • the equations below are derived from the case of multiple finite length transmitting elements of arbitrary shape and multiple finite length receiving sensors of arbitrary shape. Indeed, if we define an operator such that then the adjoint operator of is defined as:
  • the parametric formulation of the adjoint operator of the measurement model is next discretised.
  • the same process is applied as for the discretisation of the measurement model.
  • the discretisation process is more direct since it is directly applied onto p.
  • the discretised set of parameters is ⁇ d i .
  • the above equation comprises a set of convolved measurement samples m ⁇ i ⁇ u t Tx r q k + t Rx r p n p n ⁇ ⁇ d i , referred to as hypersurface measurement samples or values, which lie on the measurement hypersurface.
  • the above equation relates the hypersurface measurement samples ( m ( ⁇ i ) ⁇ u )( t Tx ( r,q k ) + t Rx ( r,p n )) to the convolved measurement samples ( m ( ⁇ i ) ⁇ u )( t l ) .
  • the output of the summation is the estimate of the reflectivity value at the considered reflectivity grid point.
  • the image reconstruction procedure is explained next.
  • the inverse problem is defined first.
  • the measurement values m can be used to generate an estimate of the reflectivity values ⁇ using the adjoint operator of the measurement model as described above.
  • an estimate of the reflectivity values ⁇ can be used to generate an estimate of the measurement values m ⁇ , which are not equal to m, using the measurement model described earlier.
  • the image reconstruction method aims to retrieve an estimate ⁇ of the reflectivity values evaluated at the reflectivity grid points, given the set of measurements values m.
  • the relationship between the measurement values and the unknown reflectivity values is defined by the following linear inverse problem: which corresponds to the measurement model estimated at each point of the measurement grid.
  • optimisation problem associated with the inverse problem is explained next.
  • the objective function involved in the optimisation problem is a lower semicontinuous functional accounting for the data discrepancy term, measuring the distance between the estimate of the measurement values H ⁇ and the measurement values m, is a lower semicontinuous functional describing an optional prior term, which accounts for additional information such as a specific statistical behaviour, on the reflectivity values and ⁇ > 0 is a regularisation parameter.
  • the problem involves the computation of H in the calculation of the derivative of .
  • the calculation of the derivative may also involve the computation of the adjoint operator of the measurement model H.
  • ⁇ (I + H ⁇ H ) -1 H ⁇ m , where I ⁇ R N ⁇ ⁇ N ⁇ is the identity matrix, the N ⁇ ⁇ N ⁇ square matrix with ones on the main diagonal and zeros elsewhere.
  • Projected gradient methods exploit the fact that the solution of the optimisation problem satisfies a fixed-point equation.
  • FISTA fast iterative shrinkage-thresholding algorithm
  • ⁇ n + 1 ⁇ soft ⁇ n ⁇ ⁇ 2 ⁇ n H ⁇ H ⁇ n ⁇ ⁇ m , ⁇ n , where ⁇ n ⁇ R + is a hyper-parameter.
  • - ⁇ ,0), where sign( x i ) is equal to 1 when x i > 0, 0 when x i 0 and -1 otherwise.
  • ADMM alternating direction method of multipliers
  • N. Parikh alternating direction method of multipliers
  • E. Chu alternating direction method of multipliers
  • B. Peleato primal-dual forward-backward
  • PDFB primal-dual forward-backward
  • a set of pulse waves is transmitted by a set of transmitting elements.
  • the transmitting elements have a finite size and arbitrary shape.
  • the set of transmitted pulses may comprise only one pulse or more than one pulses transmitted in a sequence.
  • a set of reflected echo waveforms is received by a set of sensors, which in this example have a finite size and arbitrary shape.
  • the inverse problem relating the set of measurement values to the set of reflectivity values is defined.
  • the solution of the inverse problem is expressed as an optimisation problem involving an objective function comprising a data discrepancy term and one or more image prior terms.
  • the optimisation problem is solved by using projections to obtain an estimate of the reflectivity values.
  • the flow chart of Figure 6 outlines the proposed method in more detail in the case where the functional is differentiable.
  • the first four steps, ie steps 31, 33, 35 and 37 are the same as the first four steps in the flow chart of Figure 6 .
  • the derivative ⁇ F ⁇ ⁇ v n m is computed.
  • the adjoint operator is used to compute the derivative of the data discrepancy term.
  • step 47 the proximity operators of the image prior terms are computed.
  • step 49 ⁇ n ⁇ is updated.
  • step 50 it is determined whether or not a convergence criterion is fulfilled. In the affirmative, the process comes to an end. If the criterion is not fulfilled, then the process continues in step 41.
  • the flow chart of Figure 7 outlines the proposed method in more detail in the case where the data discrepancy term is the Euclidean distance and where the inverse problem contains one image prior term.
  • the first five steps, ie steps 51, 53, 55, 57, 59 are substantially the same as the first five steps in the flow chart of Figure 6 with the difference that now the inverse problem comprises one image prior term.
  • the adjoint operator of the measurement model on the residuals H ⁇ r n is estimated according to the flow chart of Figure 10.
  • step 67 the value ⁇ n ⁇ ⁇ ⁇ n H ⁇ r n is computed.
  • the projection on the image prior term ⁇ n + 1 ⁇ prox ⁇ n R ⁇ n ⁇ ⁇ ⁇ n H ⁇ r n is computed.
  • step 70 it is determined whether or not a convergence criterion is fulfilled. In the affirmative, the process comes to an end. If the criterion is not fulfilled, then the process continues in step 61.
  • a set of reflectivity parametric equations are generated for each measurement grid point (defining a finite size receiving sensor of arbitrary shape), for each point source location in the finite size receiver of arbitrary shape corresponding to the measurement grid point, for each finite size transmitter of arbitrary shape and for each point source location in the finite size transmitter of arbitrary shape.
  • a set of reflectivity parametric equations is generated for each transmitter-receiver wave path between every pair of the point sources in each transmitter and each receiver.
  • the reflectivity parametric equations define a set of reflectivity hypersurfaces.
  • the reflectivity hypersurface is discretised to obtain a set of hypersurface reflectivity samples.
  • step 75 the hypersurface reflectivity samples are interpolated on the reflectivity grid.
  • step 77 the interpolated and discretised hypersurface reflectivity samples are summed.
  • step 79 the value obtained in step 77 is added to the measurement value estimate at the point of the measurement grid.
  • step 80 the obtained accumulated measurement value estimate is convolved with the pulse shape.
  • the flow chart of Figure 9 explains an example process of estimating or processing the adjoint operator of the measurement model.
  • the measurement value estimates are convolved with the matched filter of the pulse shape.
  • a set of measurement parametric equations are generated for each point of the reflectivity grid, for each finite size transmitter of arbitrary shape, for each finite size receiver of arbitrary shape, for each point source location in the finite size transmitter of arbitrary shape and for each point source location in the finite size receiver of arbitrary shape.
  • a set of measurement parametric equations is generated for each transmitter-receiver path between every pair of the point sources in each transmitter and each receiver.
  • the measurement parametric equations define a set of measurement hypersurfaces.
  • step 85 the measurement hypersurface is discretised to obtain a set of hypersurface measurement samples.
  • step 87 the hypersurface measurement samples are interpolated on the measurement grid.
  • step 89 the interpolated and discretised hypersurface measurement samples are summed.
  • step 90 the value obtained in step 89 is added to the reflectivity value estimate at the point of the reflectivity grid.
  • the transmitted pulse waves can be generated by at least one electromechanical transduction device excited by an electrical signal.
  • the measurement values can be obtained from an electrical signal generated by at least one reciprocal electromechanical transduction device.
  • the electromechanical transduction devices are spatially arranged into: one or more linear array probes with a multi-element structure with a substantially one-wavelength-pitch, and aligned along a straight, convex or concave line; one or more phased array probes with a multi-element structure with a substantially half-wavelength-pitch, and aligned along a straight, convex or concave line; one or more matrix array probes with a multi-element structure with a substantially one-wavelength-pitch, and aligned on a planar surface; or one or more matrix array probes with a multi-element structure with a substantially half-wavelength-pitch, and aligned on a planar surface.
  • the teachings of the present invention can be applied to photoacoustic imaging, where the transmit acoustic waves are replaced with electromagnetic-radiation pulses.
  • the whole formalism introduced in the present description can be simplified by setting all acoustic transmission propagation delays to zero, because electromagnetic radiation propagates at a speed close to the speed of light, ie typically five orders of magnitude faster than the speed of sound. For this reason, the transmission propagation delays can thus be neglected when accounting the acoustic propagation delays on reception.
  • Another modification considered in the present invention is in the use of transducer technologies different from piezoelectric materials, such as capacitive machined ultrasound transducers (CMUTs).
  • CMUTs capacitive machined ultrasound transducers
  • Yet another modification is to apply the invention in a time-multiplexed scheme, whereby subsets of transmitting and/or receiving transducer elements are successively addressed in sequence, and the resulting signals are stored for subsequent recombination.
  • All the above applications can also be translated to microbubble based contrast specific nonlinear imaging techniques known in the art, such as pulse inversion, amplitude modulation, amplitude modulation pulse inversion, harmonic imaging etc.
  • the present invention also proposes a computer program product comprising instructions stored on a non-transitory medium for implementing the steps of the method as explained above when loaded and run on computing means of an electronic data processing device, such as the imaging apparatus 1.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • General Physics & Mathematics (AREA)
  • Acoustics & Sound (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)
  • Investigating Or Analyzing Materials By The Use Of Ultrasonic Waves (AREA)
  • Investigating Or Analysing Materials By Optical Means (AREA)

Claims (15)

  1. Un procédé de reconstruction d'image d'onde de pouls comprenant :
    • la transmission (21) d'un ensemble d'ondes de pouls à un objet à reconstruire au moins partiellement au moyen d'un ensemble d'éléments de transmission (13) ;
    • la réception (23) d'un ensemble de formes d'ondes d'écho réfléchies, au moyen d'un ensemble de capteurs (13), à partir de l'objet caractérisé par un ensemble de valeurs de réflectivité définies sur une grille de réflectivité comprenant des points de grille de réflectivité, l'ensemble de formes d'ondes d'écho définissant un ensemble de valeurs de mesure définies sur une grille de mesure comprenant des points de grille de mesure ;
    • la définition (25) d'un problème inverse pour obtenir un ensemble d'estimations de valeur de réflectivité à partir de l'ensemble de valeurs de mesure au moyen d'un modèle de mesure ;
    • l'expression (27) du problème inverse comme un problème d'optimisation impliquant une fonction objective comprenant un terme de divergence de données ;
    • l'expression (27) du problème d'optimisation comme un système d'équations impliquant des contributions du terme de divergence de données ;
    • l'obtention (29) de l'ensemble d'estimations de valeur de réflectivité en effectuant des itérations dans le système d'équations jusqu'à ce qu'un seuil de convergence donné soit atteint, au moins certaines des itérations impliquant au moins une estimation de valeur de mesure pour chaque point de grille de mesure d'un ensemble de points de grille de mesure et une estimation de valeur de réflectivité pour chaque point de grille de réflectivité d'un ensemble de points de grille de réflectivité ;
    le procédé étant caractérisé en ce que l'obtention des contributions dans le système d'équations comprend :
    • l'obtention de l'estimation de valeur de mesure pour chaque point de grille de mesure de l'ensemble de points de grille de mesure à partir du modèle de mesure, comprenant :
    • la génération (71) d'un ensemble d'équations paramétriques de réflectivité par élément de transmission - trajet du capteur définissant un ensemble d'hypersurfaces de réflectivité comprenant des valeurs de réflectivité d'hypersurface ;
    • l'intégration (77) des valeurs de réflectivité d'hypersurface sur chaque hypersurface de réflectivité de l'ensemble d'hypersurfaces de réflectivité pour obtenir l'estimation de valeur de mesure pour le point de grille de mesure respectif ;
    • l'obtention de l'estimation de valeur de réflectivité pour chaque point de grille de réflectivité de l'ensemble de points de grille de réflectivité à partir d'un opérateur adjoint du modèle de mesure, comprenant :
    • la génération (83) d'un ensemble d'équations paramétriques de mesure par élément de transmission - trajet du capteur définissant un ensemble d'hypersurfaces de mesure comprenant des valeurs de mesure d'hypersurface ;
    • l'intégration (89) des valeurs de mesure d'hypersurface sur chaque hypersurface de mesure de l'ensemble d'hypersurfaces de mesure pour obtenir l'estimation de valeur de réflectivité pour le point de grille de réflectivité respectif.
  2. Le procédé selon la revendication 1, dans lequel chaque élément de transmission (13) de l'ensemble d'éléments de transmission est considéré comme un ensemble de sources ponctuelles de transmission, tandis que chaque capteur (13) de l'ensemble de capteurs est considéré comme un ensemble de sources ponctuelles de réception.
  3. Le procédé selon la revendication 1 ou 2, dans lequel, avant d'intégrer les valeurs de réflectivité d'hypersurface, le procédé comprend en outre :
    • la discrétisation (73) de chaque hypersurface de réflectivité de l'ensemble d'hypersurfaces de réflectivité pour obtenir les valeurs de réflectivité d'hypersurface ;
    • l'interpolation (75) des valeurs de réflectivité d'hypersurface discrétisées pour coïncider sensiblement avec les points de grille de réflectivité.
  4. Le procédé selon l'une quelconque des revendications précédentes, dans lequel, avant d'intégrer les valeurs de mesure d'hypersurface, le procédé comprend en outre :
    • la discrétisation (85) de chaque hypersurface de mesure de l'ensemble d'hypersurfaces de mesure pour obtenir les valeurs de mesure d'hypersurface ;
    • l'interpolation (87) des valeurs de mesure d'hypersurface discrétisées pour coïncider sensiblement avec les points de la grille de mesure.
  5. Le procédé selon l'une quelconque des revendications précédentes, dans lequel le terme de divergence de données est exprimé comme une fonction positive évaluant une distance entre la valeur de mesure et l'estimation de valeur de mesure pour chaque point de grille de mesure de l'ensemble de points de grille de mesure.
  6. Le procédé selon la revendication 5, dans lequel la fonction positive est une fonction différentiable positive.
  7. Le procédé selon l'une quelconque des revendications précédentes, dans lequel la fonction objective est exprimée sous la forme d'une combinaison du terme de divergence de données et d'au moins un terme antérieur d'image, le terme antérieur d'image comprenant des informations préalables sur les valeurs de réflectivité.
  8. Le procédé selon la revendication 7, dans lequel le au moins un terme antérieur d'image est exprimé comme la norme ℓ p à la puissance de p de la représentation de l'ensemble d'estimations de valeur de réflectivité dans un modèle donné, la norme ℓ p étant définie pour un vecteur a ∈ ℝ N
    Figure imgb0128
     comme ∥ a ∥ p = ∑ i = 1 N a i p p ,
    Figure imgb0129
     où
    Figure imgb0130
    désigne l'espace des nombres réels.
  9. Le procédé selon l'une quelconque des revendications précédentes, dans lequel les ondes de pouls sont des ondes acoustiques de pouls ou des ondes de rayonnement électromagnétique.
  10. Le procédé selon l'une quelconque des revendications précédentes, dans lequel les ondes de pouls comprennent un ensemble d'ondes planes dirigées ou un ensemble d'ondes divergentes.
  11. Le procédé selon l'une quelconque des revendications précédentes, dans lequel les ondes de pouls sont générées par au moins un dispositif de transduction électromécanique excité par un signal électrique, et les valeurs de mesure sont obtenues à partir d'un signal électrique généré par au moins un dispositif de transduction électromécanique réciproque.
  12. Le procédé selon la revendication 11, dans lequel les dispositifs de transduction électromécaniques sont disposés dans l'espace en
    • une sonde réseau linéaire avec une structure à plusieurs éléments avec un pas sensiblement d'une longueur d'onde, et alignée le long d'une ligne droite, convexe ou concave ;
    • une sonde réseau à commande de phase avec une structure à plusieurs éléments avec un pas sensiblement d'une demi-longueur d'onde, et alignée le long d'une ligne droite, convexe ou concave ;
    • une sonde réseau matricielle avec une structure à plusieurs éléments avec un pas sensiblement d'une longueur d'onde, et alignée sur une surface plane; ou
    • une sonde réseau matricielle avec une structure à plusieurs éléments avec un pas sensiblement d'une demi-longueur d'onde, et alignée sur une surface plane.
  13. Le procédé selon l'une quelconque des revendications précédentes, dans lequel les hypersurfaces de réflectivité et de mesure ont une forme conique lorsque l'ensemble de valeurs de réflectivité se trouve dans un espace 2D, et une forme quadrique lorsque l'ensemble de valeurs de réflectivité se trouve dans un espace 3D.
  14. Le procédé selon l'une quelconque des revendications précédentes, dans lequel l'intégration des valeurs de mesure ou de réflectivité d'hypersurface est réalisée au moyen d'une intégration numérique, et dans lequel le procédé comprend en outre : l'ajout (79) de l'estimation de la valeur de mesure obtenue à l'estimation de valeur de mesure précédente au point de grille de mesure respectif pour obtenir une estimation de valeur de mesure accumulée ; et la convolution (80) de l'estimation de valeur de mesure accumulée avec la forme de pouls de l'onde de pouls transmise.
  15. Un appareil d'imagerie pour reconstruire d'une image d'un objet, l'appareil comprenant des moyens pour :
    • transmettre un ensemble d'ondes de pouls à un objet à reconstruire au moins partiellement au moyen d'un ensemble d'éléments de transmission (13) ;
    • recevoir un ensemble de formes d'ondes d'écho réfléchies, au moyen d'un ensemble de capteurs (13), à partir de l'objet caractérisé par un ensemble de valeurs de réflectivité définies sur une grille de réflectivité comprenant des points de grille de réflectivité, l'ensemble de formes d'ondes d'écho définissant un ensemble de valeurs de mesure définies sur une grille de mesure comprenant des points de grille de mesure ;
    • définir un problème inverse pour obtenir un ensemble d'estimations de valeur de réflectivité à partir de l'ensemble de valeurs de mesure au moyen d'un modèle de mesure ;
    • exprimer le problème inverse comme un problème d'optimisation impliquant une fonction objective comprenant un terme de divergence de données ;
    • exprimer problème d'optimisation sous la forme d'un système d'équations impliquant des contributions du terme de divergence de données ;
    • obtenir l'ensemble d'estimations de valeur de réflectivité en effectuant des itérations dans le système d'équations jusqu'à ce qu'un seuil de convergence donné soit atteint, au moins certaines des itérations impliquant au moins une estimation de valeur de mesure pour chaque point de grille de mesure d'un ensemble de des points de grille de mesure et une estimation de valeur de réflectivité pour chaque point de grille de réflectivité d'un ensemble de points de grille de réflectivité ;
    l'appareil étant caractérisé en ce que l'obtention des contributions dans le système d'équations comprend des moyens pour :
    • obtenir l'estimation de valeur de mesure pour chaque point de grille de mesure de l'ensemble de points de grille de mesure à partir du modèle de mesure, l'estimation de valeur de mesure étant configurée pour être obtenu en :
    • générant un ensemble d'équations paramétriques de réflectivité par élément de transmission - trajet du capteur définissant un ensemble d'hypersurfaces de réflectivité comprenant des valeurs de réflectivité d'hypersurface ;
    • intégrant les valeurs de réflectivité d'hypersurface sur chaque hypersurface de réflectivité de l'ensemble d'hypersurfaces de réflectivité pour obtenir l'estimation de valeur de mesure pour le point de grille de mesure respectif ;
    • obtenir l'estimation de valeur de réflectivité pour chaque point de grille de réflectivité de l'ensemble de points de grille de réflectivité à partir d'un opérateur adjoint du modèle de mesure, l'estimation de valeur de réflectivité étant configurée pour être obtenu en :
    • générant un ensemble d'équations paramétriques de mesure par élément de transmission - trajet du capteur définissant un ensemble d'hypersurfaces de mesure comprenant des valeurs de mesure d'hypersurface ;
    • intégrant les valeurs de mesure d'hypersurface sur chaque hypersurface de mesure de l'ensemble d'hypersurfaces de mesure pour obtenir l'estimation de valeur de réflectivité pour le point de grille de réflectivité respectif.
EP17187412.6A 2017-08-23 2017-08-23 Procédé de reconstruction d'image basée sur un modèle Active EP3447525B1 (fr)

Priority Applications (8)

Application Number Priority Date Filing Date Title
EP17187412.6A EP3447525B1 (fr) 2017-08-23 2017-08-23 Procédé de reconstruction d'image basée sur un modèle
ES17187412T ES2812649T3 (es) 2017-08-23 2017-08-23 Método de reconstrucción de imágenes basado en modelo
JP2020511196A JP6924530B2 (ja) 2017-08-23 2018-08-21 モデルベース画像再構成方法
KR1020207008376A KR102326149B1 (ko) 2017-08-23 2018-08-21 모델-기반 이미지 재구성 방법
US16/641,234 US11175392B2 (en) 2017-08-23 2018-08-21 Model-based image reconstruction method
PCT/EP2018/072579 WO2019038296A1 (fr) 2017-08-23 2018-08-21 Procédé de reconstruction d'image basé sur un modèle
CN201880066558.6A CN111213066B (zh) 2017-08-23 2018-08-21 基于模型的图像重建方法
IL272819A IL272819B (en) 2017-08-23 2020-02-20 A method for image reconstruction based on a model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP17187412.6A EP3447525B1 (fr) 2017-08-23 2017-08-23 Procédé de reconstruction d'image basée sur un modèle

Publications (2)

Publication Number Publication Date
EP3447525A1 EP3447525A1 (fr) 2019-02-27
EP3447525B1 true EP3447525B1 (fr) 2020-05-06

Family

ID=59829133

Family Applications (1)

Application Number Title Priority Date Filing Date
EP17187412.6A Active EP3447525B1 (fr) 2017-08-23 2017-08-23 Procédé de reconstruction d'image basée sur un modèle

Country Status (8)

Country Link
US (1) US11175392B2 (fr)
EP (1) EP3447525B1 (fr)
JP (1) JP6924530B2 (fr)
KR (1) KR102326149B1 (fr)
CN (1) CN111213066B (fr)
ES (1) ES2812649T3 (fr)
IL (1) IL272819B (fr)
WO (1) WO2019038296A1 (fr)

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
ES2886155T3 (es) * 2018-10-08 2021-12-16 Ecole Polytechnique Fed Lausanne Epfl Método de reconstrucción de imágenes basado en un mapeo no lineal entrenado
US12357273B2 (en) * 2020-08-27 2025-07-15 E-Scopics Apparatus and method for estimating a velocity of at least one scatterer in a medium
EP4334748A4 (fr) * 2021-05-04 2025-03-12 Cloudstream Medical Imaging, Inc. Acquisition et traitement de données ultrasonores d'onde v au moyen d'un transducteur à réseau linéaire ou incurvé
CN114022728B (zh) * 2021-10-27 2023-04-07 中国科学院光电技术研究所 一种基于深度强化学习的合成孔径共相检测方法
CN114545402B (zh) * 2022-02-16 2024-06-07 西安电子科技大学 一种svd求逆分裂布雷格曼的扫描雷达前视成像方法
IT202200025848A1 (it) * 2022-12-16 2024-06-16 St Microelectronics Srl Metodo per fornire una stima di tempo di volo
CN118614898B (zh) * 2024-06-11 2025-11-21 东南大学 智能超表面辅助的非视距人体生命体征感知系统及方法

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4494089B2 (ja) 2004-06-02 2010-06-30 富士フイルム株式会社 超音波送受信装置
EP1717757A1 (fr) * 2005-04-28 2006-11-02 Bayerische Motoren Werke Aktiengesellschaft Procédé pour la représentation graphique de l'environnement d'un véhicule automobile
US8435180B2 (en) * 2007-09-17 2013-05-07 Siemens Medical Solutions Usa, Inc. Gain optimization of volume images for medical diagnostic ultrasonic imaging
US7905835B2 (en) * 2008-01-15 2011-03-15 General Electric Company Method for assessing mechanical properties of an elastic material
EP2536339B1 (fr) * 2010-02-18 2024-05-15 Maui Imaging, Inc. Transmission à source ponctuelle et correction de la vitesse du son à l'aide d'une imagerie par ultrasons à ouvertures multiples
US8315812B2 (en) * 2010-08-12 2012-11-20 Heartflow, Inc. Method and system for patient-specific modeling of blood flow
KR20140040679A (ko) * 2010-11-15 2014-04-03 인디언 인스티튜트 오브 테크놀로지 카라그푸르 향상된 초음파 이미징 시스템의 스펙클 저감/억제를 위한 향상된 초음파 이미징 방법/기술
CN102324106B (zh) 2011-06-02 2013-07-10 武汉大学 一种顾及地表光谱信息的sfs三维重建加密稀疏dem方法
EP2940487B1 (fr) * 2011-10-19 2016-07-20 Verasonics, Inc. Estimation et affichage pour imagerie doppler vectoriel utilisant des émissions d'ondes planes
IN2014DN07243A (fr) * 2012-03-26 2015-04-24 Maui Imaging Inc
CN103971404B (zh) 2014-04-14 2017-04-12 浙江工业大学 一种高性价比的3d实景复制装置
CN104408772A (zh) 2014-11-14 2015-03-11 江南大学 一种基于网格投影的自由曲面三维重建方法
JP6696782B2 (ja) 2015-03-09 2020-05-20 炭　親良 ビームフォーミング方法、計測イメージング装置、及び、通信装置
CN105069789B (zh) * 2015-08-05 2018-03-06 西安电子科技大学 基于编码网格模板的结构光动态场景深度获取方法
US20180042577A1 (en) * 2016-08-12 2018-02-15 General Electric Company Methods and systems for ultrasound imaging

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
None *

Also Published As

Publication number Publication date
IL272819B (en) 2021-06-30
JP2020534043A (ja) 2020-11-26
US11175392B2 (en) 2021-11-16
KR102326149B1 (ko) 2021-11-16
CN111213066A (zh) 2020-05-29
EP3447525A1 (fr) 2019-02-27
US20200225335A1 (en) 2020-07-16
JP6924530B2 (ja) 2021-08-25
KR20200040301A (ko) 2020-04-17
ES2812649T3 (es) 2021-03-17
WO2019038296A1 (fr) 2019-02-28
CN111213066B (zh) 2021-11-12
IL272819A (en) 2020-04-30

Similar Documents

Publication Publication Date Title
EP3447525B1 (fr) Procédé de reconstruction d'image basée sur un modèle
EP3637099B1 (fr) Procédé de reconstruction d'images basé sur un mappage non linéaire formé
Hunter et al. The wavenumber algorithm for full-matrix imaging using an ultrasonic array
Bulavinov et al. Sampling phased array a new technique for signal processing and ultrasonic imaging
Laroche et al. An inverse approach for ultrasonic imaging from full matrix capture data: Application to resolution enhancement in NDT
Zhang et al. Monte Carlo inversion of ultrasonic array data to map anisotropic weld properties
WO2018226688A1 (fr) Estimation de dispersion de la vitesse de phase dans une élastographie ultrasonore à l'aide d'une classification de signaux multiples
US12013373B2 (en) Free-encoder positioning system using acoustic features and IMU
US20210048413A1 (en) Fast pattern recognition using ultrasound
Laroche et al. Fast non-stationary deconvolution of ultrasonic beamformed images for nondestructive testing
Kang et al. Measurement of shallow defects using noncontact broadband leaky Lamb wave produced by pulsed laser with ultrasound microphone
Shi et al. Resolution enhancement of ultrasonic imaging at oblique incidence by using WTFM based on FMC-AR
Robert et al. Assessment of real-time techniques for ultrasonic non-destructive testing
Liu et al. A multiscale residual U-net architecture for super-resolution ultrasonic phased array imaging from full matrix capture data
Laroche et al. An inverse approach for ultrasonic imaging by total focusing point for close reflectors separation
CN107907591A (zh) 多组分固液两相混合物组分浓度的超声检测系统和方法
Griffa et al. Investigation of the robustness of time reversal acoustics in solid media through the reconstruction of temporally symmetric sources
US11927565B2 (en) Techniques for adapting time delays of ultrasound inspection system during data acquisition
Carcreff et al. Including frequency-dependent attenuation for the deconvolution of ultrasonic signals
Aschy et al. Enhancement of the total focusing method imaging for immersion testing of anisotropic carbon fiber composite structures
Kröning et al. Sampling phased array: a new technique for signal processing and ultrasonic imaging
Ingram et al. Ultrasonic array imaging through reverberating layers for industrial process analysis
Huang Advances in Ultrasonic Methods for Damage Imaging and Elastic Constant Identification in Structural Components
Spytek et al. Numerical investigation of self-focused Lamb waves in anisotropic media
Rodriguez et al. Topological imaging of defects by Lamb wave inspection

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN PUBLISHED

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20190722

RBV Designated contracting states (corrected)

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

INTG Intention to grant announced

Effective date: 20191121

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAJ Information related to disapproval of communication of intention to grant by the applicant or resumption of examination proceedings by the epo deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR1

GRAL Information related to payment of fee for publishing/printing deleted

Free format text: ORIGINAL CODE: EPIDOSDIGR3

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

GRAR Information related to intention to grant a patent recorded

Free format text: ORIGINAL CODE: EPIDOSNIGR71

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: GRANT OF PATENT IS INTENDED

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE PATENT HAS BEEN GRANTED

INTC Intention to grant announced (deleted)
INTG Intention to grant announced

Effective date: 20200326

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE (EPFL)

RIN1 Information on inventor provided before grant (corrected)

Inventor name: BESSON, ADRIEN

Inventor name: THIRAN, JEAN-PHILIPPE

Inventor name: PERDIOS, DIMITRIS

Inventor name: ARDITI, MARCEL

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

RAP1 Party data changed (applicant data changed or rights of an application transferred)

Owner name: ECOLE POLYTECHNIQUE FEDERALE DE LAUSANNE (EPFL)

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

Ref country code: AT

Ref legal event code: REF

Ref document number: 1267654

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200515

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602017016081

Country of ref document: DE

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20200506

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200907

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200806

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200807

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200906

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200806

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 1267654

Country of ref document: AT

Kind code of ref document: T

Effective date: 20200506

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602017016081

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2812649

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20210317

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

26N No opposition filed

Effective date: 20210209

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200823

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200831

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200831

REG Reference to a national code

Ref country code: BE

Ref legal event code: MM

Effective date: 20200831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200823

Ref country code: BE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20200831

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: MT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: CY

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20200506

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20250926

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20250818

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: IT

Payment date: 20250825

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20250815

Year of fee payment: 9

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20250815

Year of fee payment: 9