WO2024184879A1 - Unsupervised calibration of an electromagnetic shape sensor - Google Patents

Abstract

A method and system for shape sensing calibration of an elongated flexible device, the system comprising: a shape sensor configured to sense in multiple locations along the device, a measurement indicative of an orientation in each location; and a processor storing a model of expected measurements dependent on the orientation and shape constraints, the processor is configured to: record measurements of the shape sensor, while the device is manipulated to various positionings; collect, from the recordings, a plurality of uncalibrated measurements measured by the sensor in a corresponding plurality of positionings of the device; generate a calibration error function representing calibration errors, based on the stored model of expected measurements and shape constraints, the calibration error function has unknown calibration values and positioning values; and calculate calibration values and positioning values that explain the collected measurements, by minimizing the calibration error function.

Description

UNSUPERVISED CALIBRATION OF AN ELECTROMAGNETIC SHAPE SENSOR

RELATED APPLICATION/S

This application claims the benefit of priority of U.S. Provisional Patent Application No. 63/450,018 filed on 5 March 2023, the contents of which are incorporated herein by reference in their entirety.

FIELD AND BACKGROUND OF THE DISCLOSURE

The present disclosure, in some embodiments thereof, relates to electromagnetic (EM) tracking systems and more specifically, to calibration of systems thereof.

Certain EM tracking systems use DC magnetometers to sense low-frequency EM fields for position and orientation tracking. One application of such tracking systems is for EM shape sensing of a medical device, such as a fully shape tracked endoscope.

EM tracking systems usually comprise one or more tracked EM sensor (receiver), (for example, EM coil-based sensor) and EM transmitter (also known as a fields generator, for example, EM coilbased transmitter). In a standard EM setting, multiple EM fields of known geometry and shape are generated in tracking space, usually multiplexed using different known transmission frequencies and phases. An EM sensor (receiver) senses the generated fields (usually using EM coils), usually in 3 sensor axes (but sometimes only along a single axis). A time series of sampled magnetic fields is then processed to demultiplex the sensed fields into field values (for example, using Discrete Fourier Transform, correlation or any other suitable method). The sensed fields values are used to solve the sensor’s position and orientation in space. For example, using 3 generated EM fields, a 3-axis sensor picks up 3 x 3 values (a full 3D magnetic field per each of the 3 transmitted EM fields). This enables solving a 6-DOF position and orientation of the sensor relative to the transmitter using the 9 sensed EM values. With a single-axis sensor it may not be possible to solve the roll of the sensor, due to roll symmetry. The sensor then only senses magnetic fields along its single axis. It may then be possible, with a sufficient number of transmitted EM fields, to solve for the 5-DOF position and partial orientation of the EM sensor. EM tracking systems usually rely on a known EM model. For example, the generated EM fields need to be precisely known in space: for each point r = (%, y, z) in space the EM fields generated by the transmitter at point r are known. Alternatively, some systems rely on an implicitly known EM fields, for example, by training a solver which converts magnetic field measurements of the specific EM transmitter to 6-DOF position and orientation of a sensor, for example using an accurate EM mapping or an accuracy jig (shown, for example, with reference to Fig. 4).

Different EM sensors may be manufactured with different tolerances of gains and geometries. For example, a coil-based EM sensor may comprise three orthogonal tiny EM coils which pick up alternating EM fields using Faraday’s law of induction. Due to tolerances in the construction of the sensor, each of the three EM coils may not be manufactured with an exact anticipated gain. In addition, the three EM coils may not be perfectly orthogonal or concentric. Similarly, different EM generators may be manufactured with different tolerances of gains and geometries. For example, a coil-based EM transmitter may comprise of three concentric orthogonal EM transmitting coils which generate alternating EM fields in different frequencies according to Ampere’s law. Due to tolerances in the construction of the transmitter, each of the three EM coils may not be manufactured with an exact anticipated gain. In addition, the three EM coils may not be perfectly orthogonal or concentric. In addition, in a multi-coil EM transmitter, some crosstalk may exist between the different transmitting coils. For the EM tracking system to be accurate, these effects need to be accounted for.

In addition, metals which are located in the proximity of the EM tracking system may create EM distortion fields, for example due to eddy currents flowing in the conductive metal or due to magnetization of ferromagnetic materials. Some metals may be fixed relative to the EM transmitter. For example, metal bars in the walls or a metallic frame of a patient’s bed on which an EM transmitter is located. Other metals may be fixed relative to the EM sensors. These distortion fields might be picked up by the EM sensors and impact the EM tracking accuracy. Some systems use a preliminary EM mapping process to measure the distorted EM fields in space (which may be fixed relative to the EM transmitter).

For an EM tracking system to be accurate, the sensed values need to be processed such that they are independent of the specific EM sensor unit which performs the measurements, and optionally independent of the specific EM transmitter which generates the fields. An EM tracking system should also account for metals which are fixed relative to the EM transmitter or receiver. U.S. Patent No. 11,712,309, titled

SYSTEM AND METHOD USING DIGITAL MAGNETOMETERS” discloses a system for magnetic tracking of a flexible catheter device or another flexible elongated device, the system comprising: at least one generator, each configured to generate an alternating magnetic field wherein each generated magnetic field has a determined source amplitude and frequency; a device comprising: a flexible tube; a plurality of sensors, the sensors are located along the flexible tube, each configured to communicate sensed values of a local magnetic field, wherein the sensed values are at least partially due to the generated magnetic field; and a host server configured to: receive the sensed local magnetic field values from the corresponding sensors; and calculate, based on the magnetic field values and the determined source amplitude and frequency, a localization of the flexible tube, wherein the host server is optionally included in a controller of the sensors.

SUMMARY OF THE DISCLOSURE

In some aspects of embodiments of the present disclosure, there is provided a system for shape sensing calibration of an elongated flexible device, the system comprising: a shape sensor configured to sense in multiple locations along the device, a measurement indicative of an orientation in each location; and a processor storing a model of expected measurements dependent on the orientation and shape constraints, the processor is configured to: record measurements of the shape sensor, while the device is manipulated to various positionings; collect, from the recordings, a plurality of uncalibrated measurements measured by the sensor in a corresponding plurality of positionings of the device; generate a calibration error function representing calibration errors, based on the stored model of expected measurements and shape constraints, the calibration error function has unknown calibration values and positioning values; and calculate calibration values and positioning values that explain the collected measurements, by minimizing the calibration error function.

Optionally, the measurement indicative of an orientation in each location includes indication of a shape.

Optionally, the shape sensor is a shape and position sensor.

Optionally, the shape and position sensor is configured to sense in multiple locations along the device, an indication of position and orientation in each location. Optionally, the sensed orientation indication comprises a magnetic field measurement in each location.

Optionally, the sensed orientation indication comprises a measurement of a local optical feature.

Optionally, the plurality of positionings, to which the device is manipulated, are unknown.

Optionally, the shape sensor comprises an array of electromagnetic field sensor elements, and wherein for each positioning of the device the processor collects magnetic field measurements from a plurality of the sensor elements.

Optionally, the calibration error function incorporates at least one of a list comprising: a magnetic field measurement error function, a time error function, and a geometrical shape error function.

Optionally, the geometrical shape error function is based on geometrical constraints that comprises at least one of: smoothness, length, or twist constraints.

Optionally, the shape sensor is configured to sense magnetic field measurements at multiple locations along the device, and the processor is configured to calculate a position and orientation corresponding to each measurement based on a stored magnetic field model.

Optionally, the shape sensor is configured to sense magnetic field measurements at multiple locations along the device, and the processor is configured to calculate a curve corresponding to the measurements based on a stored magnetic field model.

Optionally, the calibration values comprise one or more sensor calibration matrices and one or more transmitter calibration matrices.

Optionally, the calibration values comprise distances between sensor elements of the shape sensor.

Optionally, the calibration error function incorporates shape, length, smoothness, and time constraints. Optionally, the calibration error function incorporates geometrical-mechanical considerations of the device, such that the error increases as attributes of the solution shape are farther from those allowed according to geometrical-mechanical constraints.

Optionally, the shape sensor is configured to sense optical measurements at multiple locations along the device, and the processor is configured to calculate a shape of the sensor corresponding to the entirety of the measurements based on a stored optical shape model.

Optionally, the optical measurements are indicative of the extent of bending at multiple locations along the device in at least two directions.

According to other aspects of embodiment of the present disclosure, there is provided a method for shape sensing calibration of an elongated flexible device, the method comprising: recording measurements of a shape sensor, while the device is manipulated to various positionings; collecting, from the recordings, a plurality of uncalibrated measurements measured by the sensor in a corresponding plurality of positionings of the device; generating a calibration error function representing calibration errors, based on a stored model of expected measurements and shape constraints, the calibration error function has unknown calibration values and positioning values; and calculate calibration values and positioning values that explain the collected measurements, by minimizing the calibration error function.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the invention are herein described, by way of example only, with reference to the accompanying drawings. With specific reference now to the drawings in detail, it is stressed that the particulars shown are by way of example and for purposes of illustrative discussion of embodiments of the invention. In this regard, the description taken with the drawings makes apparent to those skilled in the art how embodiments of the invention may be practiced.

In the drawings:

Fig. 1 is a schematic illustration of a system for shape sensing calibration of an elongated flexible device, according to some embodiments of the present disclosure; Fig. 2A is a schematic illustration of a top view of sensor elements, on a device, with an imaginary centerline going through the device, according to some embodiments of the present disclosure;

Fig. 2B is a schematic illustration of a side view of sensor elements on a device, with imaginary centerline going through the device, wherein each element has a respective nearest point on the centerline, according to some embodiments of the present disclosure; and

Fig. 3 is a schematic flowchart illustrating a method for unsupervised calibration of a shape sensor, according to some embodiments of the present disclosure; and

Fig. 4 is a schematic illustration of a mechanical jig of a determined shape for calibration of a device and/or shape sensor, according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

Some embodiments of the present disclosure provide a system and method for unsupervised calibration process of an elongated device, for example an electromagnetic (EM) shape sensor. The provided EM shape sensor may include a sensor array, through which a curve is algorithmically fitted. The provided calibration process may fit a calibration model (e.g., correction matrices, gains etc.) to each of the sensor elements inside the sensor array such that the sensed electromagnetic fields are well explained by a known EM model, which may be indifferent to the specific sensor calibration process and results which are being used. The provided calibration process may be unsupervised, except for some restrictions, for example, to the speed of motion, sufficient coverage of a tracking space, limited bending angles, etc. The device and/or sensor array can be manipulated, e.g. positioned and bent freely in the tracking space during the calibration process, without any strict instructions. For example, the size of the tracking space may be between 0.01 m³ to 0.1 m³, 0.1 m³ to 0.2 m³, 0.2 m³ to 0.5 m³, 0.5 m³ to 1 m³ or of any other suitable dimensions. The unsupervised calibration is performed under some restrictions such as, for example, speed restrictions, bending angle restrictions and/or coverage of the tracking space. When enough positionings are obtained, and/or when enough space is covered, the calibration may be calculated successfully. In some embodiments, the calibration calculation may automatically begin after a certain duration of device manipulation, for example, 30 seconds or a minute or any other suitable duration. The calibration process disclosed herein may include calibration of both the curve sensor as well as the EM transmitter, for example by corresponding EM models for the curve sensor and for the transmitter. The disclosed calibration process may be used as EM mapping of the EM transmitted fields in a metallic environment. The disclosed calibration process can be performed as a factory calibration as well as in real-time, e.g. during user operation, allowing the system to correct for changes in calibration due to a highly dynamic metallic environment, both relative to the transmitter and to the sensor. Additional smoothness and length constraints can be imposed on the sensor to further improve and accelerate the calibration process.

The constraints mentioned herein can further be used for a shape calibration of the curve sensor. In some embodiments of the present disclosure, the shape calibration is a calibration of the device curve as a whole. For example, the curve sensor array has a shape transform which places all the sensor elements along a smooth curve, which may be the device’s centerline curve. Some embodiments may solve an entire curve of the device/curve sensor, for example based on magnetic measurements at the sensor elements of the curve sensor and a shape calibration. In some embodiments of the present disclosure, a shape translation transformation is performed to place the calculated curve of the curve sensor at a cross-sectional center, e.g. the centerline, e.g. the center of a working channel, of the elongated device. For example, the calculated curve’s location may be translated to the center of the device, e.g. the working channel, as opposed to actual location of the sensor elements, which may be on the circumference of the device.

It will be appreciated that throughout the present disclosure, we use the following terms interchangeably: position and/or shape sensing, curve sensing, shape tracking, curve tracking and full curve tracking.

It will be appreciated that throughout the present disclosure, the term “positionings” may include positions, orientations, and/or shapes.

Unless otherwise defined, all technical and/or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the invention pertains. Although methods and materials similar or equivalent to those described herein can be used in the practice or testing of embodiments of the invention, exemplary methods and/or materials are described below. In case of conflict, the patent specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only and are not intended to be necessarily limiting. Reference is made to Fig. 1, which is a schematic illustration of a system 100 for shape sensing calibration of an elongated flexible device 14, according to some embodiments of the present disclosure. System 100 may include a processor 10, and a shape sensor 16, which may be installed on device 14. Device 14 may be deformed to various shapes, while shape sensor 16 is sensing corresponding measurements indicative of the curve and/or shape and/or position of the device, for example corresponding magnetic field measurements. System 100 may perform an unsupervised calibration process of shape sensor 16.

According to some embodiments, processor 10 includes or is configured to receive instructions from a non-volatile memory 11. For example, processor 10 is configured to read the instructions, and the instructions cause processor 10 to carry out the operations, actions and/or method steps described herein.

In some embodiments of the present disclosure, system 100 may further include an electromagnetic (EM) transmitter 12 of electromagnetic fields. Shape sensor 16 may be configured to sense in multiple locations along the device, corresponding electromagnetic fields, which may be indicative of a position and orientation at each location. In some embodiments, shape sensor 16 includes an array of sensor elements 18, wherein a sensor element 18 senses one or more electromagnetic field values corresponding to one or more generated electromagnetic fields. Then, for example, a curve of shape sensor 16, relative to a location and/or orientation of EM transmitter 12, is algorithmically fitted by processor 10, based on the measurements performed by multiple sensor elements 18. In some embodiments, a location and orientation of individual sensor element 18 may be calculated based on the measured electromagnetic field values, relative to a location and/or orientation of EM transmitter 12. However, in other embodiments the curve is calculated as a whole based on the measurements, without calculating the location and orientation of individual sensor element 18.

Processor 10 may store a model 102 of expected measurements of shape sensor 16, dependent on a curve or a shape along device 14. In some embodiments, model 102 includes expected electromagnetic measurements, dependent on a localized curve of device 14 relative to the EM transmitter 12. In case shape sensor 16 is a fiber-optics shape sensor, processor 10 may store a model 102 of expected optical measurements of fiber-optics shape sensor 16, dependent on the shape of device 14. In this case, the position of device 14 in space does not change the measurements of shape sensor 16, since a fiber-optics shape sensor only measures the shape of the sensor, but is insensitive to the sensor position in space.

Reference is now made to Fig. 3, which is a schematic flowchart illustrating a method 200 for unsupervised calibration of a shape sensor, according to some embodiments of the present disclosure. As indicated in block 210, processor 10 may record measurements of shape sensor 16, for example while device 14 is manipulated to various positionings, for example unknown positionings. As indicated in block 220, processor 10 may collect, from the recordings, a plurality of uncalibrated measurements X_k, measured by sensor 16 in a corresponding plurality of positionings of device 14. As indicated in block 230, processor 10 may generate a calibration error function (e.g. an energy function), based on model 102 of expected measurements and shape constraints, representing the calibration errors of system 100. The calibration error function has unknown calibration parameter values and positioning values as described in more detail herein. As indicated in block 240, processor 10 may calculate a solved shape of device 14 by finding the calibration parameter values and the positioning values that explain the collected measurements, by minimizing the calibration error function, as described in more detail herein.

In some embodiments, shape sensor 16 is a shape and position sensor, for example a sensor configured to sense in multiple locations along device 14, magnetic field measurements in each location, or another indication of position and orientation in each location, relative to position and orientation of EM transmitter 12. Shape sensor 16 may include an array of electromagnetic field sensor elements 18. For each positioning of device 14, processor 10 may collect magnetic field measurements from a plurality of sensor elements 18.

In various embodiments of the present disclosure, the calibration error function incorporates at least one of a list comprising: a position and orientation measurement error function, a magnetic field measurement error function, a temporal/time error function, and a geometrical shape error function. The geometrical shape error function may be based on geometrical constraints that include at least one of: smoothness, length, or twist constraints.

In some embodiments, shape sensor 16 is configured to sense magnetic field measurements at multiple locations along device 14, and processor 10 is configured to calculate a curve corresponding to the measurements, based on a stored magnetic field model. In some embodiments, shape sensor 16 is configured to sense magnetic field measurements at multiple locations along device 14, and processor 10 is configured to calculate a position and orientation corresponding to the measurements of each individual sensor element 18 based on a stored magnetic field model.

In some embodiments of the present disclosure, the calibration parameter values include one or more sensor calibration matrices and one or more transmitter calibration matrices. The calibration error function may incorporate shape, length, smoothness, and time constraints. In some embodiments, the calibration error function incorporates geometrical-mechanical considerations of device 14, such that the error increases as attributes of the solution shape are farther from those allowed according to geometrical-mechanical constraints.

In some embodiments, shape sensor 16 is an optical fiber sensor, and/or the sensed orientation indication includes a measurement of a local optical feature. Shape sensor 16 may be configured to sense optical measurements at multiple locations along device 14. Processor 10 may be configured to calculate a shape of sensor 16, corresponding to the measurements, for example corresponding to the entirety of the measurements, based on a stored optical shape model. In some embodiments, the optical measurements are indicative of the extent of bending at multiple locations along device 14. The extent of bending may be measured in at least two directions.

According to some embodiments of the present disclosure, system 100 calibrates shape sensor 16 by fitting a calibrated (or expected) EM sensing model 102 to the multiple sensor elements 18 and/or a calibrated EM transmission model to EM transmitter 12, for example, such that the electromagnetic fields sensed by the multiple sensor elements 18 are well explained by a known EM model. For example, the calibration model includes correction matrices and gains of multiple sensor elements 18. Model 102 may describe, for example, the expected magnetic field measurements by sensor element 18 located at position r and orientation 9 relative to transmitter 12.

To calibrate an EM tracking system, it is potentially necessary to model the EM fields. In an EM system with C transmitted fields, the EM fields can be described as a function B(r) where r is a 3D position in space (in transmitter coordinates) and B(r) is a 3 X C matrix containing the C transmitted EM field values at position r. B (r) can be measured on a grid of positions relative to the transmitter using calibrated sensors and accurate mechanical EM mapping jigs. Alternatively, in the setting of an EM transmitter of a specific accurate geometry (such as a Printed Circuit Board (PCB) transmitter), B(r) can be computed analytically, for example using Biot-Savart law.

In a theoretical neutral setting (for example, without any magnetic distortion), with a calibrated sensor and transmitter, a 3 -axis EM sensor located at position r and orientation R relative to the transmitter (I? is a 3 X 3 orthogonal matrix which is represented, for example, by three Euler angles) is expected to measure the magnetic fields: X(r, 0) = R 0)^tB r), a 3 X C matrix describing the 3D measurements of the C transmitted EM fields along the sensor’s 3 axes, where R(0) is the 3 x 3 rotation matrix corresponding to sensor orientation angles 0 relative to the transmitter. Consequently, in the case of a single-axis sensor, the sensor is expected to measure X^r, 0) = (0 0 1) • 7?(0)^fB(r), a 1 X C vector describing measurements of the C transmitted EM fields along the sensor’s single axis. In this case is independent of the roll angle 0_Z (a single-axis sensor cannot solve for the roll angle due to roll symmetry).

In the setting of an uncalibrated sensor and uncalibrated transmitter, a simple way to model the measured EM fields is according to the following formula: X(r, 0) = U • X(r, 0) • V, where X(r, 0) is a 3 X C matrix representing the uncalibrated magnetic field measurements, X(r, 0) is a 3 X C matrix representing the calibrated measurements, U is a 3 X 3 sensor calibration matrix, representing the sensor’s true gains and non-orthogonality, potentially including EM distortion effects caused by metals which are fixed in the sensor’s coordinate system, and V is a C X C transmitter calibration matrix, representing the transmitter’s true gains (which in some types of transmitters, may depend on resistance and/or self-inductance of the coils) and “non-orthogonality” (or crosstalk between the different transmitting coils, for example due to mutual inductance in the transmitter), including EM distortion effects caused by metals which are fixed in the transmitter’s coordinate system. U and V can therefore be viewed as calibration parameter values. X(r, 0) provides a prediction for the expected uncalibrated sensor field measurements at (r, 0), using calibration matrices U, V.

In one embodiment of the present disclosure, the problem of calibration is viewed as an energy minimization process. Processor 10 may record measurements of shape sensor 16 in various positionings of device 14, for example while device 14 is being manipulated in an unsupervised manner. Processor 10 may collect, from the recordings, a plurality of uncalibrated measurements X_k, for a corresponding plurality of unknown positionings of flexible device 14. Each such positioning may include a plurality of unknown positions and orientations (r_fe, 0_fe) of multiple sensor elements 18. Processor 10 may obtain a plurality of uncalibrated m ¹²easurements X ~_k, measured by a sensor element 18 in respective unknown formations of device 14, in which the sensor element 18 has corresponding plurality of unknown positions and orientations (r_k, 9_k) in space.

Processor 10 may find the calibration parameter values and the calibrated positioning values, e.g. positions and orientations of a certain sensor element 18, that explain the measurements X_k. Processor 10 may find, for a plurality of measurements X_k, the plurality of positioning values (r_k, 0_k) of a sensor element 18, together with calibration matrices U, V such that X_k = U • X(r_k, 9_k) • V for each recorded measurement X_k, for example, in least squares sense. That is, for example, X_k — U • X(r_k, 0_k) • V is minimized to a certain minimal error value, for example, with a requirement that the error is below a certain threshold value, for the calibration to be called successful. For example, Processor 10 may do so by minimizing a magnetic measurement error energy function F(U, V, r_k, 0_k) = X_k — U • X(r_k, 9_k) • V, for example, in least-squares sense, using non-linear optimization methods. Finding calibration parameters (in this case U, 7) and solving positions and orientations (r_k, 0_k ) which explain the measurements X_k with minimal error for a variety of unknown device positionings ensures to a great certainty that the correct calibration parameters are solved for the used sensor and transmitter.

According to some embodiments of the present disclosure, processor 10 may find the calibration parameters and calibrated positioning values, by generating and minimizing a measurement error energy function for a certain sensor element 18. Processor 10 may generate a measurement error energy function for a certain sensor element 18, depending on unknown calibration parameter values and positioning values of the sensor elements 18. In some embodiments, the measurement error energy function depends on a calibration parameters vector, incorporating a calibration matrix V of transmitter 12 and/or a calibration matrix t/j of the certain i-th sensor element 18. Device 14 may contain any number N of sensor elements 18 (for example, N = 1,2,4,7,8,12,16). Additionally, in some embodiments, the measurement error energy function is dependent on a positioning values vector, incorporating calibrated positioning values, e.g., the various positions and orientations of the certain sensor element 18 through the various positionings of device 14. In some embodiments, processor 10 may find the calibration matrices/parameters and the calibrated positioning values of a certain or all sensor element 18, by minimizing a sum of squared values of the calibration errors through the various positionings of device 14, or by minimizing any other suitable function, for example to minimize a calibration error.

The multiple sensor elements 18 along shape sensor 16 may share the same calibration parameters with transmitter 12. For example, the calibration parameters are shared in case where the sensing of each sensor element is not dependent on the transmission frequency, for example, in the setting of a low-frequency EM tracking system. It is potentially advantageous, according to some embodiments of the present disclosure, to generate a combined measurement error energy function which represents the combined measurement error energy of the entire shape sensor 16, e.g., a combination of all individual energy functions of the sensor elements 18, to be optimized in a single process. That is, processor 10 may minimize a combined measurement error energy function to minimize calibration error through the various manipulations of device 14 and through the multiple sensor elements 18, to find the calibration parameter values of transmitter 12 and of the multiple sensor elements 18, and to find the calibrated positioning values of the multiple sensor elements 18, while sharing the same calibration parameters of transmitter 12, for example the same matrix V of transmitter 12, among all calibrated sensors.

Tying together the calibration of the multiple sensor elements 18, ensures a single, more stable solution for the transmitter calibration matrix V (or for any other calibration model describing the transmitter, which is shared among all sensor elements), and therefore also provides a better solution for the sensor calibration matrices Ui (or for any other calibration model describing the sensor elements). This is, for example, in case all sensor elements 18 are indifferent to the sensed frequency, or have the same frequency-dependent behavior, for example, in low-frequency EM tracking system, as mentioned herein.

In some embodiments, the calibration model can be general such that it does not necessarily contain V (shared) or V_L (per sensor element), U_L (per sensor element) and (r_k, 0_k) but may additionally or alternatively contain other parameters, such as EM parameters, such as sensor non-linearity correction curves, specific sensing parameters of specific EM sensing models or any other kind of unknown parameters which need to be determined in order to enable successful EM tracking for a given specific EM tracking system. For example, these can include certain sensor measurement curves per applied magnetic fields (for example, non-linear measurements per linearly applied magnetic fields). These curves can be learned in the unsupervised calibration described herein, according to some embodiments of the present disclosure. As long as the number of measurements is greater than the number of unknown parameters (for example, 5% greater, 10% greater, 50% greater, 100% greater or more) then the energy minimization process (for example, using non-linear optimization) is likely to avoid overfitting. For example, in the case of the EM model mentioned above, each 3D magnetic sensor element has a U_L calibration matrix (3 x 3), the transmitter has a V calibration matrix (6 x 6), for example, assuming a same transmitter calibration matrices which is shared among all sensor elements, each sensor element at each time t_k has unknown 6DOF (r_k, 0_k) and uncalibrated magnetic field measurements X_k (3 x 6). By collecting enough magnetic field measurements, and assuming that they are taken from different positionings inside the sensing volume such that they are different enough from each other (for example, in the extreme case avoiding collecting the same magnetic field measurements from the same positioning for all times), the number of measurements (K x N x 3 x 6) exceeds the number of unknown parameters (for example, N x 3 x 3 + C x C + K x N x 6 parameters). For example, for C = 3 and for K large enough, the number of measurements exceeds the number of unknown parameters by a factor of 9/6 (150%) such that the calibration energy minimization optimization is likely to avoid overfitting.

According to some embodiments, the positions and orientations of the sensor elements may be known (or partially known) during calibration. For example, an operator may be instructed to place the EM shape sensor 16 inside accurate mechanical jigs with known positions and orientations in space relative to the EM transmitter 12. In this case (r_k, 9 ) may no longer be a parameter in the energy minimization process of the calibration. That is, the position and orientation parameters of the sensor elements can be removed from the optimization process and (r_k, 9 ) can be treated as known values inside the energy function. For example, the position and orientation parameters of the sensor elements have known values, and thus do not need to be found by processor 10 as part of solving the calibration energy function. Processor 10 may substitute the position and orientation calibration parameters of the sensor elements with the known values. This simplifies the optimization process but can make the calibration process more cumbersome and time consuming. In addition, mechanical inaccuracy of the calibration jigs can introduce calibration errors. Using an unsupervised calibration process mitigates those risks.

Additionally or alternatively, the relative positions and orientations (shapes) of the sensor elements may be known (or partially known) during calibration. For example, an operator may be instructed to place the EM shape sensor 16 inside accurate mechanical jigs with known shapes. Fig. 4 is a schematic illustration of a mechanical jig 400 of a determined shape for calibration of a device 14 and/or shape sensor 16, according to some embodiments of the present disclosure. Jig 400 may include a curved groove 40, in which elongated device 14 and/or shape sensor 16 may be tightly placed, such that, for example, elongated device 14 and/or shape sensor 16 has the same curve as curved groove 40. For example, jig 400 may have a shape of a variable radius along the curve of groove 40, e.g. the bending of the curve has a variable radius.

In this case (r_k, 0_k) may be known one relative to the other (since the relative shape is constrained by jig 400) so that (r_k, 0 ) may no longer be needed as a parameter in the energy minimization process of the calibration, and for each timestamp the plurality of (r_k, 0_k) which correspond to the plurality of sensor elements 18 can be replaced with a single position and orientation (r_k, 0_k) representing the position and orientation of jig 400 in space, while all other sensor element positions and orientations are known relative to it. As explained in more detail below. In the case of a fiber optics shape sensor, absolute positions are not sensed and not solved, and it is sufficient to use accurate shape jigs, such as jig 400, to eliminate the relative sensor element positions (r_k, 0_k) from the optimization altogether.

According to some embodiments, the relative positions and/or orientations of the sensor elements 18 may be known or solved during calibration. For example, an operator may place shape sensor 16 in jig 400 such that shape sensor 16 is fixed relative to jig 400, and only the “global” position and orientation of one of sensor elements 18, or of jig 400, need to be found per each positioning, and/or the positions and orientations of all the sensor elements 18 with respect to the global one are known or solved to a good accuracy. This greatly reduces the number of degrees of freedom or increases the number of constraints, thus speeding up the optimization, in addition to improving the resulting calibration parameters. For example, an operator may attach shape sensor 16 to jig 400 such that sensor elements 18 are all fixed in the coordinate system of jig 400. The jig transform (position and orientation) in space may be unknown for each positioning. For example, the jig transform can be described using a single 6DOF per each timestamp t_k for a total of, for example, K x 6 unknown jig transform parameters. Since sensor elements 18 are fixed relative to jig 400, it suffices to solve their position and orientation relative to jig 400 just once, since they are static for all positionings of jig 400. This adds, for example, additional 6N parameters to the orientation (N is the number of sensor elements in the shape sensor). The total unknown parameters due to positionings is then reduced, for example, from K x N x 6 for independent positioning per each sensor element, to K x 6 + N x 6 for dependent positioning where relative positions and orientation between the different sensor elements 18 are preserved throughout the calibration.

According to another embodiment, multiple EM shape sensors 16 can be calibrated together by minimizing a shared calibration energy function. The energy function of each individual shape sensor 16 may be independent of the others, except for the transmitter calibration matrix V, which is shared between the calibration of all sensors 16 (when the same transmitter is used). This may improve the accuracy of the transmitter calibration. For example, multiple EM shape sensors are simultaneously calibrated with a same EM transmitter to produce an improved calibration for the EM transmitter 12, that is, filtering out the majority of calibration error caused by the use of a single EM shape sensor 16 in the calibration process of the transmitter. In some embodiments, to avoid ambiguity between Ui and a rotation 0_k, for example, since U_L may be rotated by a rotation matrix which is canceled by an inverse rotation in 0_k, which might make the optimization process divergent or unstable, U_L can be represented as a 3x3 triangular matrix, which can be considered as rotation free. Additionally or alternatively, since a scale in U_L can be canceled by an inverse scale in V, which might again make the optimization process fail, V can be assumed to be constant (for example, in the case of a regulated EM transmitter with known currents and fields), or Ui can be assumed to have unity gain (for example,

= 1) for one or multiple sensor elements 18.

In another embodiment of the present disclosure, processor 10 may also calculate a calibration matrix for calibration of a geometrical shape of shape sensor 16. Reference is now made to Figs. 2A and 2B, which are schematic top and side illustrations of elongated device 14, according to some embodiments of the present disclosure. Fig. 2A shows a top view of sensor elements 18, on device 14, with an imaginary centerline 13 going through device 14. In some embodiments, as shown in Fig. 2A, sensor elements 18 are intended to be positioned along the centerline 13, while there are possible errors in the positioning. In other embodiments, sensor elements 18 cannot be positioned along the centerline 13 due to mechanical considerations. Fig. 2B shows a side view of sensor elements 18 on device 14, with imaginary centerline 13 going through device 14, wherein each element 18 has a respective nearest point 15 on centerline 13.

Sensor elements 18 may consist of digital magnetometer sensors. Sensor elements 18 may be soldered to a flexible printed circuit (FPC), which may be embedded inside the walls of elongated device 14, for example, by wrapping an FPC with sensor elements 18 around the working channel of device 14, such as a shape-tracked endoscope. Sensor elements 18 may be positioned and oriented in fixed positions and orientations along device 14, which may not be along the device 14 centerline, and which might not be known in advance. In some applications it is potentially advantageous to track the centerline 13 of an elongated device 14. In this case, some geometrical correction needs to be performed for the raw positions and orientations of the EM tracked sensor elements 18 before a curve is fitted between them, which represents the device’s tracked centerline 13. Additionally or alternatively, when curve tracking is performed by fitting a curve directly onto the magnetic field measurements (instead of tracking each sensor element’s position and orientation individually), as described above, consideration needs to be taken realizing that the magnetic fields are being measured where the actual sensor elements 18 are located and not along the device’s centerline, but rather at slightly different positions and orientations relative to the centerline.

For example, in one embodiment of the present disclosure, processor 10 may assign to each i- th sensor element 18 a centerline shape calibration matrix AT₍ : rx₍ -» RX₍, transforming native local sensor coordinates rx₍ to nearest centerline coordinates RX₍ . These calibration matrices can be given according to the mechanical construction of the device. For example, in the setting of an EM shape sensed endoscope, the sensor elements 18 may be positioned inside the endoscope’s wall, above the endoscope’s working channel, for example, by wrapping an FPC containing the sensor elements 18 around the endoscope’s working channel. In some embodiments, processor 10 may use a fixed translation and rotation transformation matrix AT, describing a fixed offset and rotation of each sensor element 18 relative to the nearest point 15 on endoscope’s centerline 13.

However, while fixed shape calibration matrices AT₍ may be used, it is potentially advantageous to find (or just fine-tune) these matrices in the calibration process of shape sensor 16, according to some embodiments of the present disclosure. For example, AT; may be initialized with AT₍° which may be a fixed translation and/or rotation matrix, including, for example, the geometrical calibration transformations as given by mechanical specifications of the tracked device 14. An energy function can be constructed which represents the device’s shape calibration error, given shape calibration matrices AT₍. The shape calibration matrices AT₍ can then be found in the energy minimization process. In some embodiments of the present disclosure, AT₍° is used as a constraint in the energy function, where the energy is increased in proportion to AT) — AT₍° or (AT))^{- 1} (AT)⁰) — H_4x4 or any other formula which is indicative of the difference or non-similarity between AT) and AT)⁰. Thus, in the process of energy minimization of the calibration, the shape solution is prevented from moving too far away from what is expected based on mechanical considerations.

In one embodiment, a shape error energy function is used which may comprise two sub-energy elements, which represent smoothness and length constraints of the device’s centerline 13. This works under the assumption that the device’s centerline 13 is usually smooth (the level of smoothness depends on the mechanical properties of the tracked device 14). Additionally or alternatively, since the device cannot usually stretch or shrink, the length along the centerline 13 between two adjacent points 15 needs to be essentially fixed, regardless of device’s different manipulations, such as different positions, orientations, bending and twisting in space.

Accordingly, processor 10 may construct an energy function E^_mooth, representing the shape smoothness of device 14 for a certain measurement in a certain positioning of device 14. For example, the smoothness energy function Smooth may represent the smoothness of the entire shape tracked device 14 for a single -th measurement (which consists of N measurements of N sensor elements 18) and tie between all N positions and orientations of N sensor elements 18 in that measurement. The smoothness energy function may use the shape calibration matrices (AT₁₍ AT_N) of N sensor elements 18, and corresponding N parametrized positionings

of the sensor elements 18 in the same measurement k. The smoothness energy function Smooth ^may be computed, for example, by fitting a curve (such as a spline) between all N shape-corrected (according to AT)) centerline positions and orientations of respective N sensor elements 18, for example while the positions and orientations are being searched for and solved, and evaluating the smoothness of that curve (for example, by integrating squared second derivative of a fitted spline). For example, by minimizing second derivatives of the fitted curve, AT) are solved by optimization such that the sum of squares of the curve’s second derivatives or another suitable combination of the second derivatives is minimal. That is, for example, in cases in which positions and orientations per sensor element 18 are not solved in advance and/or separately, but rather solved together with other shape/geometrical/temporal energies such that the solved positions belong to a shared smooth curve, constrained by shape constraints and/or other mechanical or temporal constraints, as described in detail herein.

Additionally or alternatively, according to some embodiments of the present disclosure, the length errors between adjacent points 15 may be encoded as an energy function Eie_ngth> which participates in the calibration optimization. Processor 10 may construct an energy function ^length' representing the length errors between adjacent points 15 corresponding to respective adjacent sensor elements 18, in the -th measurement.

For example, the length error energy function Eie_ngth ^may represent the combined length error of the entire EM shape tracked device 14 for a single k-th measurement and tie between all N positions and orientation of N sensor elements 18 in that measurement. The length error energy function may use the shape calibration matrices (AT AT_N) of N sensor elements 18, and corresponding N positionings (r^, 0^,

0^) of sensor elements 18 in the same measurement k. The length error energy function E^k _ength may be computed, for example, by fitting a curve (such as a spline) between all N shape-corrected (according to AT₍) centerline positions and orientations of respective N sensor elements 18, and evaluating the lengths between neighboring points 13, which correspond to respective adjacent sensor elements 18, along that curve (for example, by integrating differential length elements of a fitted spline). By comparing the lengths of the fitted curve to pre-calibrated lengths (for example, from a known mechanical construction, or from sensor measurements), AT) can be further fine-tuned such that the lengths conform to the known anticipated lengths between sensor elements 18. Additionally, E_smoot > length ^{are a}l^so effective in accelerating the calibration process of the EM shape sensor, and to produce more accurate calibration results (as they provide constraints for the optimization).

In some embodiments, E^k _engih may compare the differential length elements of the fitted centerline curve to ones which are shared among all positionings, dLj, which may be added as unknown parameters to the optimization. E^k _ength then forces each solved curve (for each k-th positioning) to respect the same differential length elements, such that the curve cannot stretch or squeeze between the different positionings. In some embodiments, instead of adding unknown parameters dLj per each differential length element along the centerline curve, dLj may only represent the length between neighboring sensor elements 18, which may reduce the total number of unknown parameters in the optimization.

In some embodiment, temporal/time energy function Etime can be added to the optimization, which poses time constraints on the solved curves, to further improve the optimization. For example, a sensor cannot move or rotate too fast (for example, move faster than 10 mm/sec, 30 mm/sec, 100 mm/sec, 200 mm/sec or rotate faster than 180° or 360° or 1000° or 2000° or 10000° per second). Such energy can be T_{i k}(t + At) — T_{i k}(t), or any other similar formula. T_{i k} is the position and orientation of the i-th sensor in the k-th measurement. The time period between frames is At (for example 5ms, 10ms, 16ms, 20ms, or any other suitable time period between frames).

In another embodiment, the sensor cannot change in shape too fast over time (for example, cannot bend faster than 30 degrees/sec, 60 degrees/sec, 100 degrees/sec, etc.). In this case, a timebased energy function can be introduced, measuring the change in shape over time, for example by computing the derivative of the relative transforms (of a sensor element 18 in a coordinate system of the shape sensor 16, or of a neighboring sensor).

More generally, a single unified calibration energy function that incorporates the measurement error calibration with the geometrical shape calibration can be constructed for the calibration process.

The unified calibration energy function includes both the raw sensor measurement calibration energies which involve t/j, V, r_k, 0_k, as described herein, as well as a smoothness and length terms ^smooth ^an£l ^length f°^{r eac}h -th measurement. Including E_smooLh, Ei_ength results in finding the shape calibration matrices AT) of each i-th sensor element, and also ties between the calibration of the multiple sensor elements 18 thus accelerating and stabilizing the unified calibration, as discussed above. Additionally or alternatively, E_time ^can be added to the optimization, which poses time constraints on the solved curves, as explained in more detail herein.

According to one embodiment of the present disclosure, the calibration of EM transmitter 12 may be known in advance. For example, transmitter 12 may have been calibrated in advance using a calibrated EM sensor. Alternatively, transmitter 12 may be manufactured with negligible tolerances such that it can be assumed to transmit theoretically known fields (for example, a precisely manufactured PCB EM transmitter may be assumed to transmit known fields, according to Biot-Savart law applied on its precisely known traces geometry). In the case of a calibrated transmitter 12, the transmitter’s calibration parameters have known values, and thus do not need to be found by processor 10 as part of solving the calibration energy function. Processor 10 may substitute the transmitter’s calibration parameters with the known values inside the calibration energy function. For example, the transmitter calibration matrix V may be known. This can accelerate and stabilize the sensor calibration process.

In some embodiments, an EM mapping model can be incorporated inside the unsupervised EM calibration process. For example, in case where large metals are present in the tracking space, for example, when a flat EM transmitter is located on a patient bed with a metallic frame, a potentially large EM distortion, which is static relative to the EM transmitter, can be present in tracking space and can distort the EM tracking. In this case it may be beneficial to model the actual EM fields which are present in space (which include a superposition of the intentionally generated EM fields and the EM distortion fields) and use them for tracking. To model the actual EM fields, additional mapping model can be used by the EM calibration. For example, in some embodiments the model can comprise a grid of control points and distortion field values for each grid point. The distortion grid values are then added as parameters to the optimization, and the optimization is configured to solve them as part of the general energy minimization process. To avoid overfitting and to make sure that the optimization assumes minimal EM distortion, additional mapping energy can be added, which, according to some embodiments, adds cost which is correlative with the size of additional distortion grid values, such that the optimization will strive to explain the EM measurements without use of the additional distortion grid (assuming substantially zero or minimal additional distortion during the calibration, for example, distortion which is below 10% or 20% or 30% or 50% in size relative to the intentionally generated EM fields). In cases where the optimization is unable to explain the measurements by using the intentionally transmitted EM fields alone, it will be forced to add some amount of distortion grid values as superposition to the intentionally generated fields such that the measurements will be explained, and the total energy will be minimized. This method, according to some embodiments, allows to calibrate an EM shape sensor in a general, not necessarily distortion-free, environment, which can be beneficial in some clinical scenarios where an EM-neutral environment is not available.

According to some embodiments of the present disclosure, optimization, as described herein, may be performed until certain determined success and/or finish criteria are met, for example based on energy and/or error scores. Such scores may be presented to an operator. For example, an operator may be prompted by the system to collect more measurements in certain locations in space, for which more data is required in order to perform calibration. For example, processor 10 may compute a preliminary calibration and use it to determine where the operator has already visited, based on, for example, for which sensors/locations and/or shapes processor 10 achieves a stable calibration and/or a high-quality calibration, for example with low error score, or based on, for example, a preliminary calibration which is applied to the preliminary collected data to create a map of visited areas based on the preliminary solved shapes and positions. Processor 10 may communicate, for example for presentation on a display, colors indicating visited/unvisited areas inside the tracking space, thus indicating the operator to cover those areas which were not visited or have poor coverage. At the end of the calibration process, processor 10 may communicate, for example for presentation on a display, a score indicative of the quality of the calibration, which can be based, for example, on the optimization error during calibration. Additionally or alternatively, processor 10 may communicate, for example for presentation on a display, a score indicative of the calibration quality for a sensor or a location along device 14, or a score map indicating the quality of calibration along device 14, for example by colors indicating the scores. Additionally or alternatively, processor 10 may communicate to another device or to a certain module, for example for presentation on a display, a 3D score map, for example a color map, showing the calibration performance in space. For example, if there are metals present in space, then these areas of low calibration quality near those metals will be colored, for example, in red, because of the high error probability.

According to another embodiment of the present disclosure, instead or in addition to performing an offline calibration of the EM shape sensor 16, shape sensor 16 may be calibrated during operation, for example, during a medical procedure. For example, in a navigational bronchoscopy procedure, sensor 16 may be calibrated, or its calibration may be improved (for example, fine-tuned) during the registration preliminary step of the procedure: Multiple sensor measurements are collected during the registration process to register between the transmitter coordinates and the patient’s anatomy. These collected measurements can be used to fine-tune the sensor’s calibration, using energy minimization methods which are identical or similar to those mentioned herein (for example, starting at the sensor’s initial calibration in the optimization). This online calibration process may also account for unexpected nearby metals in the proximity of the transmitter, which can be modeled inside the calibration process using any kind of EM distortion model (given as parameters to the calibration energy to be minimized), among the simplistic model mentioned above (transmitter calibration matrix ). While the methods disclosed herein describe an EM shape sensor, it should be appreciated that some of them apply to the unsupervised calibration of a single EM sensor element. It should also be appreciated that the energy-based geometrical shape calibration methods apply to any type of shape sensor, including fiber optics shape sensor. In the case of a fiber optics shape sensor, most of the energy functions may apply (for example E_smoot > length- ^time) but some may be replaced to fit the optical model onto the optical measurements rather than fitting the EM model onto the magnetic measurements. For example, instead of directly integrating the bends which are sensed by a fiber optics shape sensor, solving the sensor’s shape using optimization, as mentioned herein, while combining “high-level” constraints, such as shape smoothness and length constraints, and/or such as temporal constraints, in energy minimization settings as described herein, can potentially provide much more stable solution for the shape.

As will be appreciated by one skilled in the art, some embodiments of the present invention may be embodied as a system, method or computer program product. Accordingly, some embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “circuit,” “module” or “system.” Furthermore, some embodiments of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon. Implementation of the method and/or system of some embodiments of the disclosure can involve performing and/or completing selected tasks manually, automatically, or a combination thereof. Moreover, according to actual instrumentation and equipment of some embodiments of the method and/or system of the disclosure, several selected tasks could be implemented by hardware, by software or by firmware and/or by a combination thereof, e.g., using an operating system.

For example, hardware for performing selected tasks according to some embodiments of the disclosure could be implemented as a chip or a circuit. As software, selected tasks according to some embodiments of the disclosure could be implemented as a plurality of software instructions being executed by a computer using any suitable operating system. In an exemplary embodiment of the disclosure, one or more tasks according to some exemplary embodiments of method and/or system as described herein are performed by a data processor, such as a computing platform for executing a plurality of instructions. Optionally, the data processor includes a volatile memory for storing instructions and/or data and/or a non-volatile storage, for example, a magnetic hard-disk and/or removable media, for storing instructions and/or data. Optionally, a network connection is provided as well. A display and/or a user input device such as a keyboard or mouse are optionally provided as well.

Any combination of one or more computer readable medium(s) may be utilized for some embodiments of the disclosure. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable readonly memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a computer readable medium and/or data used thereby may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

Computer program code for carrying out operations for some embodiments of the present disclosure may be written in any combination of one or more programming languages, including an object-oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, for example as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

Some embodiments of the present disclosure may be described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.

The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.

Some of the methods described herein are generally designed only for use by a computer, and may not be feasible or practical for performing purely manually might be expected to use completely different methods, e.g., making use of expert knowledge and/or the pattern recognition capabilities of the human brain, which would be vastly more efficient than manually going through the steps of the methods described herein.

Claims

1. A system for shape sensing calibration of an elongated flexible device, the system comprising: a shape sensor configured to sense in multiple locations along the device, a measurement indicative of an orientation in each location; and a processor storing a model of expected measurements dependent on the orientation and shape constraints, the processor is configured to: record measurements of the shape sensor, while the device is manipulated to various positionings; collect, from the recordings, a plurality of uncalibrated measurements measured by the sensor in a corresponding plurality of positionings of the device; generate a calibration error function representing calibration errors, based on the stored model of expected measurements and shape constraints, the calibration error function has unknown calibration values and positioning values; and calculate calibration values and positioning values that explain the collected measurements, by minimizing the calibration error function.

2. The system of claim 1, wherein the measurement indicative of an orientation in each location includes indication of a shape.

3. The system of claim 1, wherein the shape sensor is a shape and position sensor.

4. The system of claim 3, wherein the shape and position sensor is configured to sense in multiple locations along the device, an indication of position and orientation in each location.

5. The system of claim 1, wherein the sensed orientation indication comprises a magnetic field measurement in each location.

6. The system of claim 1, wherein the sensed orientation indication comprises a measurement of a local optical feature.

7. The system of claim 1, wherein the plurality of positionings, to which the device is manipulated, are unknown.

8. The system of claim 1, wherein the shape sensor comprises an array of electromagnetic field sensor elements, and wherein for each positioning of the device the processor collects magnetic field measurements from a plurality of the sensor elements.

9. The system of claim 1, wherein the calibration error function incorporates at least one of a list comprising: a magnetic field measurement error function, a time error function, and a geometrical shape error function.

10. The system of claim 9, wherein the geometrical shape error function is based on geometrical constraints that comprises at least one of: smoothness, length, or twist constraints.

11. The system of claim 1 , wherein the shape sensor is configured to sense magnetic field measurements at multiple locations along the device, and the processor is configured to calculate a position and orientation corresponding to each measurement based on a stored magnetic field model.

12. The system of claim 1, wherein the shape sensor is configured to sense magnetic field measurements at multiple locations along the device, and the processor is configured to calculate a curve corresponding to the measurements based on a stored magnetic field model.

13. The system of claim 1, wherein the calibration values comprise one or more sensor calibration matrices and one or more transmitter calibration matrices.

14. The system of claim 1, wherein the calibration values comprise distances between sensor elements of the shape sensor.

15. The system of claim 1, wherein the calibration error function incorporates shape, length, smoothness, and time constraints.

16. The system of claim 1, wherein the calibration error function incorporates geometrical- mechanical considerations of the device, such that the error increases as attributes of the solution shape are farther from those allowed according to geometrical-mechanical constraints.

17. The system of claim 1, wherein the shape sensor is configured to sense optical measurements at multiple locations along the device, and the processor is configured to calculate a shape of the sensor corresponding to the entirety of the measurements based on a stored optical shape model.

18. The system of claim 17, wherein the optical measurements are indicative of the extent of bending at multiple locations along the device in at least two directions.

19. A method for shape sensing calibration of an elongated flexible device, the method comprising: recording measurements of a shape sensor, while the device is manipulated to various positionings; collecting, from the recordings, a plurality of uncalibrated measurements measured by the sensor in a corresponding plurality of positionings of the device; generating a calibration error function representing calibration errors, based on a stored model of expected measurements and shape constraints, the calibration error function has unknown calibration values and positioning values; and calculate calibration values and positioning values that explain the collected measurements, by minimizing the calibration error function.

20. The method of claim 19, wherein the shape sensor is a shape and position sensor, and the method comprises sensing in multiple locations along the device, an indication of position and orientation in each location.

21. The method of claim 19, wherein the sensed orientation indication comprises at least one of: a magnetic field measurement in each location, or a measurement of a local optical feature.

22. The method of claim 19, wherein the plurality of positionings, to which the device is manipulated, are unknown.

23. The method of claim 19, wherein the shape sensor comprises an array of electromagnetic field sensor elements, and the method comprises collecting magnetic field measurements from a plurality of the sensor elements for each positioning of the device.

24. The method of claim 19, wherein the calibration error function incorporates at least one of a list comprising: a magnetic field measurement error function, a time error function, and a geometrical shape error function.

25. The method of claim 24, wherein the geometrical shape error function is based on geometrical constraints that comprises at least one of: smoothness, length, or twist constraints.

26. The method of claim 19, wherein the shape sensor is configured to sense magnetic field measurements at multiple locations along the device, and the method comprises calculating a position and orientation corresponding to each measurement based on a stored magnetic field model.

27. The method of claim 19, wherein the shape sensor is configured to sense magnetic field measurements at multiple locations along the device, and the method comprises calculating a curve corresponding to the measurements based on a stored magnetic field model.

28. The method of claim 19, wherein the calibration values comprise a sensor calibration matrix and a transmitter calibration matrix.

29. The method of claim 28, wherein the calibration values comprise distances between sensor elements of the shape sensor.

30. The method of claim 19, wherein the calibration error function incorporates shape, length, smoothness, and time constraints.

31. The method of claim 19, wherein the calibration error function incorporates geometrical- mechanical considerations of the device, such that the error increases as attributes of the solution shape are farther from those allowed according to geometrical-mechanical constraints.

32. The method of claim 19, wherein the shape sensor is configured to sense optical measurements at multiple locations along the device, and the method comprises calculating a shape of the sensor corresponding to the entirety of the measurements based on a stored optical shape model.

33. The method of claim 32, wherein the optical measurements are indicative of the extent of bending at multiple locations along the device in at least two directions.