[go: up one dir, main page]

WO2025054466A1 - Système de simulation d'échographie au point d'intervention, procédés et programmes pour une utilisation éducative dans des environnements médicaux non stationnaires - Google Patents

Système de simulation d'échographie au point d'intervention, procédés et programmes pour une utilisation éducative dans des environnements médicaux non stationnaires Download PDF

Info

Publication number
WO2025054466A1
WO2025054466A1 PCT/US2024/045609 US2024045609W WO2025054466A1 WO 2025054466 A1 WO2025054466 A1 WO 2025054466A1 US 2024045609 W US2024045609 W US 2024045609W WO 2025054466 A1 WO2025054466 A1 WO 2025054466A1
Authority
WO
WIPO (PCT)
Prior art keywords
imu
orientation
platform
information
probe
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/045609
Other languages
English (en)
Inventor
Lauren M. MALONEY
Griffin WALKER
Thea M. VIJAYA KUMAR
Alexander E. MERTZ
Daniella R. HEBERT
Taelyn KUPEC
Sean REAGAN
Alexander J. EICHERT
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Research Foundation of the State University of New York
Original Assignee
Research Foundation of the State University of New York
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Research Foundation of the State University of New York filed Critical Research Foundation of the State University of New York
Publication of WO2025054466A1 publication Critical patent/WO2025054466A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00—Simulators for teaching or training purposes
    • G09B9/56—Simulation of sonar
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B8/00—Diagnosis using ultrasonic, sonic or infrasonic waves
    • A61B8/42—Details of probe positioning or probe attachment to the patient

Definitions

  • Point-of-care ultrasonography is when a focused diagnostic ultrasound exam is performed at the patient’s bedside by a clinician in order to answer a specific clinical question in real-time.
  • the clinician performing the POCUS exam obtains and interprets the ultrasound images within the given clinical context, and then uses the information to immediately guide the proceeding clinical management.
  • POCUS is invaluable when treating patients with shortness of breath; asthma and heart failure can cause similar lung sounds when listening with a stethoscope, however treating a patient with heart failure as if they have asthma, or vice versa, can be harmful to the patient.
  • POCUS can quickly and reliably distinguish between the two conditions. Given the operational constraints of what diagnostic equipment is reasonable and safe to have in the prehospital environment, the development of portable ultrasound systems that are functional within ambulances and medical helicopters is a game changer for prehospital care. However, unlike other diagnostic imaging modalities such as CT or MRI, POCUS is operator dependent in both obtaining and interpreting the images, which can be impacted by operational challenges unique to the prehospital environment such as ambulance or medical helicopter movement, poor ambient lighting, and performing the exams while in a safety restraint. As such, it is crucial for prehospital clinicians to be able to learn and maintain POCUS skills within their own unique work environment.
  • a point-of-care ultrasonography (POCUS) training system comprising a first inertial measurement unit (IMU) platform, a pressure sensing system, a hand- held probe, and a terminal.
  • IMU inertial measurement unit
  • the first IMU platform may be fixed to a position in a moving vehicle.
  • the first IMU platform comprises a sensor system and a communication interface.
  • the sensor system is configured to provide information regarding an orientation of the moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the first direction is orthogonal to the second direction.
  • the communication interface is configured to transmit the information from the sensor system.
  • the pressure sensing system comprises a flexible pressure mat.
  • the mat may be mounted to a target.
  • the pressure sensing system is configured to output information indicative of a position and an amount of the pressure change.
  • the hand-held probe comprises a housing with a distal tip at a distal end.
  • the distal tip is configured to press against the target via the flexible pressure mat.
  • the housing has a grip portion for a user to hold the hand-held probe.
  • the housing also comprises a second inertial measurement unit (IMU) platform.
  • the second IMU platform comprises a sensor system configured to provide information regarding a combined orientation, which is the orientation of the moving vehicle and the hand-held probe with respect to at least the first direction and the second direction based on the reference frame and a communication interface configured to transmit the information from the sensor system.
  • the terminal device comprises a memory, a communication interface, and a processor.
  • the memory is configured to store a plurality of ultrasound images corresponding to defined positions with respect to the target and orientation of the hand-held probe.
  • the communication interface is configured to directly or indirectly receive information from the first IMU platform, the second IMU platform, and the pressure sensing system.
  • the processor is configured to determine the orientation of the hand-held device in at least the first direction and the second direction using the information from the first IMU platform and the second IMU platform, select one ultrasound image from the plurality of ultrasound images stored in memory for display based on the determined orientation and a position of the probe and cause the selected ultrasound image to be displayed.
  • the terminal may further comprise a display.
  • the selected ultrasound image may be displayed on the display.
  • the processor may further cause the determined orientation and/position of the probe to be displayed on the display.
  • the first IMU platform may be installed in the target such as a manikin strapped to a stretcher.
  • each sensor system may comprise a 9-axis IMU having a three-axis gyroscope, a three axis accelerometer and a three axis magnetometer.
  • Each system sensor may be configured to execute a fusion algorithm to determine the respective information in at least the first direction and the second direction.
  • the determination orientation may be with respect to an initial orientation of the probe such as where the probe is orthogonal to the surface of the target. The initial orientation may be defined at a reference position of the flexible pressure mat.
  • the processor may repeat the determinations and the selections at a predetermined repetition rate to achieve a set frame rate for display of each selected ultrasound image.
  • the system may further comprise a relay device.
  • the relay device comprises a plurality of communication interfaces including at least a first communication interface, a second communication interface and a third communication interface.
  • the relay device may receive the information from the first IMU platform via the first communication interface and receive information from the second IMU platform via the second communication interface and the information from the pressure sensing system via the third communication interface and relay information to the terminal based on the received information.
  • the relay device may further comprise a fourth communication interface.
  • the relay device may transmit a single communication, per cycle, via the fourth communication interface to the terminal. This single communication may comprise unique identifiers, respectively for the information from the first IMU platform, the second IMU platform and the pressure sensing system.
  • the flexible pressure mat comprises a flexible conductive layer sandwiched between non-conductive layers including a first non-conductive layer and a second non-conductive layer.
  • the first non-conductive layer comprises a first set of conductive strips and the second non-conductive layer comprise a second set of conductive strips.
  • the first set of conductive strips and the second set of conductive strips may be orthogonal to each other to form a sensing matrix for determining a 2D position.
  • the distance between adjacent strips in each set may be the same.
  • the flexible conductive layer may be a conductive material that changes intrinsic resistance based on applied pressure.
  • the conductive strips may be conductive adhesive tape.
  • the pressure sensing system may further comprise a readout circuit configured to sequentially apply a voltage or current from a voltage or current source to either the first set of conductive strips or the second set of conductive strips and readout out a change in the voltage or current from the other of the first set of conductive strips or the second set of conductive strips.
  • the 2D position of the hand-held probe may be based on an intersection of conductive strips associated with a maximum change from different non- conductive layers and a third dimension of the position is based on a magnitude of the change.
  • the communication interface in the first IMU platform and the second IMU platform may be wireless communication interfaces such as BLUETOOTH®.
  • the terminal may be a portable device.
  • the moving vehicle may be an ambulance, helicopter, or boat.
  • there are multiple stored ultrasound images for each intersection in the sensing matrix for each intersection there are different ultrasound images for different magnitudes of pressure and orientation of the hand-held probe.
  • Each of the multiple stored ultrasound images corresponds to a range of angles for at least the first direction and the second direction.
  • the plurality of ultrasound images, which are stored are acquired using an ultrasound system for anatomical positions on a patient to define the associated position and orientation for the respective ultrasound images.
  • the relay device may determine the 2D position of the probe.
  • the first direction second IMU is the value of the first direction determined from the information from the second IMU platform
  • the first direction first IMU is value of the first direction determined from the information
  • the communication interface is configured to receive information, either directly or indirectly, from a first inertial measurement unit (IMU) platform, a second IMU platform and a pressure sensing system.
  • the first IMU platform may be fixed to a position within a moving vehicle.
  • the first IMU platform may comprise a sensor system and a communication interface.
  • the sensor system may provide information regarding an orientation of the moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the first direction is orthogonal to the second direction.
  • the second IMU platform comprises a sensor system and a communication interface
  • the sensor system provides information regarding a combined orientation, which is the orientation of the moving vehicle and a hand-held probe with respect to at least the first direction and the second direction based on the reference frame, where is transmitted by the communication interface.
  • the pressure sensing system comprises a flexible pressure mat mountable to the target. A distal tip of the held-held probe presses the target via the flexible pressure mat.
  • the processor of the terminal is configured to determine the orientation of the hand-held device in at least the first direction and the second direction using the information from the first IMU platform and the second IMU platform; select one ultrasound image from a plurality of ultrasound images for display based on the determined orientation and a position of the probe; and cause the selected ultrasound image to be displayed.
  • the plurality of ultrasound images may be locally stored in the terminal. However, in other aspects, the plurality of ultrasound images may be stored in a server and the selected one may be downloaded to the terminal.
  • the terminal may further comprise a display and the selected ultrasound image is displayed on the display.
  • the processor may also cause the determined orientation and/or the position to be displayed on the display.
  • the determined orientation may be determined with respect to an initial orientation of the probe. The initial orientation may be orthogonal to a surface of the target.
  • the processor may be configured to repeat the determinations and the selections at a predetermined repetition rate to achieve a set frame rate for display of each selected ultrasound image.
  • the indirect communication may be via a relay device terminal.
  • the terminal may receive a single communication from the relay device, per cycle.
  • the single communication may comprise unique identifiers, respectively for the information from the first IMU platform, the second IMU platform and the pressure sensing system.
  • the communication interface may be a wireless communication interface.
  • the terminal may be a portable terminal.
  • a corresponding method for the terminal is also disclosed.
  • the method may comprise receiving, directly or indirectly, information, from a first IMU platform, regarding an orientation of a moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the first direction is orthogonal to the second direction.
  • the first IMU platform may be fixed to a position within the moving vehicle.
  • the method may further comprise receiving, directly or indirectly, information, from a second IMU platform, regarding a combined orientation, which is the orientation of the moving vehicle and a hand-held probe with respect to at least the first direction and the second direction based on the reference frame.
  • the second IMU platform may be located within the hand-held probe.
  • the method may further comprise receiving, directly or indirectly, from a pressure sensing system information regarding a pressure and location of the pressure exerted by the hand-held probe on a flexible pressure mat.
  • the flexible pressure mat may be positioned on a target.
  • the method may further comprise determining the orientation of the hand-held device in at least the first direction and the second direction using the information from the first IMU platform and the second IMU platform, selecting one ultrasound image from a plurality of ultrasound images for display based on the determined orientation and a position of the probe; and causing the selected ultrasound image to be displayed.
  • the method may further comprise determining the position of the probe based on the pressure and location of the pressure exerted by the hand-held probe on the flexible pressure mat.
  • the non-transitory computer-readable medium storing computer-readable instructions.
  • the non-transitory computer-readable medium when executed by a processor in a terminal device may cause the terminal device to determine an orientation of a hand-held device in at least a first direction and a second direction using information from a first inertial measurement unit (IMU) platform and a second IMU platform, select one ultrasound image from a plurality of ultrasound images for display based on the determined orientation and a position of the probe and cause the selected ultrasound image to be displayed.
  • the first direction is orthogonal to the second direction.
  • the first IMU platform may be fixed to a position within a moving vehicle.
  • the information from the first IMU platform may comprise information regarding an orientation of the moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the second IMU platform may be located within the hand-held probe.
  • the information from the second IMU platform may comprise information regarding a combined orientation, which is the orientation of the moving vehicle and the hand-held probe with respect to at least the first direction and the second direction based on the reference frame.
  • Fig.1 illustrates an example of a point-of-care ultrasonography (POCUS) simulation system in accordance with aspects of the disclosure
  • Fig.2A illustrates an example of the housing for a probe for the POCUS system simulation in accordance with aspects of the disclosure
  • Fig.2B illustrates a block diagram of an example of the probe in accordance with aspects of the disclosure
  • Fig.2C illustrates a block diagram of an example of a sensing system for the probe in accordance with aspects of the disclosure
  • Fig.3 illustrates a block diagram of an example of the vehicle motion suppression system in accordance with aspects of the disclosure
  • Fig.4 illustrates a block diagram of an example of a relay device in accordance with aspects of the disclosure
  • Fig.5 illustrates a block diagram of an example of a pressure sensing system in accordance with aspects of the disclosure
  • aspects of the disclosure provide point-of-care ultrasonography (POCUS) simulation systems and associated methods which may be used in a moving environment such as, but not limited to, a moving vehicle.
  • the moving vehicle may be an ambulance, a medical helicopter, or a boat such as one used by the Coast Guard.
  • the system can correct for or compensate for the motion of the moving environment such that the motion of the probe, which is used to simulate a real ultrasound image device, can report its true orientation with respect to a target without any background motion caused by the moving environment. This enables an appropriate image to be displayed on a terminal for the position and orientation of the probe with respect to the target.
  • Fig.1 illustrates an example of a POCUS simulation system in accordance with aspects of the disclosure.
  • the system may comprise a probe 100, a motion suppression system (vehicle motion suppression system 105), a terminal 110, a pressure sensing system 115 and a relay device 120.
  • the probe 100 is used to contact with a target for the simulation.
  • the target may be a manikin.
  • the target may also be a person such as a volunteer or actor used for training a user (trainee).
  • the user of the probe may be a person desired to train such as an emergency medical technician (EMT), paramedic, nurse, physician, pilot, , U.S. military service member, etc....
  • EMT emergency medical technician
  • the probe 100 may be a handheld device.
  • the handheld device may be cordless such as shown in Fig.2A.
  • the probe 100 has a housing.
  • the housing may provide a handle 200 and a distal tip 205.
  • the housing may be fabricated using 3D printing.
  • the housing preferably is made of a non-conductive material such as a plastic to avoid any interferences with sensing of the orientation of the probe.
  • the outside of the housing may have texture to facilitate a non- slipping grip for the user.
  • the shape of the handle 200 may mimic an actual ultrasound probe.
  • the distal tip 205 may be made of the same material as the handle. In other aspects, the distal tip 205 may be made of a flexible material to contour to the shape of the target (body part).
  • the tip 205 may have a tapered surface to facilitate rotation in at least two degrees of freedom (at least two axes of rotation).
  • Figs.10A and 10B illustrates an example of the rotation.
  • the probe 100 is rotated in a first direction and as shown in Fig.10B, the probe 100 is rotated in a second direction.
  • the first direction is orthogonal to the second direction.
  • the second direction of rotation may be called a roll (such as rotation about the longitudinal axis).
  • the first direction of rotation may be called a pitch rotation.
  • the direction of rotations may be defined by the orientation of the sensing device within the probe (to be discussed later).
  • Figs.10A and 10B illustrate a linear probe (linear tip) with a substantially rectangular shape.
  • the tip 205 may be curvilinear such that the distal end of the tip has an arched profile.
  • the shape may be more square such as for a phase array probe.
  • the type of probe 100 used in the POCUS simulation may be application based.
  • linear probes may be used for soft tissues examinations and vascular access procedures.
  • Curvilinear probes may be used for abdominal examinations such as liver, stomach, gallbladder.
  • Curvilinear probes may also be used for the aorta.
  • Phase array probes may be used for cardiac, and abdominal examinations.
  • the tip 205 may be interchangeable such that the tip 205 may be disconnected with the handle 200 and replaced by a different type of probe tip. This increases the versatility of the probe 100 for simulation.
  • the probe 100 may sense the tip type attached to the handle 200.
  • each tip type for the probe may have a dedicated resistance value within the tip and when the tip 205 is attached to the handle 200, a sensor in the handle may sense this resistance.
  • the sensed resistance identifies the tip type. The sensed type may be reported by the probe to the relay device 120 and to the terminal 110.
  • Figs.16A and 16B illustrate another example of a probe.
  • Fig.2B illustrates a block diagram of the probe 100 in accordance with aspects of the disclosure.
  • the probe 100 comprises a mount 220, a sensing system 215 and a communication interface 210 (an example of a IMU platform).
  • the probe 100 may also comprise the tip type sensor.
  • the mount 220 is configured to hold the sensing system 215 within the housing.
  • the handle 200 of the housing may be fabricated in two pieces and attached to each other.
  • the mount 220 may have two parts, a first portion on the interior surface of the front portion of the handle (front is shown in Fig.2A) and a second portion on the interior surface of the back portion of the handle (rear is hidden in Fig. 2A).
  • the sensing system 215 may be positioned within the two portions to hold the sensing system 215 in place.
  • the top of the sensing system 215 may face the distal end of the distal tip 205.
  • the mount 220 may be on only one of the front or rear portions of the handle.
  • the sensing system 215 may be positioned on the mount with the top of the sensing system 215 orthogonal to the distal end of the distal tip 205.
  • Fig.2C illustrates an example of the sensing system 215 in accordance with aspects of the disclosure.
  • the sensing system 215 may comprise a nine-axes inertial measurement unit (IMU) such as a three-axis gyroscope 215-1, a three-axis accelerometer 215-2 and a three-axis magnetometer 215-3.
  • the sensing system 215 also comprises a processor 215-4.
  • the processor 215-4 may be a microcontroller or a microprocessor.
  • the processor 215-4 may be a field programable gate array.
  • the processor 215-4 is configured to use the information from the nine-axes IMU to determine the orientation of the probe with at least two degrees of freedom (at least the first direction and the second direction).
  • the processor 215-4 implements one or more fusion algorithms which combines the nine-axis measurements to obtain the orientation in at least the first direction and the second direction (also referred to herein as “with respect to”).
  • the orientation in the third direction of rotation (orthogonal to the first direction and the second direction may be determined).
  • the third direction of rotation may be relevant. Different fusion algorithms may be used to determine the orientation.
  • Fusions algorithms may determine a change in angle between consecutive measurements. Other fusion algorithms may determine the actual angle (cumulative) of the probe in each direction, e.g., total change since being at in initial orientation. [0064] Fusions algorithms are used to combat drift in the measurements from the sensors 215-1- 215-3. Drift is a particular problem with angular measurements from a gyroscope. [0065] Prior to each simulation, a sensor calibration may be performed for each type of sensor, three-axis gyroscope 215-1, a three-axis accelerometer 215-2 and a three-axis magnetometer 215-3. However, in other aspects, the calibration may be done periodically, such as weekly or monthly.
  • the calibration may be performed as needed such as if the trainer perceives an error in the determined orientation. This may be done to combat the drift from the gyroscope 215-1, the bias/offset in the accelerometer 215-2 and hard- and soft-iron distortions in the magnetometer such may impact the accuracy of the angular determinations.
  • the initial calibration may be executed in two-phases (stages). For example, the three-axis gyroscope 215-1 and the three-axis accelerometer 215-2 may be calibrated by placing the IMU on a flat surface for a preset period of time to calculate the offset bias.
  • the preset period of time may be several seconds. For example, the preset period of time may be 10 seconds.
  • the three-axis magnetometer 215-3 may be calibrated by translating and/or rotating the IMU, repeatedly to determine the offset bias.
  • the IMU may be moved in a “figure eight” rotation where the device is rotated in the vertical direction, such that it begins with the IMU facing horizontally (as if it were laying on a flat, horizontal surface).
  • the IMU is smoothly tilted sideways.
  • the IMU is returned to a horizontal position (flipped 180 degrees now). While in the lower loop, the IMU may be titled sideways in the opposite direction.
  • the magnetic biases in the x, y, and z directions are taken and their maximum values are stored throughout the process.
  • the processor 215-4 may implement a Mahony Explicit Complementary Filter (MECF) as the fusion algorithm.
  • the MECF uses quaternions to implement the fusion.
  • the quaternions may be based on the measurement from the gyroscope 215-1.
  • the processor 214-5 may use an open-source library to determine the orientation(s).
  • the open-source library may be Reefwing Attitude and Heading Reference System (AHRS).
  • AHRS Heading Reference System
  • the processor 214-5 using the information from the nine axes IMU may call the open-source library for values and receive integrated angles (cumulative angular changes from the initial orientation).
  • the processor 215-4 may implement a Kalman filter as the fusion algorithm. In other aspects, the processor 215-4 may implement a Madgwick filter as the fusion algorithm. A Kalman filter determines change in angle between consecutive measurements. Madwick filter and Kalman filter may also be implemented, in part, using an open-source library. [0069] In an aspect of the disclosure, the initial start orientation of the probe 100 is at a preset angle. For example, the preset angle may be orthogonal to a specific area of the target. [0070] In an aspect of the disclosure, the processor 215-4 may be installed on a board such as an iOS Nano 33 BLE device. The IMU may also be installed on the same board.
  • the IMU may be MEMS sensor available from STMicroelectronics® model LSM9DS1.
  • the communication interface 210 may be a wireless communication interface such as Wi-Fi® or Wi-Fi Direct®.
  • the wireless communication interface may be a near-field communication interface.
  • the wireless communication interface may be BLUETOOTH® and BLUETOOTH® Low Energy.
  • the interface 210 may be a BLUETOOTH module HC-05 which may be directly connected to the PC.
  • the probe 100 may be the master of the connection with an external device when paired.
  • Fig.3 illustrates an example of a VMSS 105 in accordance with aspects of the disclosure.
  • the VMSS 105 comprises a communication interface 310, a sensor mount 320 and a sensing system 315.
  • the VMSS 105 also has a housing.
  • the housing may be fabricated using 3D printing.
  • the housing preferably is made of a non-conductive material such as a plastic to avoid any interferences with sensing of the orientation.
  • the VMSS 105 may be mounted to a predetermined portion of the moving environment.
  • the VMSS 105 may be anchored to the floor of the vehicle. The anchoring of the VMSS 105 enables the VMSS 105 to move in concert with the vehicle such that any changes in the orientation are causes entirely by the movement of the vehicle.
  • the VMSS 105 may be indirectly attached or affixed to the vehicle.
  • a stretcher or a board may be installed in the vehicle (for training purposes).
  • the stretcher or board may be anchored to the vehicle such as to the wall or floor.
  • the VMSS 105 may be attached to (anchored) to the stretcher or board.
  • a manikin may be strapped to the stretcher or board.
  • the VMSS 105 may be installed within this manikin.
  • the VMSS 105 may be fixed to the bottom side of the manikin (side in contact with the stretcher or board).
  • the sensing system 315 may comprise a similar nine-axes inertial measurement unit (IMU) as in the probe 100 such as the three-axis gyroscope 215-1, the three-axis accelerometer 215-2 and the three-axis magnetometer 215-3.
  • the IMU may be MEMS sensor available from STMicroelectronics model LSM9DS1.
  • the sensing system 315 may also comprise a similar processor such as described above, e.g., processor 215-4. Similar to above, the processor 215-4 may be installed on a board such as an electrician Nano 33 BLE device. The IMU may also be installed on the same board.
  • the sensing system 315 may be attached to the interior surface of any wall of the housing of the VMSS 105. Since the VMSS 105 and the probe 100 measure changes from an initial known position, the relative orientations of the mounting of the two IMU may not be relevant and impact the determined orientations. In other aspects, the mounting orientation of the two IMUs (one in the VMSS 105 and the other in the probe 100) may be set to match.
  • the sensing system 315 is mounted to the interior wall using a sensor mount 320. In an aspect of the disclosure, the sensing system 315 may be epoxied to the sensor mount 320 which in turn may be epoxied to a wall of the VMSS 105.
  • the sensor mount 320 may utilized an interference fit.
  • the interior board may have protruding projections such as nubs slightly thicker at its ends than holes in the sensing system 315 enabling a secure “snap” fit.
  • the processor 215-4 in the VMSS 105 may implement the same fusion algorithm as the processor 215-4 in the probe 100.
  • the sensing systems 215 and 315 may be set to have a similar reporting cycle or frequency.
  • both IMUs may use the same frame of reference for the orientation determinations, e.g., earth frame of reference.
  • a sensor calibration may be performed for each type of sensor, three- axis gyroscope 215-1, a three-axis accelerometer 215-2 and a three-axis magnetometer 215-3 in the VMSS 105. This may be done to combat the drift from the gyroscope 215-1, the bias/offset in the accelerometer 215-2 and hard- and soft-iron distortions in the magnetometer such may impact the accuracy of the angular determinations.
  • the calibration may be similar to described above (two phase) (prior to fixing the VMSS to the vehicle).
  • the communication interface 310 may be the same communication as in the probe 100 such as a wireless communication interface. However, the same communication interface is not required.
  • the communication interface 310 may be Wi-Fi® or Wi-Fi Direct®. In other aspects of the disclosure, the wireless communication interface may be a near-field communication interface. In another aspect of the disclosure, the wireless communication interface may be BLUETOOTH® and BLUETOOTH® Low Energy. For example, the interface 310 may be a BLUETOOTH module HC-05 which may be directly connected to the PC. In some aspects, the VMSS 105 may be the master of the connection with an external device when paired. [0081] Although not shown in the figures, the probe 100 and the VMSS 105 may further comprise an internal power supply. The internal power supply may provide required power for the sensing systems 215, 315, respectively, and the communication interfaces 210, 310.
  • each of the probe 100 and the VMSS 105 may have a switch such as a button to turn ON or OFF the respective devices for use.
  • the probe 100 and the VMSS 105 may be configured to transmit the determined orientation(s) in at least the first direction and the second direction to a relay device 120.
  • Fig.4 illustrates a block diagram of an example of the relay device 120.
  • the relay device 120 comprises a plurality of communication interfaces (e.g., 410A-410D).
  • the first communication interface 410A may be for communication with the probe 100 and the second communication interface 410B may be for communication with the VMSS 105.
  • These interfaces 410A and 410B may match the respective interfaces in the probe 100 and the VMSS 105.
  • the first communication interface 410A and the second communication interface 410B may also be BLUETOOTH® interfaces.
  • the BLUETOOTH interfaces would be paired with each other (probe to relay device and VMSS to relay device).
  • the first communication interface 410A and the second communication interface 410B may be Wi-Fi® interfaces where the relay device 120 communication with the probe 100 and the VMSS 105 via a wireless access point.
  • the relay device 120 may be a parent station in a Wi-Fi Direct network (WFD).
  • the relay device 120 may be configured to receive the determined orientation (of the probe 100) via the first communication interface 410A and receive the determined orientation (of the VMSS 105) via the second communication interface 410B.
  • the probe 100 and the VMSS 105 push the respective orientations to the relay device 120 periodically.
  • the relay device 120 may pull the respective orientations from the probe 100 and the VMSS 105 periodically.
  • the relay device 120 further comprises a memory 425.
  • the memory 425 may be any piece of hardware that is capable of storing information, either on a temporary basis and/or a permanent basis.
  • the memory 425 may be RAM, persistent storage or removable storage.
  • the memory 425 may temporarily store the received orientations from the probe 100 and the VMSS 105.
  • the relay device 120 may have a third communication interface 410C.
  • the third communication interface 410C may be used for communication with a pressure sensing system 115 (shown in Fig.5).
  • the relay device 120 may be part of the pressure sensing system 115.
  • the third communication interface 410C may be a wired communication interface.
  • the wired communication interface may be analog input PINs on a circuit board.
  • the third communication interface 410C is not limited to a wire communication interface, but may also be wireless communication interfaces as described above.
  • the relay device 120 may have a fourth communication interface 410D.
  • the fourth communication interface 410D may be used for communication with a terminal 110.
  • the fourth communication interface 410D may be a wired communication interface.
  • the fourth communication interface 410 may be a universal serial bus (USB) communication interface.
  • the relay device 120 may transmit the determined orientation of the probe 100 and the orientation of the VMSS 105 and the pressure information from the pressure sensing system 115 to the terminal 110 via the USB to a serial port (USB port) of the terminal device 110.
  • the fourth communication interface 410D may be a wireless communication interface as described above.
  • the relay device 120 further comprises a processor 415.
  • the processor 415 may be a microcontroller or microprocessor. In some aspects of the disclosure, the processor 415 may be a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The processor 415 may be other processing hardware such as one or more one or more central processing unit(s) CPUs or graphics processing unit(s). [0089] In some aspects of the disclosure, the processor 415 may be installed on a board such as an electrician (e.g., an chicken UNO). The relay device 120 may have an internal power supply 430 such as a battery. The battery may be connected to the PC via its power jack. [0090] Fig.5 illustrates a block diagram of a pressure sensing system 115 in accordance with aspects of the disclosure.
  • the pressure sensing system 115 comprises a readout circuit 500 and a pressure mat 505.
  • the pressure mat 505 may be resistance-based meaning the resistance of the pressure mat changes when under a pressure.
  • Figs.6A and 6B illustrate an example of the pressure mat 505 in accordance with aspects of the disclosure.
  • the pressure mat 505 may comprise a conductive layer 610.
  • the conductive layer may be a thin sheet of a suitably flexible material.
  • the conductive layer 610 may be fabricated from a conductive plastic such as a thermoplastic.
  • the conductive plastic may be a polymetric sheet (such a polyethylene carbon black).
  • the sheet has a volumetric conductivity.
  • An example of the sheet may be Velostat (which is a tradename).
  • the conductive layer may be an elastomer material such as a conductive rubber.
  • the rubber may be a composite including a liquid-metal such as polydimethylsiloxane (PDMS).
  • PDMS polydimethylsiloxane
  • the pressure mat 505 further comprises a conductive grid.
  • the conductive grid is made up of conductive strips 605A and 605B.
  • the conductive strips 605A and 605B are orthogonal with respect to each other to create a 2-Dimensional grid.
  • the conductive strips 605A may be arranged in columns and the conductive strips 605B may be arranged in rows.
  • the conductive strips 605A and conductive strips 605B are arranged on different sides of the conductive layer 610.
  • the conductive strips may be conductive threading, conductive tape, conductive wires (such as nanowires).
  • the conductive tape may be a copper tape.
  • the conductive threading may be made of a conductive material such as silver and/or stainless steel.
  • the conductive strips 605A and conductive strips 605B may be attached to respective non-conductive layers 600A and 600B (as shown in Fig.6B).
  • the non-conductive layers 600A, 600B may be any non-conductive material or fabric.
  • the conductive tape may be stuck to the non-conductive material since it is adhesive.
  • the conductive strips 605A, 605B may be sewn onto the non-conductive fabric.
  • the non-conductive layers with conductive strips may be fabricates using 3D printing (integrally formed).
  • the conductive strips 605A may be equidistant (equal spacing between strips in the column) and the conductive strips 605B may be equidistant (equal spacing between strips in the rows).
  • the spacing in the columns may be the same as the spacing in the rows.
  • the number of conductive strips 605A, 605B determines the resolution of the pressure mat 505. The closer the spacing in the strips provides a higher resolution.
  • the conductive strips 605A, 605B would be hidden.
  • the length of the conductive strips 605A, 605B is longer than the dimensions (length and width) of the conductive layer 610 to enable readout of the resistance (change in resistance) which may be measured by a change in voltage (or current).
  • the layers are respectively attached using a non-conductive epoxy to maintain the relative position of each layer.
  • the change in resistance may be proportional to the pressure applied to the mat 505.
  • Velostat resistance is inversely proportional to the pressure applied to it.
  • Fig.7 illustrates an example of a readout circuit 500 for the pressure mat 505 in accordance with an aspect of the disclosure.
  • the readout circuit 500 may comprises two multiplexers 700, 705. One is for the input side and the other is for the output side.
  • multiplexer 700 is connected to the rows of conductive strips 605B and the multiplexer 705 is connected to the columns of conductive strips 600A.
  • the readout circuit 500 may also comprise a resistor network 710 connected between the multiplexer 700 and the conductive strips 605B.
  • the resistor network 710 provides a voltage divider scheme for each “channel” of the multiplexer 700. As depicted, there are N channels in the multiplexer 700, where N equals the number of conductive strips 605B.
  • a predetermined voltage is serially applied to each readout channel (N) for each readout cycle.
  • the readout may be controlled by the relay device 120.
  • the processor 415 may cause the power 430 to be applied to the channels.
  • the timing may be also determined by the processor 415.
  • the power may be supplied via the wired interface (410C).
  • the processor 415 may add a timestamp for each measurement.
  • a separate readout processor and a separate power supply may be used for the readout.
  • the voltage is readout of each of the M channels of the multiplexer 705 to pinpoint “the intersection” of row(s)/column(s) of the pressure.
  • the intersection(s) is/are determined by the region of the pressure mat 505 where the resistance, as measured, changes.
  • a change in the resistance is detected by a change in voltage.
  • the M-channels of voltages (detected) are sent to the relay device 120 via the communication interface 410C. For each readout cycle, all N channels are sequentially applied the voltage and M-channels are readout. Thus, for each readout cycle, there are N *M detected voltages.
  • the memory 425 may have a look up table, for the magnitude of pressure and the change in voltage (from no-pressure).
  • the look-up table may be populated during a calibration process with known pressures applied to the pressure mat 505.
  • only the intersection (row and column) corresponding to the highest voltage change (highest pressure) is identified as the region of the pressure.
  • each intersection may be assigned a 2-D coordinate. Thus, the determined intersection may be converted into a 2-D coordinate.
  • the relay device 120 may transmit the determine magnitude of pressure (which is indicative of a z-coordinate), the 2-D coordinate (x, y) and the determined orientations (at least with respect to the first and second directions) from the probe 100 and the VMSS 105 to the terminal 110.
  • a predetermined voltage instead of a predetermined voltage being applied; a predetermined current may be used.
  • a current divider scheme may replace the voltage divider scheme as shown in Fig.7.
  • Fig.8 illustrates an example block diagram of the terminal 110 in accordance with aspects of the disclosure.
  • the terminal 110 may be a portable device such as a laptop or a mobile phone.
  • the terminal 110 may be a device integrated in the vehicle. In other aspects, the terminal 110 may be incorporated into a heads-up device system associated with the vehicle.
  • the terminal 110 may comprise a processor 800, a memory 805, a communication interface(s) 810, a display 815, and a user interface 820.
  • the processor 800 may be one or more central processing unit(s) CPUs.
  • the processor 800 may also be one or more graphics processing unit(s) GPUs.
  • the processor 800 may be a microcontroller or microprocessor or any other processing hardware such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).
  • FPGA field programmable gate array
  • ASIC application specific integrated circuit
  • the processor 800 may be configured to execute one or more programs stored in a memory 805 to execute the functionality described herein such as to correct the determined orientation(s), select the corresponding image and cause the same to be displayed.
  • the memory 805 can be, for example, RAM, persistent storage, or removable storage.
  • the memory 805 may be any piece of hardware that is capable of storing information, such as, for example without limitation, images, lookup tables (LUTs), programs, instructions, program code, and/or other suitable information, either on a temporary basis and/or a permanent basis.
  • the memory 805 may store ultrasound images.
  • the ultrasound images may be real ultrasound images acquired using an ultrasound system on test patients or volunteers.
  • the memory 805 may have an LUT for the stored image (used for selection).
  • the look up table may have associated multiple images.
  • Each different image for the same (x, y) coordinate may correspond to a different pressure and/or different angular orientation with respect to the first direction and the second direction.
  • a technician may acquire real images at different angles and/or pressures at corresponding points to the (x, y) coordinates of the pressure mat 505 and store each image in the memory 805 with the associated pressure measurement and orientation in at least the first direction and the second direction.
  • the images may be stored in a server with their associated position, angular orientation with respect to at least the first direction and the second direction and pressure and the terminal 110 may access the server to obtain only the image corresponding to a determined position (x, y) and pressure (such as a z-position) and a corrected orientation with respect to at least the first direction and the second direction.
  • a determined position x, y
  • pressure such as a z-position
  • images for each potential orientation with respect to the first direction and the second direction and potential pressure of the probe on the target are not acquired. This because the number of possible combinations may be exhaustive and acquiring real images for all potential orientations and pressure may not be feasible.
  • the same image may correspond to an angular orientation of 30+-5° in the first direction and 50+-5° in the second direction.5° is an example of the angular tolerance and has been presented only for descriptive purposes and the angular tolerance is not limited to 5°.
  • Other angular tolerances may be used such as +- 10°, +-15° or +-2.5°.
  • Different angular tolerances may be used for the different directions (first direction v. second direction).
  • the angular tolerance may be set based on the application (body part of interest) or type of probe tip.
  • a pressure tolerance may be used as well.
  • the pressure tolerance may be based in part of the type of ADC converter used. For example, for a 10 bit ADC, the output may be a value between 0 and 1023 (“pressure” reading).
  • the pressure tolerance may be +-50 (units). For example, if the “pressure” is 500, the same image may correspond to 500 +-50.
  • the communication interface 810 may match the communication interface 410D of the relay device 120 (when a relay device 120 is used).
  • the communication interface 810 may be a USB interface.
  • the terminal 110 may also comprise a second communication interface such as a wireless communication interface to access the server.
  • the wireless communication interface may be a Wi-Fi® interface.
  • the display 815 may display a selected ultrasound image corresponding to a determine position (x, y) and the pressure magnitude and the corrected orientation with respect to at least the first direction and the second direction.
  • This display 815 may be a touchscreen, which also doubles as the user interface 820.
  • the display 815 may also display the determined position, corrected orientation and pressure magnitude.
  • the display 815 may also display a model of a person with images representative of internal body parts. This model may also have a marker or identifier showing the determined position of the probe 100.
  • the marker may be a colored dot.
  • Fig.13 illustrates two examples of items being displayed on the display 815 of the terminal 110.
  • probe values are included on the display.
  • the probe values may include the orientation.
  • the probe values also include frame rate (current and average).
  • the pressure magnitude may also be shown.
  • the probe values may be omitted such as shown in the second example.
  • the user interface 820 may be any type of interface to interact with the processor 800 and memory 805 such as a keyboard, a mouse.
  • the user interface 820 may be used to store the information for the LUT, e.g., enter the position (x, y), pressure magnitude (and tolerance) and orientations (and tolerances).
  • Fig.9 illustrates a diagram showing an example relationship of certain components of the system in accordance with aspects of the disclosure.
  • the manikin 905 has the VMSS 105.
  • This manikin 905 is strapped to the stretcher 900.
  • the stretcher 900 is anchored to the vehicle.
  • the pressure mat 505 is positioned on the manikin 905. Since the pressure mat 505 is flexible, the pressure mat 505 is able to conform to the semi-rigid surface of the target (manikin).
  • the surface of the pressure mat 505 facing the manikin 905 may have a reusable adhesive to position the pressure mat 505 such that it does not move during the simulation.
  • the instructor (actor) or trainee causes the processors 215-4 in the IMUs (probe 100/VMSS 105 to calibrate the sensors (such as described above), prior to each use.
  • the processor 800 in the terminal 110 may cause a prompt to be displayed to instruct the trainee to place the distal tip of the probe at this preset position (Fig.14, S50).
  • the preset position may be the center coordinate of the pressure mat 505 (xcenter, ycenter). In other aspects, the preset position may be one of the corners of the pressure mat 505.
  • the display 815 may display a marker on the model of the patient to show the preset position.
  • This prompt may also include an instruction to position the probe 100 in a predetermined orientation.
  • the predetermined orientation in the first direction and the second direction
  • the predetermined orientation in the first direction and the second direction
  • the predetermined orientation may be used as a reference orientation to determine the rotation (cumulative or change).
  • the predetermined orientation may be displayed.
  • a representation of the probe may be shown on the display 815 of the terminal 110 in the predetermined orientation.
  • the user trainee
  • the user may position the probe 100 on the pressure mat 505 at this preset position (initial position).
  • the relay device 120 or another controller may cause the readout circuit 500 to read out the position of the probe 100 on the pressure mat 505 to confirm that the probe 100 is at the initial position.
  • the raw voltage changes may be relayed by the relay device 120 to the terminal 110 such that the terminal 110 determines the position (x, y coordinates).
  • the relay device 120 determines the position (x, y coordinates).
  • the processor 800 in the terminal 110 determines whether the probe 100 is at the preset position.
  • the preset position (coordinates thereof) may be stored in the memory 805.
  • the processor 800 may determine whether the determined position matches the stored preset position.
  • the user may manually confirm that the probe 100 is at the preset position.
  • the user may input a YES via the user interface 820 for confirmation.
  • the processor 800 at S52 may determine whether the probe is at the predetermined orientation.
  • the processor 800 may instruct the relay device 120 to obtain the angle of the probe 100.
  • the relay device 120 may communicate with the sensing system 215 in the probe 100 via the respective communication interfaces 410A, 210 to obtain the orientation (with respect to the first direction and the second direction). If the initialization is occurring while the moving environment is stationary, then there may not be a need to obtain the orientation of the VMSS 105.
  • the relay device 120 may also communicate with the sensing system 315 in the VMSS 105 via the respective communication interfaces, 410B and 310 to obtain the orientation.
  • the relay device receives the respective determined orientations (with respect to at least the first direction and the second direction).
  • the relay device 120 transmits the received orientations to the terminal 110 via the respective communication interfaces 410D, 810.
  • the processor 800 may determine the true (corrected orientation) by subtracting the determined orientations.
  • directions may also be determined by a similar subtraction.
  • the predetermined orientation (with respect to at least the first direction and the second direction may be stored in the memory 805).
  • the processor 800 may compare the corrected orientation with the predetermined orientation in memory 805 to confirm that the probe 100 is orientated in the predetermined orientation.
  • the processor 800 of the terminal 110 may issues an instruction/confirmation to the relay device 120 to begin the simulation at S54 and display the ultrasound images determined in accordance with aspects of the disclosure such as shown in Fig.12.
  • the relay device 120 may determine the corrected orientation based on the received respective orientations from the sensing systems 215, 315.
  • the user may manually confirm that the probe 100 is at the predetermined orientation. For example, the user may input a YES via the user interface 820 for confirmation.
  • the instruction at S54 may cause the relay device 120 to begin the periodic readout process of the pressure mat 505 in response to certain conditions.
  • Fig.11 illustrates a flow chart of an example method for the relay device 120 in accordance with aspects of the disclosure such as after receiving the instruction from the terminal 110 (at S54).
  • the relay device 120 may wait to receive the orientation information from the respective sensing systems 215, 315. In other aspects, periodically, the relay device 120 may poll the respective sensing systems 215, 315 for the orientation information.
  • Orientation information used here refers to either the cumulative orientation with respect to at least the first direction and the second direction determined by the respective sensing systems 215, 315 or change orientations (between two consecutive readings) with respect to at least the first direction and the second direction determined by the respective sensing systems 215, 315.
  • the orientation information may be timestamped.
  • the processor 415 in the relay device 120 receives the orientation information from the sensing system 215 in the probe 100.
  • the processor 415 may check to see of the orientation information is valid prior to storage in the memory 425. A determination of validity may include confirming the number of bits in the orientation information, within an expected range, contain information for the first direction and the second direction, etc....
  • the processor 415 may confirm validity based on the timestamp. For example, if the received orientation information contains a stale timestamp, the processor 415 may discard the orientation information because it is old. A stale timestamp may be older than a predetermined time. The predetermined time may be based on the image refreshed rate (frame rate). [0133]
  • the processor 415 in the relay device 120 receives the orientation information from the sensing system 315 in the VMSS 105. In some aspects, the processor 415 may check to see if the orientation information is valid prior to storage in the memory 425 such as described above.
  • a determination of validity may include confirming the number of bits in the orientation information, within an expected range, contain information for the first direction and the second direction, etc....
  • the processor 415 may confirm validity based on the timestamp. For example, if the received orientation information contains a stale timestamp, the processor 415 may discard the orientation information because it is old. A stale timestamp may be older than a predetermined time. The predetermined time may be based on the image refreshed rate (frame rate). [0134] After both sets of orientation information is received (for a cycle), the processor 415 may associate them in memory 425 as corresponding to the same time period or cycle.
  • the processor 415 may consider the orientation information a set for a cycle when the timestamps indicate that the two are within a set time.
  • the set time may also be based on the image refreshed rate (frame rate). For example, the set time may be a few seconds such as 5 seconds or 10 seconds or 15 seconds.
  • the processor 415 may set a timer when the orientation information is received from one of the probe 100 or the VMSS 105 to predefined time.
  • the processor 415 may determine for a complete scan, if there was multiple positions (x, y coordinates) row and column combinations that exhibited a voltage change at S9. If there is only one combination, (one position of pressure), then S11 may be skipped. Otherwise, if there are multiple combinations/intersections where there are voltage changes, then pressure may have been applied to two regions. For example, this may occur where the distal tip 205 and a trainee’s hand are on the pressure mat 505. However, in this case, it is likely that the pressure of the distal tip 205 is higher than the pressure of the hand. [0142] At S11, the processor 415 may determine the position having the highest change in voltage (pressure change) and store the position and magnitude in the memory 425 for subsequent reporting.
  • the processor 415 may average the adjacent coordinates to determine the position to report. In some aspects, the processor 415 may use a weighted average to determine the position to report. [0144] Once the pressure position and magnitude are stored in memory for the cycle, the processor 415 associates this with the stored orientation information for the same cycle in memory 425. [0145] At S13, the relay device 120 transmits the sensed information for the same cycle to the terminal 110. In some aspects of the disclosure, the sensed information from the different devices (probe 100, VMSS 105, pressure sensing system 115) may be transmitted to the terminal 110 as a single packet for the cycle.
  • processor 415 may assign an identifier which is unique to each piece of sensed information for the cycle.
  • orientation from probe 100 may have IDENTIFER 1
  • orientation from the VMSS 105 may have IDENTIER 2
  • the position and magnitude from the pressure sensing system 115 may have IDENTIFIER 3.
  • the header of the packet may include as the source an identifier of the relay device 120 and as the destination an identifier of the terminal 110 and the payload may have the IDENTIFER 1 followed by the orientation from the probe 100, IDENTIFER 2 followed by the orientation from the VMSS 105 and IDENTIFIER 3 followed by the position and magnitude of the pressure.
  • the packet may be numbered such that the terminal 110 can determine whether a transmitted packet was missed.
  • the relay device 120 may transmit separate packets to the terminal 110 for the sensed information from each device per cycle (such as three packets). The packets may be numbered such as by cycle such that the terminal 110 may identify the packets from the same cycle. [0148] The relay device 120 transmits the packets to the terminal 110 via the respective interfaces 410D, 810 such as a USB interface. [0149] Fig.12 illustrates a flow chart of an example method for the terminal 110 in accordance with aspects of the disclosure. The terminal 110 waits for information (data packet(s)) from the relay device 120.
  • the processor 800 determines whether the sensed information is from the probe 100 (S20), from the VMSS 105 (s22) or from the pressure sensing system 115 (position and magnitude) (S24) for a current cycle. [0150] The determination may be based on the identifier(s) included in the payload of the received data packet and packet number. In an aspect of the disclosure, the processor 800 may store the expected packet number in the memory 805 (1 greater than the last received packet). In other aspects, the processor 800 may store the last received packet number 805. If the packet number is greater than the last received packet number, the processor may determine that the packet is current.
  • the processor 800 also determines whether an identifier in the payload of the data packet (received) is for the probe 100.
  • the identifier for the probe, identifier for the VMSS and pressure sensing system 115 are store in the memory 805.
  • the relay device 120 notifies the terminal 110 of the identifier.
  • the processor 800 stores the sensing information (orientation with respect to at least the first direction and the second direction) in the memory 805.
  • the processor 800 continues to wait for the sensed information from the probe 100 (via the relay device 120) (NO at S20).
  • the processor 800 determines whether an identifier in the payload of the data packet (received) is for the VMSS 105.
  • the processor 800 stores the sensing information (orientation of the VMSS 105 with respect to at least the first direction and the second direction) in the memory 805. Otherwise, the processor 800 continues to wait for the sensed information from the VMSS 105 (via the relay device 120) (NO at S22).
  • the processor 800 determines whether an identifier in the payload of the data packet (received) is for the pressure sensing system 115. When the identifier is for the pressure sensing system 115 and the sensed information is for the current cycle (YES at S24), the processor 800 stores the sensing information (pressure position and magnitude) in the memory 805. Otherwise, the processor 800 continues to wait for the sensed information from the pressure sensing system (via the relay device 120) (NO at S24). [0154] In Fig.12, S20, S22, and S24 are shown as being sequential, however, the processor 800 may perform these steps in parallel or any order. Also, when a single packet is used, the processor 800 may perform S20, S22, S24 together.
  • timestamps may be used to determine whether the sensed information is for a current cycle such that data packets received with timestamps within a set period of time may be deemed by the processor 800 as being within the same cycle and the sensed information associated with each other in memory 805 (for the current cycle).
  • the processor 800 may begin the correction process for correcting the orientation of the probe to account for the moving of the moving environment (true orientation).
  • the processor 800 does not need to wait to receive the sensed information from the pressure sensing system 115 to begin.
  • the correction may be with respect to each received orientation direction (at least the first direction and the second direction).
  • the correction may be independently determined.
  • the processor 800 may correct the orientation with respect to the first direction.
  • the correction may be different for the different fusion algorithms (cumulative v. change).
  • each sensing system 215, 315 delineates the direction with an identifier such that the first direction, the second direction, etc.... are able to be distinguished.
  • the relay device 120 also used an identifier to distinguish between orientation information from the different directions, and thus the processor 800 is able to tell the angles with respect to a particular direction.
  • the correction may be a subtraction of the two received values with respect to the first direction for the current cycle and the initial orientation.
  • the processor 800 may correct the orientation with respect to the second direction.
  • the correction may be different for the different fusion algorithms (cumulative v. change).
  • the corrected orientation with respect to at least the first direction and the second direction may be stored in the memory 805.
  • the processor 800 may determine the corresponding image to be displayed for the corrected orientation, the pressure position and magnitude.
  • the processor 800 may retrieve from memory 805 all of the associated sensed information for the cycle and the LUT.
  • the processor 800 Based on the corrected orientation, the pressure position and magnitude, the processor 800 selected the image to be displayed. As described above a plurality of images may be stored in association with respective sets of sensed information. The processor 800 selects the image having a match for the respective sensed information (including correction for motion of the moving environment). A match may include being within the tolerance. [0166] When there is no match, the processor 800 may select the image having the closest sensed information matching. For example, in some aspects, the closest sensed information matching may be based on a least square error determination for the position, pressure magnitude and orientation, which accounts for the difference in the orientation with respect to the first direction, orientation with respect to the second direction, position and pressure magnitude. In some aspects, a weight may be assigned to the different variables to place different importance on different variables.
  • the processor 800 may randomly select one of the two store images. [0168] At S32, the processor 800 causes the selected image to be displayed on the display 815.
  • the corrected orientation with respect to at least the first direction and the second direction may also be displayed on display 815 such as shown in one of the example displays in Fig.13.
  • a representation of the probe 100 may be shown and displayed in the corrected orientation.
  • a marker or representation of the location of the probe may be superposed on a model of the person such as shown in both examples in Fig.13.
  • the relay device 120 may be omitted and the probe 100 and VMSS 105 may directly communicate with the terminal 110A such as shown in Fig.15.
  • the pressure sensing system 115 may also directly communication with the terminal 110A.
  • the processor 800 may directly process the sensed information received from the respective devices.
  • the terminal 110A may control the readout circuit 500.
  • the terminal 110A may have additional communication interfaces such as Wi-Fi® or BLUETOOTH®.
  • a point-of-care ultrasonography (POCUS) training system may comprise a first inertial measurement unit (IMU) platform, a pressure sensing system, a hand-held probe, and a terminal.
  • the first IMU platform may be fixed to a position in a moving vehicle.
  • the first IMU platform comprises a sensor system and a communication interface.
  • the sensor system is configured to provide information regarding an orientation of the moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the first direction is orthogonal to the second direction.
  • the communication interface is configured to transmit the information from the sensor system.
  • the pressure sensing system comprises a flexible pressure mat. The mat may be mounted to a target.
  • the pressure sensing system is configured to output information indicative of a position and an amount of the pressure change.
  • the hand-held probe comprises a housing with a distal tip at a distal end. The distal tip is configured to press against the target via the flexible pressure mat.
  • the housing has a grip portion for a user to hold the hand-held probe.
  • the housing also comprises a second inertial measurement unit (IMU) platform.
  • the second IMU platform comprises a sensor system configured to provide information regarding a combined orientation, which is the orientation of the moving vehicle and the hand-held probe with respect to at least the first direction and the second direction based on the reference frame and a communication interface configured to transmit the information from the sensor system.
  • the terminal device comprises a memory, a communication interface, and a processor.
  • the memory is configured to store a plurality of ultrasound images corresponding to defined positions with respect to the target and orientation of the hand-held probe.
  • the communication interface is configured to directly or indirectly receive information from the first IMU platform, the second IMU platform, and the pressure sensing system.
  • the processor is configured to determine the orientation of the hand-held device in at least the first direction and the second direction using the information from the first IMU platform and the second IMU platform, select one ultrasound image from the plurality of ultrasound images stored in memory for display based on the determined orientation and a position of the probe and cause the selected ultrasound image to be displayed.
  • the terminal further comprises a display and the selected ultrasound image is displayed on the display.
  • the POCUS training system according to the second aspect, wherein the processor further causes the determined orientation and/or the position to be displayed on the display.
  • the POCUS training system according to any one of the first aspect through the third aspect, wherein the first IMU platform is installed in the target.
  • the target is a manikin strapped to a stretcher.
  • each sensor system comprises a 9-axis IMU having a three-axis gyroscope, a three axis accelerometer and a three axis magnetometer.
  • Each system sensor is configured to execute a fusion algorithm to determine the respective information in at least the first direction and the second direction.
  • the POCUS training system according to any one of the first aspect through the sixth aspect, wherein the determined orientation is with respect to an initial orientation of the probe, and the initial orientation is orthogonal to a surface of the target.
  • the POCUS training system according to the seventh aspect, wherein the initial orientation is defined at a reference position on the flexible pressure mat.
  • the POCUS training system according to any one of the first aspect through the ninth aspect, wherein the processor is configured to repeat the determinations and the selections at a predetermined repetition rate to achieve a set frame rate for display of each selected ultrasound image.
  • the POCUS training system according to any one of the first aspect through the tenth aspect, further comprising a relay device comprising a plurality of communication interfaces including at least a first communication interface, a second communication interface and a third communication interface.
  • the relay device is configured to receive the information from the first IMU platform via the first communication interface and receive the information from the second IMU platform via the second communication interface and the information from the pressure sensing system via the third communication interface and relay information to the terminal based on the received information.
  • the relay device comprises a fourth communication interface.
  • the relay device is configured to transmit a single communication, per cycle, via the fourth communication interface to the terminal.
  • the POCUS training system according to the twelfth aspect, wherein the single communication comprises unique identifiers, respectively for the information from the first IMU platform, the second IMU platform and the pressure sensing system.
  • the flexible pressure mat comprises a flexible conductive layer sandwiched between non-conductive layers including a first non- conductive layer and a second non-conductive layer.
  • the first non-conductive layer comprises a first set of conductive strips and the second non-conductive layer comprise a second set of conductive strips.
  • the POCUS training system further comprises a readout circuit configured to sequentially apply a voltage or current from a voltage or current source to either the first set of conductive strips or the second set of conductive strips and readout out a change in the voltage or current from the other of the first set of conductive strips or the second set of conductive strips.
  • the 2D position of the hand-held probe is based on an intersection of conductive strips associated with a maximum change from different non-conductive layers and a third dimension of the position is based on a magnitude of the change.
  • the flexible conductive layer is a conductive material that changes intrinsic resistance based on applied pressure.
  • the conductive strips are conductive adhesive tape.
  • the POCUS training system according to the eleventh aspect or twelfth aspect, wherein the communication interface in the first IMU platform and the second IMU platform are wireless communication interfaces.
  • the wireless communication interfaces are BLUETOOTH® and are respectively paired with the first communication interface and the second communication interface in the relay device.
  • the POCUS training system according any one of the first aspect through the nineteenth aspect, wherein the terminal is a portable terminal.
  • the POCUS training system according any one of the first aspect through the twentieth aspect, wherein the moving vehicle is an ambulance, helicopter, or boat.
  • the POCUS training system according the any one of the fourteenth aspect through the seventeenth aspect, wherein there are multiple stored ultrasound images for each intersection in the sensing matrix, for each intersection there are different ultrasound images for different magnitudes of pressure and orientation of the hand- held probe.
  • each of the multiple stored ultrasound images corresponds to a range of angles for at least the first direction and the second direction.
  • a terminal for a point-of-care ultrasonography (POCUS) training system comprises a communication interface and a processor.
  • the communication interface configured to directly or indirectly receive information from a first inertial measurement unit (IMU) platform, a second IMU platform and a pressure sensing system.
  • the first IMU platform is fixed to a position in a moving vehicle.
  • the first IMU platform comprises a sensor system and a communication interface.
  • the sensor system provides information regarding an orientation of the moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the first direction is orthogonal to the second direction.
  • the second IMU platform comprises a sensor system where the sensor system provides information regarding a combined orientation, which is the orientation of the moving vehicle and a hand-held probe with respect to at least the first direction and the second direction based on the reference frame.
  • the second IMU platform also comprises a communication interface configured to transmit the information from the sensor system.
  • the second IMU platform is located within a housing the hand-held probe.
  • the pressure sensing system comprises a flexible pressure mat mountable to the target. A distal tip of the held-held probe presses the target via the flexible pressure mat.
  • the processor is configured to determine the orientation of the hand-held device in at least the first direction and the second direction using the information from the first IMU platform and the second IMU platform, select one ultrasound image from a plurality of ultrasound images for display based on the determined orientation and a position of the probe, and cause the selected ultrasound image to be displayed.
  • the terminal according to the twenty-sixth aspect further comprising a memory configured to store the plurality of ultrasound images corresponding to defined positions with respect to the target and orientation of the hand-held probe.
  • the terminal according to the twenty-sixth aspect or the twenty-seventh aspect further comprises a display and the selected ultrasound image is displayed on the display.
  • the processor further causes the determined orientation and/or the position to be displayed on the display.
  • the terminal according to any one of the twenty- sixth aspect to the twenty-ninth aspect wherein the determined orientation is with respect to an initial orientation of the probe. The initial orientation is orthogonal to a surface of the target.
  • the processor is configured to repeat the determinations and the selections at a predetermined repetition rate to achieve a set frame rate for display of each selected ultrasound image.
  • the terminal according to any one of the twenty-sixth aspect to the thirty-first aspect wherein the indirect communication is via a relay device.
  • the terminal receives a single communication from the relay device, per cycle.
  • the single communication comprises unique identifiers, respectively for the information from the first IMU platform, the second IMU platform and the pressure sensing system.
  • the communication interface is a wireless communication interface.
  • the terminal according to any one of the twenty- sixth aspect to the thirty-fourth aspect wherein there are multiple stored ultrasound images for a respective position on the flexible pressure mat.
  • each of the multiple stored ultrasound images corresponds to a range of angles for at least the first direction and the second direction.
  • the terminal according to the thirty-sixth aspect wherein the plurality of ultrasound images which are stored are acquired using an ultrasound system for anatomical positions on a patient to define the associated position and orientation for the respective ultrasound images.
  • a method comprises receiving, directly or indirectly, information, from a first IMU platform, regarding an orientation of a moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the first direction is orthogonal to the second direction.
  • the first IMU platform is fixed to a position within the moving vehicle.
  • the method further comprises receiving, directly or indirectly, information, from a second IMU platform, regarding a combined orientation, which is the orientation of the moving vehicle and a hand-held probe with respect to at least the first direction and the second direction based on the reference frame.
  • the second IMU platform is located withing the hand-held probe.
  • the method further comprises receiving, directly or indirectly, from a pressure sensing system information regarding a pressure and location of the pressure exerted by the hand-held probe on a flexible pressure mat.
  • the flexible pressure mat is positioned on a target.
  • a non-transitory computer-readable medium storing computer readable instructions, when executed by a processor in a terminal device cause the terminal device to determine an orientation of a hand-held device in at least a first direction and a second direction using information from a first inertial measurement unit (IMU) platform and a second IMU platform, select one ultrasound image from a plurality of ultrasound images for display based on the determined orientation and a position of the probe and cause the selected ultrasound image to be displayed.
  • the first direction is orthogonal to the second direction.
  • the first IMU platform is fixed to a position within a moving vehicle.
  • the information from the first IMU platform comprises information regarding an orientation of the moving vehicle with respect to at least a first direction and a second direction based on a reference frame.
  • the second IMU platform is located within the hand-held probe.
  • the information from the second IMU platform is regarding a combined orientation, which is the orientation of the moving vehicle and the hand- held probe with respect to at least the first direction and the second direction based on the reference frame.
  • references in the specification to “one aspect”, “certain aspects”, “some aspects” or “an aspect”, indicate that the aspect(s) described may include a particular feature or characteristic, but every aspect may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same aspect. Further, when a particular feature, structure, or characteristic is described in connection with an aspect, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other aspects whether or not explicitly described.
  • any numerical range expressly includes each numerical value (including fractional numbers and whole numbers) encompassed by that range.
  • reference herein to a range of “at least 50” or “at least about 50” includes whole numbers of 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, etc., and fractional numbers 50.1, 50.250.3, 50.4, 50.5, 50.6, 50.7, 50.8, 50.9, etc.
  • reference herein to a range of “less than 50” or “less than about 50” includes whole numbers 49, 48, 47, 46, 45, 44, 43, 42, 41, 40, etc., and fractional numbers 49.9, 49.8, 49.7, 49.6, 49.5, 49.4, 49.3, 49.2, 49.1, 49.0, etc.
  • the term “about” indicates that the value listed may be somewhat altered, as long as the alteration does not result in nonconformance of the process or device. For example, for some elements the term “about” can refer to a variation of ⁇ 0.1%, for other elements, the term “about” can refer to a variation of ⁇ 1% or ⁇ 10%, or any point therein.
  • the term about when used for a measurement in mm may include +/ 0.1, 0.2, 0.3, etc., where the difference between the stated number may be larger when the state number is larger.
  • about 1.5 may include 1.2-1.8, where about 20, may include 19.0-21.0.
  • the term “substantially”, or “substantial”, is equally applicable when used in a negative connotation to refer to the complete or near complete lack of an action, characteristic, property, state, structure, item, or result.
  • a surface that is “substantially” flat would either completely flat, or so nearly flat that the effect would be the same as if it were completely flat.
  • processor may include a single core processor, a multi-core processor, multiple processors located in a single device, or multiple processors in wired or wireless communication with each other and distributed over a network of devices, the Internet, or the cloud.
  • functions, features or instructions performed or configured to be performed by a “processor”, may include the performance of the functions, features or instructions by a single core processor, may include performance of the functions, features or instructions collectively or collaboratively by multiple cores of a multi-core processor, or may include performance of the functions, features or instructions collectively or collaboratively by multiple processors, where each processor or core is not required to perform every function, feature or instruction individually.
  • a single FPGA may be used or multiple FPGAs may be used to achieve the functions, features or instructions described herein.
  • multiple processors may allow load balancing.
  • a server also known as remote, or cloud
  • a server also known as remote, or cloud processor may accomplish some or all functionality on behalf of a client processor.
  • processor also includes one or more ASICs as described herein.
  • processor may be replaced with the term “circuit”.
  • processor may refer to, be part of, or include processor hardware (shared, dedicated, or group) that executes code and memory hardware (shared, dedicated, or group) that stores code executed by the processor.
  • processor hardware shared, dedicated, or group
  • memory hardware shared, dedicated, or group
  • a non-transitory computer-readable storage medium comprising electronically readable control information stored thereon, configured in such that when the storage medium is used in a processor, aspects of the functionality described herein is carried out.
  • any of the aforementioned methods may be embodied in the form of a program.
  • the program may be stored on a non-transitory computer readable medium and is adapted to perform any one of the aforementioned methods when run on a computer device (a device including a processor).
  • the non-transitory, tangible computer readable medium is adapted to store information and is adapted to interact with a data processing facility or computer device to execute the program of any of the above-mentioned embodiments and/or to perform the method of any of the above mentioned embodiments.
  • the computer readable medium or storage medium may be a built-in medium installed inside a computer device main body or a removable medium arranged so that it can be separated from the computer device main body.
  • Non-limiting examples of the non-transitory computer-readable medium include, but are not limited to, rewriteable non-volatile memory devices (including, for example flash memory devices, erasable programmable read-only memory devices, or a mask read-only memory devices); volatile memory devices (including, for example static random access memory devices or a dynamic random access memory devices); magnetic storage media (including, for example an analog or digital magnetic tape or a hard disk drive); and optical storage media (including, for example a CD, a DVD, or a Blu-ray Disc).
  • rewriteable non-volatile memory devices including, for example flash memory devices, erasable programmable read-only memory devices, or a mask read-only memory devices
  • volatile memory devices including, for example static random access memory devices or a dynamic random access memory devices
  • magnetic storage media including, for example an analog or digital magnetic tape or a hard disk drive
  • optical storage media including, for example a CD, a DVD, or a Blu-ray Disc
  • Examples of the media with a built-in rewriteable non-volatile memory include but are not limited to memory cards; and media with a built-in ROM, including but not limited to ROM cassettes; etc. Furthermore, various information regarding stored images, for example, property information, may be stored in any other form, or it may be provided in other ways.
  • the term memory hardware is a subset of the term computer-readable medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Business, Economics & Management (AREA)
  • Physics & Mathematics (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Ultra Sonic Daignosis Equipment (AREA)

Abstract

L'invention concerne un système d'entraînement d'échographie de point d'intervention (POCUS), un procédé, un terminal qui peuvent être utilisés dans un environnement mobile tel qu'un véhicule en mouvement. Le système est conçu pour compenser le mouvement du véhicule en mouvement lors de la détermination d'une orientation d'une sonde par rapport à une cible. Après que l'orientation compensée de la sonde est déterminée, le système peut sélectionner et afficher une image ultrasonore parmi une pluralité d'images. L'image sélectionnée correspond à l'orientation déterminée de la sonde et à une position de la sonde par rapport à la cible, qui est basée sur un système de capteur de pression à l'aide d'une carte de pression flexible positionnée sur la cible, une pointe distale de la sonde étant pressée sur la cible par l'intermédiaire de la carte de pression flexible.
PCT/US2024/045609 2023-09-07 2024-09-06 Système de simulation d'échographie au point d'intervention, procédés et programmes pour une utilisation éducative dans des environnements médicaux non stationnaires Pending WO2025054466A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363581164P 2023-09-07 2023-09-07
US63/581,164 2023-09-07

Publications (1)

Publication Number Publication Date
WO2025054466A1 true WO2025054466A1 (fr) 2025-03-13

Family

ID=94924437

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/045609 Pending WO2025054466A1 (fr) 2023-09-07 2024-09-06 Système de simulation d'échographie au point d'intervention, procédés et programmes pour une utilisation éducative dans des environnements médicaux non stationnaires

Country Status (1)

Country Link
WO (1) WO2025054466A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100179428A1 (en) * 2008-03-17 2010-07-15 Worcester Polytechnic Institute Virtual interactive system for ultrasound training
US20110125404A1 (en) * 2009-11-20 2011-05-26 Qualcomm Incorporated Spatial alignment determination for an inertial measurement unit (imu)
US20150154890A1 (en) * 2013-09-23 2015-06-04 SonoSim, Inc. System and Method for Augmented Ultrasound Simulation using Flexible Touch Sensitive Surfaces
US20190231317A1 (en) * 2017-12-28 2019-08-01 Massachusetts Institute Of Technology Ultrasound scanning system
US20230213616A1 (en) * 2019-11-28 2023-07-06 Thales System and method for automatically harmonizing the position and/or orientation between an apparatus on board a mobile carrier and a reference frame of the mobile carrier

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20100179428A1 (en) * 2008-03-17 2010-07-15 Worcester Polytechnic Institute Virtual interactive system for ultrasound training
US20110125404A1 (en) * 2009-11-20 2011-05-26 Qualcomm Incorporated Spatial alignment determination for an inertial measurement unit (imu)
US20150154890A1 (en) * 2013-09-23 2015-06-04 SonoSim, Inc. System and Method for Augmented Ultrasound Simulation using Flexible Touch Sensitive Surfaces
US20190231317A1 (en) * 2017-12-28 2019-08-01 Massachusetts Institute Of Technology Ultrasound scanning system
US20230213616A1 (en) * 2019-11-28 2023-07-06 Thales System and method for automatically harmonizing the position and/or orientation between an apparatus on board a mobile carrier and a reference frame of the mobile carrier

Similar Documents

Publication Publication Date Title
CN101141929B (zh) 用于为介入设备生成空间路线图的方法、系统和用于保护其空间精度的质量控制系统
US8532738B2 (en) Visualization of catheter-tissue contact by map distortion
US20210174706A1 (en) Injection site training system
US10285618B2 (en) EIT measurement device, EIT measurement method and program
EP3193765B1 (fr) Système de traitement conçu pour coopérer avec un dispositif d'intervention de forme optique à détection activée
WO2007017771A3 (fr) Systeme de navigation de catheter
CN1973790B (zh) 用于对引入检查对象体内的医疗器械进行定位的装置
US20200242971A1 (en) Optical Surface Tracking for Medical Simulation
EP2233099A2 (fr) Système assistée par ordinateur pour guider un instrument chirurgical pendant diagnostic percutané où opération thérapeutique
CN112562094A (zh) 光学跟踪系统及光学跟踪系统的坐标系整合方法
JP2015139708A (ja) 改良された心電図記録（ｅｃｇ）グラフの表示
US11051694B2 (en) Systems and methods for tracking imaging attenuators
JP2007105145A (ja) 体脂肪測定装置、測定ユニットおよび体脂肪測定プログラム
CN107361843A (zh) 具有真实触感的沉浸式神经外科手术模拟方法
CN106491133A (zh) 不一致的基于场的贴片位置坐标的校正
CN109381260A (zh) 使用虚拟装置和物理3d模型以显现医疗装置在患者器官中的导航
US20130137988A1 (en) Method and Apparatus for the Augmentation of Physical Examination over Medical Imaging Data
WO2025054466A1 (fr) Système de simulation d'échographie au point d'intervention, procédés et programmes pour une utilisation éducative dans des environnements médicaux non stationnaires
US20180221093A1 (en) Systems and methods for soft tissue navigation
CN120267266A (zh) 基于电阻抗成像的头部空间定位方法、装置及系统
Kumar et al. A sensor system to assess the ocular digital massage in an ophthalmic anaesthesia training system
EP4129168B1 (fr) Système et procédé de localisation automatique de la position spatiale d'électrodes sur un corps de conducteur
CN112991854A (zh) 一种超声教学方法、装置及系统、电子设备
JP2001033534A (ja) 心内電気現象診断装置
CN223450474U (zh) 一种可标定穿刺装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24863682

Country of ref document: EP

Kind code of ref document: A1