[go: up one dir, main page]

WO2018089703A1 - Procédé et système de localisation et de cartographie simultanées, précises et à long terme avec détection d'orientation absolue - Google Patents

Procédé et système de localisation et de cartographie simultanées, précises et à long terme avec détection d'orientation absolue Download PDF

Info

Publication number
WO2018089703A1
WO2018089703A1 PCT/US2017/060954 US2017060954W WO2018089703A1 WO 2018089703 A1 WO2018089703 A1 WO 2018089703A1 US 2017060954 W US2017060954 W US 2017060954W WO 2018089703 A1 WO2018089703 A1 WO 2018089703A1
Authority
WO
WIPO (PCT)
Prior art keywords
robot
location
sensor
attitude
mapping system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2017/060954
Other languages
English (en)
Inventor
Saurav Agarwal
Suman CHAKRAVORTY
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Texas A&M University System
Texas A&M University
Original Assignee
Texas A&M University System
Texas A&M University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Texas A&M University System, Texas A&M University filed Critical Texas A&M University System
Priority to US16/342,273 priority Critical patent/US11536572B2/en
Publication of WO2018089703A1 publication Critical patent/WO2018089703A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1652—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with ranging devices, e.g. LIDAR or RADAR
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G01C21/1656—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments with passive imaging devices, e.g. cameras
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
    • G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
    • G05D1/0248—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means in combination with a laser
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0268—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means
    • G05D1/0274—Control of position or course in two dimensions specially adapted to land vehicles using internal positioning means using mapping information stored in a memory device

Definitions

  • the present disclosure relates to robotic mapping.
  • accurate long term simultaneous localization and mapping with absolute orientation sensing are provided.
  • SLAM Simultaneous Localization and Mapping
  • SLAM SLAM visual-inertial localization methods exhibit error of ⁇ 0.3%-0.5% which may be unsuitable for precision tasks, e.g. for a 25km trajectory, such error results in 75m-125m position error.
  • Some robots may be operated indoors where GPS is unavailable or degraded. For example, material handling robots that move goods (boxes, pallets etc.) in large warehouses and distribution centers do not have access to GPS satellites. Installing beacons, markers, or guide cables is expensive, and robots are often expected start without prior knowledge of the map of their operating environment. Further, warehouse environments are highly dynamic due to a mix of industrial vehicles (pallet jacks, forklifts etc.) and people moving rapidly across large floor spaces, thus a robot must continuously update its knowledge of the map and react to changes in its vicinity.
  • robots may be driven manually to gather and store measurements (e.g., laser scans, visual landmarks etc.). This data is then processed offline to build detailed maps. Computed maps are subsequently used for positioning and navigation by taking measurements to known features in the world.
  • measurements e.g., laser scans, visual landmarks etc.
  • a vertical (e.g., upward facing) imaging sensor to compute vehicle attitude (e.g., orientation or heading) and combines the computed vehicle attitude with range bearing measurements (from an imaging sensor, LiDAR, sonar, etc.) to features in the vicinity of the vehicle to compute accurate position and map estimates.
  • a mapping system may comprise an upward facing sensor; a range bearing sensor; and a processor in communication with the upward facing sensor and the range bearing sensor.
  • the processor may be configured to determine an attitude of the mapping system based upon first data received from the upward facing sensor; determine a location of local landmarks based upon second data received from the range bearing sensor; and determine a location of the mapping system based upon the attitude and the location of the local landmarks.
  • the first data may comprise bearing measurements to one or more features.
  • the upward facing sensor may be a camera.
  • the mapping system may further comprise an inertial sensor, wherein the processor may be further configured to determine a relative pose based upon a scan match, wherein the scan match comprises inputs of the second data and a third data received from the inertial sensor.
  • the processor configured to determine the location of the mapping system may comprise the processor configured to fuse a result of the scan match with the attitude.
  • the processor configured to fuse the result of the scan match with the attitude may comprise the processor configured to input the result of the scan match and the attitude to a Kalman filter.
  • the processor may be further configured to update a map based upon the location of the mapping system and the location of the local landmarks.
  • an autonomous vehicle may comprise an upward facing sensor; a range bearing sensor; and a mapping system in communication with the upward facing sensor and the range bearing sensor.
  • the mapping system may comprise a processor configured to determine an attitude of the autonomous vehicle based upon first data received from the upward facing sensor; determine a location of local landmarks based upon second data received from the range bearing sensor; and determine a location of the autonomous vehicle based upon the attitude and the location of the local landmarks.
  • the first data may comprise bearing measurements to one or more features.
  • the upward facing sensor may be a camera.
  • the autonomous vehicle may further comprise an inertial sensor, wherein the processor may be further configured to determine a relative pose based upon a scan match, wherein the scan match comprises inputs of the second data and a third data received from the inertial sensor.
  • the processor configured to determine the location of the autonomous vehicle may comprise the processor configured to fuse a result of the scan match with the attitude.
  • the processor configured to fuse the result of the scan match with the attitude may comprise the processor configured to input the result of the scan match and the attitude to a Kalman filter.
  • the processor may be further configured to update a map based upon the location of the autonomous vehicle and the location of the local landmarks.
  • a method for mapping may comprise determining an attitude of a mapping system based upon first data received from an upward facing sensor; determining a location of local landmarks based upon second data received from a range bearing sensor; and determining a location of the mapping system based upon the attitude and the location of the local landmarks.
  • the first data may comprise bearing measurements to one or more features.
  • the method may further comprise updating a map based upon the location of the mapping system and the location of the local landmarks.
  • Figure 1 is a diagram of an embodiment of a robot configured for long term SLAM with absolute orientation sensing
  • Figure 2 is a flow diagram of an embodiment of a method for long term SLAM with absolute orientation sensing
  • Figure 3A is a diagram of an embodiment of robot to feature relative measurement by a robot
  • Figure 3B is a diagram of an embodiment of feature to feature relative measurement by a robot
  • Figure 4 is a diagram of an embodiment of a robot viewing a feature from two poses in robot to feature relative measurement
  • Figure 5A is a graph of error growth in estimate of a last pose as a function of how far a robot moves of an embodiment described herein;
  • Figure 5B is a graph of reduction in error growth rate as a number of landmarks increases of an embodiment described herein;
  • Figure 5C is a graph of localization error for a feature bank most distant from a starting location before and after loop closure of an embodiment described herein;
  • Figure 5D is a graph of localization error in a last pose after loop closure as the trajectory length (number of banks mapped) increases of an embodiment described herein;
  • Figure 6A is a diagram of an embodiment of a robot starting to make observations to a first bank of features;
  • Figure 6B is a diagram of an embodiment of a robot starting to making relative observations between a second bank of features and a third bank of features;
  • Figure 6C is a diagram of an embodiment of a robot moving towards its start location
  • Figure 6D is a diagram of an embodiment of a robot re-observing the first bank of features
  • Figure 7A is a graph of linear error growth in estimate of a last pose as a function of how far a robot moves away from its start of an embodiment described herein;
  • Figure 7B is a graph of reduction in error growth rate as a number of landmarks increases of an embodiment described herein;
  • Figure 7C is a graph of localization error for a feature bank most distant from a starting location before and after loop closure of an embodiment described herein;
  • Figure 7D is a graph of localization error in a last bank after loop closure as the trajectory length (number of banks mapped) increases of an embodiment described herein;
  • Figure 8A is a diagram of an embodiment of robot trajectory over 5 square kilometers (km) with a 25.9 km trajectory;
  • Figure 8B is a diagram of an embodiment of robot trajectory over 10 square km with a 107.9 km trajectory
  • Figure 9A is a graph of average terminal pose localization error as the bank size increases with the 25.9 km trajectory of an embodiment described herein;
  • Figure 9B is a graph of average terminal pose localization error as the bank size increases with the 107.9 km trajectory of an embodiment described herein;
  • Figure 10A is a diagram of an embodiment of a camera view of a ceiling.
  • Figure 10B is a diagram of an embodiment of a thresholded image of the ceiling.
  • Estimation drift during exploration may be caused by robot heading uncertainty.
  • reliable absolute orientation measurements may not be available in SLAM. These approaches may rely on odometery and relative pose or feature measurements to estimate robot orientation and position.
  • Embodiments of the present disclosure may attain an accuracy (i.e., error in position as percentage of distance travelled) of 0.0016% for a 107.9 km trajectory without loops using absolute orientation sensor technology.
  • An instantaneous location and/or heading of a robot may be referred to herein as a pose.
  • a robot using SLAM with a heading sensor in Extended Kalman Filter-based SLAM may move much further into unknown areas with consistent estimates. Consistency in a filter implies that the estimation uncertainty captures the true error; conversely an inconsistent filter does not capture true error and may give a false sense of confidence in the robot's belief.
  • Two methods of SLAM may be used, filtering-based methods and graph- based methods.
  • Filtering-based methods may maintain a recursive estimate using current robot pose and map.
  • Graph-based methods use robot poses as nodes of a graph and constraints as edges.
  • Graph based SLAM may use a two-pronged approach, 1 ) a front-end which maintains an estimate of the robot pose and computes data association between current and past observations, and 2) a back-end which solves the non-linear optimization to compute the history of robot poses.
  • EKF-SLAM shows that heading, i.e., robot orientation may be unobservable in the EKF-SLAM formulation.
  • Analysis of the consistency of EKF-SLAM further shows that heading estimation errors may be a cause of inconsistency due to erroneous Jacobian computations.
  • EKF-SLAM heading estimate may drift.
  • the EKF-SLAM filter may be overconfident, i.e., uncertainty estimates may not reflect the true error and heading uncertainty may be the major cause of inconsistency.
  • non-linear optimization techniques may be used to solve for the maximum likelihood estimate.
  • Graph-based SLAM techniques may rely on an initial guess to bootstrap the optimizer. This initial guess may be based on odometery and may be arbitrarily bad leading to local minima.
  • a special property of SLAM is that when robot orientation is known, SLAM may be posed as a linear estimation problem in position.
  • Estimating orientation as the first step and using these estimates to initialize pose graph optimization may result in a robust solution.
  • the separation of orientation and position may be extended to feature-based SLAM.
  • Estimating orientation first may avoid catastrophic failure, e.g. local minima.
  • a robot or autonomous vehicle includes a long term SLAM with absolute orientation sensing system.
  • the robot or autonomous vehicle may be configured to travel over land, through air, water, or any other medium of travel.
  • the system may include a sensor array.
  • the sensor array may include one or more orientation sensors, odometery sensors (such as inertial measurement units, wheel encoders, etc.), and at least one exteroceptive sensor.
  • the orientation sensor may be, for example, a star tracker, a sun sensor, a magnetometer, or a gyrocompass, or an upward facing camera.
  • the inertial sensor may include a combination of accelerometer and gyroscope which measure the vehicles acceleration and angular rates.
  • the odometery sensors may include a rotary encoder coupled to a wheel of the vehicle, or other device for determining distance traveled by the vehicle over a time interval.
  • the exteroceptive sensor i.e.,. a range bearing sensor
  • the system includes a processor (e.g., a microprocessor, digital-signal-processor, etc.) coupled to the sensor array.
  • the processor receives measurements from the various sensors of the sensor array.
  • the navigation system also includes memory (e.g., volatile or non-volatile semiconductor memory) coupled to the processor.
  • the processor may store the measurements received from the sensor array in the memory.
  • the memory may also store instructions that can be executed by the processor to process the measurements and provide the long term SLAM with absolute orientation sensing functionality described herein.
  • the processor may process the measurements to determine a location of the vehicle as described herein, to control a motor or other propulsion system of the vehicle, and/or to control a steering system of the vehicle based on the determined location of the vehicle.
  • Magnetometers may function adequately when the Earth's magnetic field is not corrupted by external influences.
  • Gyrocompasses measure the planet's rotation to determine accurate heading with respect to geographic north.
  • Microelectromechanical Systems (MEMS)-based gyrocompasses have been proposed and may be useful for providing absolute orientation. The methods presented in this disclosure may use magnetometer or gyrocompass for orientation sensing alone or in combination with a star tracker or sun sensor or other upward facing sensor.
  • FIG. 1 is a diagram of an embodiment of a vehicle 100 with a long term SLAM with absolute orientation sensing system.
  • the long term SLAM with absolute orientation sensing system may comprise an upward facing sensor 1 10, a range bearing sensor 120, a camera 130, an inertial measurement unit 140, wheel encoders 150, and a processor 160.
  • the processor 160 may receive inputs from one or more of the upward facing sensor 1 10, the range bearing sensor 120, the camera 130, the inertial measurement unit 140, or the wheel encoders 150.
  • Upward facing sensor 1 10 may be used to determine an orientation of the vehicle 100 based upon landmarks above the vehicle 100.
  • the upward facing sensor 1 10 may be a camera that captures images of the ceiling of a building where the vehicle 100 is located.
  • the upward facing sensor 1 10 may be a star tracker that captures images of stars or other landmarks in the sky.
  • the orientation determined by the upward facing sensor 1 10 may be used with data from one or more of the range bearing sensor 120, the camera 130, the inertial measurement unit 140, or the wheel encoders 150 by the processor 160 to determine a location and orientation of the vehicle 100. This information may be used to create or update a virtual map of the environment where the vehicle is operating.
  • One or more of the range bearing sensor 120, the camera 130, the inertial measurement unit 140, or the wheel encoders 150 may be optional in some embodiments.
  • the range bearing sensor 120 may a LiDAR, sonar, or some other sensor that detects objects in the environment surrounding the vehicle 100.
  • Camera 130 may a stereo camera or some other image capture device that detects objects in the environment surrounding the vehicle 100.
  • the IMU 140 may be configured to measure the movement of the vehicle 100.
  • wheel encoders 150 may count revolutions of the wheels and determine a distance traveled by vehicle 100.
  • FIG. 2 is a flow diagram of an embodiment of a method for long term SLAM with absolute orientation sensing 200.
  • the embodiment may be implemented by a mapping system.
  • an orientation sensing camera e.g., upward facing sensor 1 10
  • the heading may be provided to a LOGO slam solver 260.
  • the LOGO slam solver 260 may be executed by a processor, e.g., processor 160.
  • the heading may also be provided to a Kalman filter 250.
  • a scan matcher 240 may determine a relative pose based on input from a range finding sensor 220 and/or a movement sensor 230.
  • Range finding sensor 220 may be a LiDAR, sonar, radar or some other sensor configured to detect objects and their range from the vehicle.
  • Movement sensor 230 may an IMU, a wheel encoder, or any other sensor configured to detect movement of the vehicle and estimate a distance traveled based on the movement detected.
  • the relative pose information from scan matcher 240 may be provided to the Kalman filter 250.
  • the Kalman filter 250 may fuse the relative pose information with the heading to determine a location of the vehicle. If the mapping system determines with a predetermined accuracy that the vehicle is revisiting a previously visited location, then the mapping system may assume loop closure has occurred. If loop closure has occurred, the LOGO slam solver 260 may use the output of the scan matcher 240 along with the heading to update the map.
  • the mapping system may perform incremental mapping. While the embodiment of Figure 2 is described in terms of a mapping system on a vehicle, the method may be implemented by mapping system on a handheld device or any other device comprising the elements described in the description of Figure 2. Further, the vehicle may be autonomous or manually controlled.
  • Star trackers may be automated camera-based devices that compute inertial attitude with high accuracy. Some star trackers may deliver RMS error down to 10 arcseconds or 0.0028° by using measurements to known celestial bodies and comparing them to star charts. Star trackers may rely on measurements to persistent beacons in space whose trajectories across the sky relative to Earth or other planets may be fixed with great precision based on long-term astronomical observations. Star trackers may be used when GPS is unavailable during both day and night operation.
  • the method of the present disclosure may combine any combination of proprioceptive (odometery) sensors (e.g., inertial sensors, wheel odometer etc.) and/or exteroceptive sensors (e.g., camera, LIDARs etc.) with star tracking or sun sensors for accurate global navigation.
  • proprioceptive odometery
  • exteroceptive sensors e.g., camera, LIDARs etc.
  • x k £ X, 3 ⁇ 4 e U, and z k E TL represent the system state, control input, and observation at time step k respectively, where , ⁇ , ⁇ . denote the state, control, and observation spaces respectively.
  • a keyframe pose is designated as "x .
  • the robot belief is defined as the probability distribution over all possible states of the robot.
  • Z k is the history of observations up to time t k .
  • the map (unknown at t 0 ) is a set of landmarks (i.e., features) distributed throughout the environment.
  • the y -th landmark is defined as / . and l j as the estimate of / . .
  • the observation for landmark I j at time t k is denoted by z ⁇ e z k .
  • 'df is the relative feature measurement, from feature /, to / . in the local frame of the robot at time t k .
  • a relative feature measurement is an estimate of the displacement vector from one feature to another.
  • ⁇ J k are relative positions of features /, and I j respectively with respect to the robot in its local frame. Thus it is linear in positions of the two features in the local frame.
  • C(Q k ) denote the direction cosine matrix (DCM) of the robot orientation at state x k .
  • C is a function of the robot orientation parameter (e.g., Euler angles, Quaternions etc.).
  • the position estimation problem may be linear.
  • Embodiments disclosed herein may include various approaches for long-term localization.
  • Two such approaches are Robot to Feature Relative Measurement Model (R2F) and Feature to Feature Relative Measurement Model (F2F).
  • R2F may be designed for systems where a robot moves continuously and has access to odometery, orientation sensor and exteroceptive sensing e.g., Lidar, cameras, etc.
  • R2F may convert local relative measurements from the robot to features at each pose to global frame measurements as shown in Figure 3A. These measurement may then be used to solve a linear estimation problem of the robot and feature positions to attain a high-degree of accuracy.
  • F2F may extend the R2F for systems where extremely high-precision is required.
  • a robot may observe four banks of features prior to final pose, i.e., last keyframe. There may be two features in each bank. For example, the first bank indicated by the dashed ellipse may include and / 2 . A bank may be described as the set of features observed at a particular pose.
  • Figure 3A shows how R2F makes relative measurements from robot to features. The relative measurements are indicated by the lines with arrowheads. The keyframes are indicated by the black and white triangles and indicate positions where the robot may make relative measurements.
  • the R2F approach includes the following steps. 1 ) Range bearing measurements to features are converted into relative displacement vectors from robot to features at each pose as the robot moves as shown in Figure 4.
  • the robot may detect the landmark /, from two poses x 1 and x 2 .
  • the transformation of local relative measurements to the global frame may be used to solve for robot and feature positions.
  • An upper threshold may be set on the number of keyframes to keep in the map after which the oldest keyframe is deleted. The first pose may be saved even if it is the oldest keyframe.
  • L k ⁇ z 2 ,... , z, k " ⁇ be the set of range bearing observations to the set of landmarks visible at time t k .
  • the position of landmark /, in robot's local frame is 1 M k - g(z ⁇ ) .
  • the vector of local robot to feature relative measurements is:
  • [p *r ,l *r are of interest.
  • Eq. 5 embodiments may replace p 0 with estimates of past keyframes and corresponding landmarks observed at those keyframes.
  • the R2F approach may be analyzed for location accuracy and the effect of loop closure as a robot explores an unknown map.
  • the robot makes independent measurements of global frame displacement from robot to feature and error covariance of every global relative measurement is R a .
  • independence may be achieved by capturing heading observations such that the same heading observation is not used to transform all local measurements to world frame.
  • the first and last pose of the robot are considered as keyframes with first pose known.
  • a robot may move into unknown areas and make measurements to a bank of features at each timestep as shown in Figure 3A.
  • Figures 5A-5D show analysis of the pose and map estimation error as the robot moves and the effect of loop closure in the estimates using the R2F approach.
  • Figure 5A shows that the error growth is linear as final pose moves further away from its start.
  • Figure 5B the rate of error growth drops as ⁇ where ⁇ ⁇ is the number of features in each bank.
  • ⁇ ⁇ is the number of features in each bank.
  • Loop closure may be considered a necessary action in SLAM to limit error growth. However, in the case of long-term point-to-point navigation, loop closure may not be possible.
  • the effect of loop closure on the bank of features farthest from the start location in a loop and on the last pose is analyzed in Figures 5C and 5D.
  • Figure 5C shows that the effect of loop closure diminishes as the bank size increases as shown by the gap between solid and dashed lines, e.g., after mapping 9 feature banks, with 1 feature in each bank error drops to 60.49% and with 4 features drops to 66.14% after loop closure.
  • the ratio of error growth rate after and before loop closure is 0.5185, i.e., loop closure approximately halves the error growth in the farthest bank.
  • loop closure approximately halves the error growth in the farthest bank.
  • Figure 5D shows that error in last pose after loop closure converges to a fixed value as the trajectory length increases. This indicates that estimation error in the last pose is dominated by the relative measurement to the first bank, i.e., as the trajectory length grows, the effect of the longer "pathway" from start may have almost no effect on estimation accuracy.
  • the mapping algorithm may transform range bearing observations from robot to features into relative position measurements between features by fusing them with heading estimates from a heading sensor.
  • Figure 3B shows how F2F estimates keyframe poses and may neglect odometery between poses. Localization accuracy may be a function of how far the robot has moved and the number of features in each bank.
  • the key steps of F2F include the following. 1 ) Range bearing measurements to features may be converted into relative displacement vectors between the features at each pose as the robot moves. 2) Robot to feature relative position measurements may be acquired at keyframes, then a linear estimation problem may be solved for keyframe poses and map features using the recorded data.
  • An upper threshold may be set on the number of keyframes to keep in the map after which the oldest keyframe is deleted. The first pose may be saved even if it is the oldest keyframe.
  • Equation (9) shows that 'df is independent of robot position p k and orientation ⁇ , .
  • the vector of local relative measurements is as follows:
  • Equation (10) that though measurements to each feature are independent, the set of relative feature measurements may be correlated due to the correlations between relative measurements using the same range-bearing measurement. This is where a difference from some approaches arises. There may be a benefit of capturing independent measurements of relative feature displacements. Independent may be achieved by capturing heading and range bearing observations by stopping the robot at certain times (e.g., keyframes).
  • embodiments have keyframe K x k .
  • At each pose embodiments have a noisy unbiased heading measurement which gives the vector of orientation estimates 0 O:fc ⁇ N(9 0 , k , Re 0 . k ) .
  • Vh is the Jacobian of measurement function h given by
  • the localization error grows as a robot explores an unknown map. It may be assumed that the robot makes independent relative feature measurements and the error covariance of each global relative measurement is R a .
  • the first and last pose of the robot are considered as keyframes with the first pose known.
  • ⁇ has a symmetric tridiagonal structure that permits an analytical inversion to compute the error covariance matrix.
  • An analytical solution of the error covariance matrix may allow prediction of feature localization uncertainty at the goal given certain environment characteristics, e.g., the number of features in each bank and how many banks the robot may map as it traverses to the goal.
  • the capability to predict future localization uncertainty implies that given a desired goal accuracy, active sensing to control error growth may be applied.
  • the linear estimation problem of equation (12) may be analytically solved and error covariance ⁇ computed for multiple cases by varying bank size n fi and the number of banks that the robot maps.
  • Figures 7A, 7B, 7C, and 7D show results of analysis of the feature mapping and localization error as the robot moves and the effect of loop closure.
  • Figure 7A shows that localization error grows linearly as the robot moves away from the start location.
  • the error growth rate shown in Figure 7B is inversely proportional to the square of the size of each feature bank, i.e., -7- where ⁇ holiday is the number features in one bank.
  • error growth may be controlled by the number of features mapped in each bank.
  • Figures 6A, 6B, 6C, and 6D show a simple graphical depiction of loop closure, the left half of each figure shows the robot 600 making range bearing measurements and the right half shows the map being built.
  • Fig. 6D concerns the estimation error of the farthest feature bank (encircled by ellipse 610) and the last pose (encircled by ellipse 620).
  • Figures 6A, 6B, 6C, and 6D depict loop closure when a robot makes relative feature measurements while moving in a circular trajectory.
  • the exercise of solving equation (16) is repeated, albeit with loop closure and the error covariance ⁇ is computed.
  • the results of this analysis are plotted in Figures. 7C and 7D.
  • the ratio of error growth rate with loop closure to error growth rate without loop closure is a constant value of 0.5181 for all values of n fl computed as the ratio of slopes of curves plotted in Figure 7C before and after loop closure.
  • Figure 7D shows that error in last pose after loop closure converges to a fixed value as the trajectory length (number of banks) increases.
  • Fig. 6D (right half), there are two “pathways" that link the farthest bank of features to the first bank. Prior to loop closure there is only one path for the relative measurements to constrain feature estimates to the first bank, however, after loop closure there is a second pathway from the opposite direction.
  • An interesting point to be made is that if two observations ( R a is halved) were taken for each relative feature displacement, embodiments would effectively end up with the same estimation error at the farthest bank were the robot not to close the loop.
  • the second observation shows that the estimation error in the last bank of features is dominated by the relative measurement to the first bank, i.e., as the trajectory length grows, the effect of the longer "pathway" from start has almost no effect on estimation accuracy.
  • Figures 8A and 8B present simulation scenarios for waypoint following in a 2D environment. These simulations study the case of a long term exploration task where a robot may not visit prior locations.
  • the robot may move at a speed of lO m/s and simulation time step is 0.05 s.
  • Figure 8A shows a scenario with a 2D world (5km x 5km) with a trajectory of length 25.9km.
  • Figure 8A shows a scenario with a 2D world (10km x 10km) with a trajectory of length 107.9km. Both scenarios are obstacle free and landmarks are randomly placed along the trajectory. Both trajectories terminate far from the start location and there are no loop closures.
  • Trigonometric functions of robot orientation are the primary source of non- linearity in SLAM which makes predicting long-term error growth difficult.
  • Analysis shows that given unbiased heading measurements, localization error growth is linear as a robot moves away from its start location. The error growth rate may be controlled by the quality and number of measurements. Further, loop closure may be avoided when absolute heading is available as the same effect may be achieved by taking prolonged measurements.
  • Feature estimates may be consistent due to the linear nature of the problem which may lead to a global minimizer. Consistent feature estimates may lead to better localization as the robot has a reliable notion of uncertainty about its estimated mean.
  • the following embodiments describe systems that operate in an indoor environment where the upward facing sensor detects objects on or attached to the ceiling of an indoor environment. Some of the techniques in these embodiments may be used in either the indoor and outdoor embodiments.
  • the system may use stable structural features as structural cues, for example, in a warehouse building, the ceiling corrugation or ceiling lights are usually aligned along one direction.
  • the orientation of the building may be fixed in the direction of ceiling direction and a vehicle may estimate its orientation with respect to the building by observing the ceiling.
  • W A A'p
  • A' is a matrix composed of elements in the set ⁇ -1,0,1 ⁇
  • p is the vector of robot positions in the global frame.
  • the indoor approach includes three aspects. 1 ) Sensing absolute orientation of the robot using structural cues. 2) Fusing absolute orientation measurements to the front-end, i.e., a scan matching algorithm. 3) Solving the batch optimization problem to compute global estimates at loop closure.
  • Independent absolute orientation estimates of the robot heading may be determined.
  • the orientation sensing method may detect structural features of the environment.
  • the ceiling structure usually has straight line features.
  • ceiling corrugation or ceiling lights in most industrial buildings are aligned along one direction which may be detected by a ceiling facing camera.
  • Ceiling direction may be estimated as follows.
  • the line features may be an edge of a light fixture or a part of a corrugated ceiling.
  • compute orientation of the line features in the image frame The system may then create a histogram of the orientation data with bins of width b in range [0, 2 ⁇ ). The system may then create a window of width W around the bin with highest frequency, i.e., the bin with maximum observations. The system then computes the weighted mean of observations in the window. The ceiling direction may then be computed as an angle $ ⁇ ⁇ ⁇ [ ⁇ , ⁇ ).
  • Line direction may be ambiguous, i.e., it may be difficult to differentiate north from south. Therefore, gyro data may be used in the intermediate time between absolute orientation measurements. Gyros may provide data at >100 Hz and therefore may be used to account for the angle wrap-around issue in absolute orientation detection. To estimate the robot heading, initial heading at time t 0 is assumed to be known.
  • Fusing orientation information may add robustness to the front-end. Small errors in relative orientation measurements may add up over time to create a super linear growth in localization error. This problem may arise from the non-linear nature of the orientation.
  • the SLAM back-end may use the graph generated by the front-end along with absolute orientation data and solve a two-step optimization problem.
  • the first step may be the estimation of robot orientation using the absolute orientation and relative orientation measurements followed by a second step in which a linearized least-squares optimization problem may be solved for the robot position.
  • Robot orientation ⁇ ⁇ (- ⁇ , ⁇ ] thus as the robot navigates, the relative orientation measurements may not provide information about the angle wrap around.
  • ⁇ j be the relative orientation measurement from pose x ; to x j , then
  • kj j is the integer ambiguity.
  • the integer ambiguity can be simply be calculated as:
  • R R(0) is the corresponding composition of rotation matrices parametrized by the estimated heading 0
  • p is the vector of robot positions
  • A is a matrix with each row containing elements of the set [-1 ,0 +1 ]
  • Vh w is the Jacobian of measurement function h w given by:
  • Vh w [R M ' ⁇ ]
  • a robot may be configured with a LiDAR with 360 sensing, a monocular ceiling facing camera, an IMU, and a processor.
  • the robot may be deployed in a warehouse where GPS signal is degraded or unavailable.
  • the ceiling of the warehouse may be equipped with rectangular light fixtures at regular intervals which may be leveraged for orientation estimation.
  • the processor may threshold the image such that a binary image is created.
  • the ceiling lights may appear as rectangular bright spots while rest of the image appears black.
  • Figure 10A shows the ceiling camera's view and
  • FIG. 10B shows the thresholded binary image. Heading estimates may be computed at 30 Hz.
  • a first image captured by the camera may be used to determine an orientation of the robot.
  • the edge of the light fixture in the picture may be determined to be 0 degrees.
  • the angle of the light fixture in the subsequent image may be determined relative to 0 degree angle. This determination may be used to determine the absolute orientation.
  • the processor may create a map of the warehouse based on inputs from the LiDAR, camera, and IMU using the techniques described above.
  • Coupled or “couples” is intended to mean either an indirect or a direct connection. Thus, if a first device couples to a second device, that connection may be through a direct connection, or through an indirect connection accomplished via other devices and connections.
  • the term "software” includes any executable code capable of running on a processor, regardless of the media used to store the software.
  • code stored in memory e.g., non-volatile memory
  • embedded firmware is included within the definition of software.
  • the recitation "based on” is intended to mean “based at least in part on.” Therefore, if X is based on Y, may be based on Y and any number of other factors.

Landscapes

  • Engineering & Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Electromagnetism (AREA)
  • Mechanical Engineering (AREA)
  • Optics & Photonics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Control Of Position, Course, Altitude, Or Attitude Of Moving Bodies (AREA)

Abstract

Des modes de réalisation de l'invention concernent un procédé et un système qui utilisent un capteur d'imagerie vertical ou orienté vers le haut pour calculer l'attitude, l'orientation ou la direction d'un véhicule puis combine l'attitude, l'orientation ou la direction calculée du véhicule à des mesures de distance et de relèvement provenant d'un capteur d'imagerie, d'un LiDAR, d'un sonar, etc. par rapport à des éléments à proximité du véhicule de façon à calculer des estimations de position et de carte précises.
PCT/US2017/060954 2016-11-09 2017-11-09 Procédé et système de localisation et de cartographie simultanées, précises et à long terme avec détection d'orientation absolue Ceased WO2018089703A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/342,273 US11536572B2 (en) 2016-11-09 2017-11-09 Method and system for accurate long term simultaneous localization and mapping with absolute orientation sensing

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201662419624P 2016-11-09 2016-11-09
US62/419,624 2016-11-09

Publications (1)

Publication Number Publication Date
WO2018089703A1 true WO2018089703A1 (fr) 2018-05-17

Family

ID=62109930

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2017/060954 Ceased WO2018089703A1 (fr) 2016-11-09 2017-11-09 Procédé et système de localisation et de cartographie simultanées, précises et à long terme avec détection d'orientation absolue

Country Status (2)

Country Link
US (1) US11536572B2 (fr)
WO (1) WO2018089703A1 (fr)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108983769A (zh) * 2018-06-22 2018-12-11 理光软件研究所（北京）有限公司 即时定位与地图构建的优化方法及装置
CN109489658A (zh) * 2018-10-18 2019-03-19 深圳乐动机器人有限公司 一种运动目标定位方法、装置及终端设备
CN110271015A (zh) * 2019-06-25 2019-09-24 浙江大学 一种摄像机姿态可调的主动式视觉slam移动平台
TWI679511B (zh) * 2018-08-22 2019-12-11 和碩聯合科技股份有限公司 軌跡規劃方法與系統
CN112484740A (zh) * 2021-02-02 2021-03-12 奥特酷智能科技(南京)有限公司 用于港口无人物流车的自动建图与自动地图更新系统
CN112946660A (zh) * 2021-01-28 2021-06-11 西北工业大学 一种基于多波束前视声呐的同时定位与建图方法
US20210263531A1 (en) * 2018-06-15 2021-08-26 Orange Mapping and simultaneous localisation of an object in an interior environment
CN115453599A (zh) * 2022-08-11 2022-12-09 国网江苏省电力有限公司盐城供电分公司 一种多传感器协同的管道机器人精准定位方法

Families Citing this family (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018191818A1 (fr) 2017-04-18 2018-10-25 Clearpath Robotics Inc. Véhicule de transport de matériau autonome sans conducteur
WO2019090417A1 (fr) * 2017-11-10 2019-05-16 Clearpath Robotics Inc. Systèmes et procédés de mise à jour d'un plan électronique
US11960008B2 (en) * 2018-05-17 2024-04-16 Cmos Sensor, Inc. Method and system for pseudo 3D mapping in robotic applications
CN111655542A (zh) * 2019-04-23 2020-09-11 深圳市大疆创新科技有限公司 一种数据处理方法、装置、设备及可移动平台
US11960297B2 (en) * 2019-05-03 2024-04-16 Lg Electronics Inc. Robot generating map based on multi sensors and artificial intelligence and moving based on map
DE102019123538A1 (de) * 2019-09-03 2021-03-04 Bayerische Motoren Werke Aktiengesellschaft Verfahren und Vorrichtung zur Ermittlung einer Trajektorie eines Fahrzeugs
JP2021120837A (ja) * 2020-01-31 2021-08-19 ソニーグループ株式会社 自律移動装置、自律移動制御方法、並びにプログラム
US11875519B2 (en) * 2020-08-13 2024-01-16 Medhat Omr Method and system for positioning using optical sensor and motion sensors
US12372370B2 (en) * 2020-11-23 2025-07-29 Electronics And Telecommunications Research Institute Method and apparatus for generating a map for autonomous driving and recognizing location
CN112833890B (zh) * 2020-12-30 2024-08-13 深圳市海柔创新科技有限公司 地图构建方法、装置、设备、机器人及存储介质
CN112923919B (zh) * 2021-01-21 2023-05-16 湖南云箭格纳微信息科技有限公司 基于图优化的行人定位方法及系统
JP7582504B2 (ja) * 2021-03-15 2024-11-13 オムロン株式会社 ロボットの自己位置推定に用いる環境マップの更新方法及び更新装置
CN115435775A (zh) * 2022-09-23 2022-12-06 福州大学 基于拓展卡尔曼滤波的多传感器融合slam方法
JP2025535047A (ja) * 2022-10-14 2025-10-22 オムロン株式会社 移動ロボット位置特定システム
CN115790582A (zh) * 2022-11-29 2023-03-14 电子科技大学 一种用于gnss拒止环境的无人机全局定位方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040168148A1 (en) * 2002-12-17 2004-08-26 Goncalves Luis Filipe Domingues Systems and methods for landmark generation for visual simultaneous localization and mapping
US20130048722A1 (en) * 2011-08-30 2013-02-28 Bruce L. Davis Methods and arrangements for sensing identification information from objects
US20160033272A1 (en) * 2014-08-04 2016-02-04 Elbit Systems Of America, Llc Systems and methods for northfinding
US20160070265A1 (en) * 2014-09-05 2016-03-10 SZ DJI Technology Co., Ltd Multi-sensor environmental mapping
WO2016077703A1 (fr) * 2014-11-13 2016-05-19 Worcester Polytechnic Institute Localisation et cartographie visuelles simultanées évolutives assistées par gyroscope
WO2016130719A2 (fr) * 2015-02-10 2016-08-18 Amnon Shashua Carte éparse pour la navigation d'un véhicule autonome

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4429016A1 (de) * 1994-08-16 1996-02-22 Linde Ag Navigation für fahrerlose Fahrzeuge
US7437230B2 (en) * 2003-03-20 2008-10-14 Hemisphere Gps Llc Satellite based vehicle guidance control in straight and contour modes
WO2012103002A2 (fr) * 2011-01-24 2012-08-02 Sky-Trax, Inc. Suivi inférentiel de charge
US10725478B2 (en) * 2013-07-02 2020-07-28 The Boeing Company Robotic-mounted monument system for metrology systems
JP6608456B2 (ja) * 2015-10-05 2019-11-20 パイオニア株式会社 推定装置、制御方法、プログラム及び記憶媒体

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040168148A1 (en) * 2002-12-17 2004-08-26 Goncalves Luis Filipe Domingues Systems and methods for landmark generation for visual simultaneous localization and mapping
US20130048722A1 (en) * 2011-08-30 2013-02-28 Bruce L. Davis Methods and arrangements for sensing identification information from objects
US20160033272A1 (en) * 2014-08-04 2016-02-04 Elbit Systems Of America, Llc Systems and methods for northfinding
US20160070265A1 (en) * 2014-09-05 2016-03-10 SZ DJI Technology Co., Ltd Multi-sensor environmental mapping
WO2016077703A1 (fr) * 2014-11-13 2016-05-19 Worcester Polytechnic Institute Localisation et cartographie visuelles simultanées évolutives assistées par gyroscope
WO2016130719A2 (fr) * 2015-02-10 2016-08-18 Amnon Shashua Carte éparse pour la navigation d'un véhicule autonome

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210263531A1 (en) * 2018-06-15 2021-08-26 Orange Mapping and simultaneous localisation of an object in an interior environment
CN108983769A (zh) * 2018-06-22 2018-12-11 理光软件研究所（北京）有限公司 即时定位与地图构建的优化方法及装置
CN108983769B (zh) * 2018-06-22 2022-06-21 理光软件研究所（北京）有限公司 即时定位与地图构建的优化方法及装置
TWI679511B (zh) * 2018-08-22 2019-12-11 和碩聯合科技股份有限公司 軌跡規劃方法與系統
CN109489658A (zh) * 2018-10-18 2019-03-19 深圳乐动机器人有限公司 一种运动目标定位方法、装置及终端设备
CN110271015A (zh) * 2019-06-25 2019-09-24 浙江大学 一种摄像机姿态可调的主动式视觉slam移动平台
CN112946660A (zh) * 2021-01-28 2021-06-11 西北工业大学 一种基于多波束前视声呐的同时定位与建图方法
CN112484740A (zh) * 2021-02-02 2021-03-12 奥特酷智能科技(南京)有限公司 用于港口无人物流车的自动建图与自动地图更新系统
CN115453599A (zh) * 2022-08-11 2022-12-09 国网江苏省电力有限公司盐城供电分公司 一种多传感器协同的管道机器人精准定位方法

Also Published As

Publication number Publication date
US11536572B2 (en) 2022-12-27
US20190323845A1 (en) 2019-10-24

Similar Documents

Publication Publication Date Title
US11536572B2 (en) Method and system for accurate long term simultaneous localization and mapping with absolute orientation sensing
Doer et al. An ekf based approach to radar inertial odometry
Alsadik et al. The simultaneous localization and mapping (SLAM): An overview
EP3715785B1 (fr) Système de navigation inertielle assisté par slam
Li et al. LIDAR/MEMS IMU integrated navigation (SLAM) method for a small UAV in indoor environments
CN113776519B (zh) 一种无光动态开放环境下agv车辆建图与自主导航避障方法
Delaune et al. Range-visual-inertial odometry: Scale observability without excitation
US20200124421A1 (en) Method and apparatus for estimating position
Deilamsalehy et al. Sensor fused three-dimensional localization using IMU, camera and LiDAR
Kinnari et al. GNSS-denied geolocalization of UAVs by visual matching of onboard camera images with orthophotos
Konovalenko et al. UAV Navigation On The Basis Of The Feature Points Detection On Underlying Surface.
Samadzadegan et al. Autonomous navigation of Unmanned Aerial Vehicles based on multi-sensor data fusion
Ghosh et al. Multi sensor data fusion for 6D pose estimation and 3D underground mine mapping using autonomous mobile robot
Hassani et al. Experimental integrity evaluation of tightly-integrated IMU/LiDAR including return-light intensity data
Hassani et al. Lidar data association risk reduction, using tight integration with INS
Xian et al. Fusing stereo camera and low-cost inertial measurement unit for autonomous navigation in a tightly-coupled approach
Daniec et al. Embedded micro inertial navigation system
Khoshelham et al. Vehicle positioning in the absence of GNSS signals: Potential of visual-inertial odometry
Yi et al. Metrics for the evaluation of localisation robustness
Gupta et al. Terrain‐based vehicle orientation estimation combining vision and inertial measurements
Wu et al. Precise pose graph localization with sparse point and lane features
Hoang et al. Localization estimation based on Extended Kalman filter using multiple sensors
Chen et al. A single frame depth visual gyroscope and its integration for robot navigation and mapping in structured indoor environments
Ruotsalainen et al. Monocular visual SLAM for tactical situational awareness
Huntsberger et al. Sensory fusion for planetary surface robotic navigation, rendezvous, and manipulation operations

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17869021

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17869021

Country of ref document: EP

Kind code of ref document: A1