JP6993638B2 - A mobile robot that imitates the walking behavior of pedestrians - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Law East Japan Passenger Railway Co., Ltd., "Station Service Robot-Is it possible to move autonomously in a pedestrian space? Realization of a robot-", July 4, 2017

The embodiments described relate to a system and a method for guiding a mobile robot in a crowded environment with pedestrians.

Robotic vehicles that operate in service environments such as hospitals and public transport stations are frequently required to operate autonomously in crowded spaces. A mobile robot that travels along a predetermined route in a crowded environment with pedestrians may significantly obstruct the flow of pedestrians and may collide with pedestrians. On the contrary, it is difficult, if not impossible, for a mobile robot to move in a crowded environment with pedestrians avoiding surrounding pedestrians to reach a desired destination position. Improper guidance of a mobile robot in a pedestrian-crowded environment causes dangerous situations such as the mobile robot operating in the immediate vicinity of a person. This is of particular concern when such mobile robots carry heavy objects or are capable of rapid acceleration.

That is, in order to improve the safety of operation in an environment crowded with pedestrians, it is desired to improve the design and control of the robot vehicle with wheels. Specifically, a solution for conflicting purposes of avoiding significant obstruction of pedestrian traffic flow and guiding to a desired destination position is desired.

This application describes a method and system for guiding a mobile robot in a pedestrian-crowded environment based on a trained navigation model. In one aspect, the trained navigation model provides measurement data that identifies nearby pedestrians, the speed of each pedestrian's mobile robot, the current position of the mobile robot in a pedestrian-crowded environment, and the desired destination position. Receive. The trained navigation model is based on this information to generate a command signal that causes the mobile robot to adjust its speed. By repeatedly sampling the speed and current position of the surrounding pedestrians, the navigation model guides the mobile robot to the destination position with minimal disruption to the pedestrian's traffic flow.

In a further aspect, the navigation model is trained to mimic pedestrian walking behavior in a pedestrian-crowded environment. Navigation models are trained on the basis of a large group of pedestrian walking movements. Each movement involves tracking the movement of a behavioral trainer traveling from a particular starting position to a particular destination through a pedestrian-crowded environment. The position and speed of the pedestrian near the trainer is measured periodically along with the position and speed of the trainer. This information is used to train the navigation model.

In a further aspect, behavioral trainers for generating teaching data for navigation model training are elected to enhance preferred pedestrian walking behavior. It is desirable to limit the behavior trainer group to behavior trainers who show favorable pedestrian walking behavior, and exclude behavior trainers who show unfavorable pedestrian walking behavior.

The above is an overview and is therefore inevitably simplified and generalized, with details omitted. Those skilled in the art will appreciate that the outline is exemplary and not limiting in any way. Other aspects of the apparatus and / or process described herein, the features and effects of the invention will be revealed by the detailed description without limitation described herein.

It is a figure which shows the embodiment of the mobile robot 100 in at least one new aspect. It is a top view of the mobile robot 100 in one embodiment. It is a schematic diagram which shows some components of a mobile robot 100. It is a figure which shows the example of the navigation model training engine 170 in one Embodiment. It is a figure which shows the example of the navigation engine 185 in one Embodiment. It is a figure which shows the environment 230 crowded with pedestrians as an example. As an example, the environment 210 crowded with pedestrians is shown, and one pedestrian is emphasized as a behavior trainer. It is the figure which showed the environment 210 crowded with the pedestrian which is the same example, and emphasized another pedestrian as a behavior trainer. It is a figure which shows the method 300 which performs the guidance of a mobile robot in a crowded environment with a pedestrian as described in this application.

The background examples and some embodiments of the present invention will be described in detail. An example of the embodiment is shown in the attached drawing.

Based on a trained navigation model, a method and system for guiding a mobile robot in a pedestrian-crowded environment will be described. A crowded environment with pedestrians includes a vigorous flow of people on the move (eg, train stations, hospitals, airports, etc.).

In one aspect, the mobile robot navigates in a pedestrian-crowded environment based on a trained navigation model. At any sampling point, the trained navigation model receives measurement data that identifies the speed of nearby pedestrians and their respective mobile robots. The trained navigation model generates a command signal that causes the mobile robot to adjust the speed based on the speed information of the pedestrian and the current position of the mobile robot in the service environment and the desired target position. The navigation model repeatedly samples the speed and current position of the surrounding pedestrians to minimize the disruption of pedestrian traffic in the service environment and guide the mobile robot to its destination position in the service environment. ..

In a further aspect, the navigation model is trained to mimic pedestrian walking behavior in a pedestrian-crowded environment. Navigation models are trained on the basis of a large group of pedestrian walking movements. Each movement tracks the movement of a behavioral trainer (eg, a specific pedestrian, a mobile robot controlled by a person) traveling from a specific starting position to a specific destination position through a crowded environment with pedestrians. include. The various groups of pedestrian gait movements include a number of different starting and destination positions within the service environment. During each pedestrian walking motion, the position and speed of the pedestrian near the trainer is measured periodically along with the position and speed of the trainer. This information is used to train the navigation model. Navigation models are trained in this way to mimic the walking behavior of a person passing through a particular pedestrian-crowded environment. As a result, mobile robots using the navigation model can follow the flow of pedestrians walking in the same or similar directions, rather than resisting pedestrians moving in the opposite or different directions. .. The trained navigation model is specific to the service environment from which the training data was obtained.

FIG. 1 shows a mobile robot 100 configured as a service robot in one embodiment. The mobile robot 100 includes a robot vehicle 101 with wheels. A drive wheel and a control wheel are attached to the frame 105 of the robot vehicle 101 with wheels. Further, the mobile robot 100 includes a loading platform 102 configured for carrying the luggage 104. Further, the mobile robot 100 includes an upper body robot 103 configured to fix the luggage 104 to the loading platform 102 and interact with the user of the mobile robot 100. As an example, the mobile robot 100 operates at a public transportation station (for example, a railway station) to assist passengers in carrying luggage on the premises. As another example, the mobile robot 100 operates in a public area to help the general public dispose of garbage. In this example, the luggage 104 includes a garbage container.

FIG. 2 shows a top view of a robot vehicle 101 with wheels and a loading platform 102 of the mobile robot 100. As shown in FIG. 2, the wheeled robot vehicle 101 includes drive wheels 106A and 106B and control wheels 106C and 106D. In some embodiments, the control wheels 106C, 106D are passive wheels that rotate freely around a plurality of axes. In these embodiments, the control wheels 106C, 106D primarily function to support the load perpendicular to the ground. On the other hand, the rotation of the drive wheels 106A and 106B determines the moving trajectory of the robot vehicle 101 with wheels. In some other embodiments, the orientation of the control wheels 106C, 106D about an axis perpendicular to the ground is actively controlled. Further, in these embodiments, the control wheels 106C and 106D function to control the direction of the moving trajectory of the wheeled robot vehicle 101. In some other embodiments, both the rotation of the control wheels 106C, 106D and the orientation of the control wheels 106C, 106D about an axis perpendicular to the ground are actively controlled. In these embodiments, the control wheels 106C, 106D function to control both the direction of the moving track of the wheeled robot vehicle 101 and the speed along the moving track.

As shown in FIG. 1, the mobile robot 100 includes an image capturing system 107, a distance measurement sensor system 108, and a lidar sensor system 109. The image capturing system 107 includes one or more image capturing devices (for example, a camera or the like) for photographing a person or an object in the vicinity of the mobile robot 100. The distance measurement sensor system 108 includes the mobile robot 100 and one or more distance measurement sensors (for example, a laser-based, sonar, radar-based distance measurement sensor) that estimate the distance between the mobile robot 100 and a person or an object in the vicinity of the mobile robot 100. The rider sensor system 109 includes a mobile robot 100 and one or more rider devices that estimate the distance between a person or an object in the vicinity of the mobile robot 100. Presence of pedestrians in the vicinity of the mobile robot 100 using image data collected by the imaging system 107, distance data collected by the distance measurement sensor system 108, distance data collected by the rider sensor system 109, or any combination. And the speed of the pedestrian with respect to the mobile robot 100 is estimated.

The imaging system, distance measurement sensor system and rider sensor system are configured with an appropriate field of view. In some embodiments, the imaging system, the distance measurement sensor system and the rider sensor system are configured to detect a person or object in a 360 degree field of view centered on the mobile robot 100. On the other hand, in some other embodiments, the field of view of the imaging system, the distance measurement sensor system and the rider sensor system is limited to a specific angle range or group of angles around the mobile robot 100. The fields of view of the imaging system, distance measurement sensor system and rider sensor system are configured to be completely overlapped, partially overlapped, or not overlapped at all. In a preferred embodiment, the fields of view of at least two sensor subsystems overlap, resulting in duplication of collected sensor data. In some examples, collecting duplicate data improves the identification and estimation of the speed of pedestrians near the mobile robot 100.

The mobile robot 100 shown in FIG. 1 includes an image capturing system, a distance measurement sensor system, and a rider sensor system, and the mobile robot described in the present application may include any combination of these systems. Similarly, a mobile robot as described herein may have any sensor arrangement as long as it is suitable for identifying and estimating the speed of a pedestrian near the mobile robot 100.

In some embodiments, the mobile robot 100 comprises an electronic device (not shown) suitable for position detection of the mobile robot 100 in a service environment (eg, a railway station) in which the mobile robot 100 is housed and operated. In one embodiment, the mobile robot 100 comprises a radio frequency (RF) beacon that communicates with a similar beacon fixedly placed in the service environment as part of an indoor positioning system. Based on the communication between these beacons, the mobile robot 100 can detect its position in the service environment.

In some other embodiments, the image data collected by the imaging system 107, the distance data collected by the distance measurement sensor system 108, the distance data collected by the rider sensor system 109, or any combination is used. Estimate the current position of the mobile robot in the service environment. As an example, the image data collected by the imaging system 107 includes reference images that are fixedly attached throughout the service environment (eg, railway stations, hospitals, etc.). By performing an image processing operation on the reference image, the computing system 200 determines the position and orientation (for example, x position, y position, Rz direction) of the mobile robot related to the service environment. As an example, standards and image processing software are available from the April Robotics Laboratory at the University of Michigan, Ann Arbor, Michigan, United States. As another example, the computing system 200 assembles an image from the distance data collected by the rider sensor system 109. Further, the image map of the service environment is stored in the memory-equipped mobile robot 100. The computing system 200 uses SLAM (Simultaneus Localization And Mapping) technology to compare the assembled image with the image map of the service environment to estimate the position of the mobile robot 100 in the service environment.

FIG. 3 is a diagram showing components of a mobile robot 100 including a computing system 200, an imaging system 107, a distance measurement sensor system 108, a rider sensor system 109, and a vehicle actuator 162. In the embodiment shown in FIG. 3, the computing system 200 is communicably connected to the imaging system 107, the distance measurement sensor system 108, the rider sensor system 109, and the vehicle actuator 162 by a wired communication link. However, the computing system 200 may be communicably connected to the sensors and devices described in the present application by any of a wired communication link and a wireless communication link. Normally, these sensors are computing systems, regardless of the number of sensors that detect the position of the mobile robot 100 in the service environment and specify the speed of nearby pedestrians and each of the pedestrians with respect to the mobile robot 100. It is connected to the 200 so that it can communicate with it.

As shown in FIG. 3, the computing system 200 includes a sensor interface 110, at least one processor 120, a memory 130, a bus 140, a wireless communication transceiver 150, and a controlled device interface 160. The sensor interface 110, the processor 120, the memory 130, the wireless communication transceiver 150, and the controlled device interface 160 are configured to be communicable via the bus 140.

As shown in FIG. 3, the sensor interface 110 includes a digital input / output interface 112. In some other embodiments, the sensor interface 110 comprises an analog-to-digital conversion (ADC) electronic device (not shown), a wireless communication transceiver (not shown) or a combination thereof and communicates with a sensor system. The measurement data is received from the above sensors. In general, the sensors described herein are digital or analog sensors that are communicably connected to the computing system 200 by appropriate interfaces.

As shown in FIG. 3, the digital interface 112 is configured to receive a signal 113 from the imaging system 107, a signal 114 from the distance measurement sensor system 108, and a signal 115 from the rider sensor system 109. In this example, the imaging system 107 includes a built-in electronic device that produces a digital signal 113 indicating a detected image. Similarly, the distance measurement sensor system 108 comprises an internal electronic device that produces a digital signal 114 indicating a distance measurement value obtained by the distance measurement system 108, and the rider sensor system 109 is obtained by the rider sensor system 109. It comprises a built-in electronic device that generates a digital signal 115 indicating a distance measurement.

The controlled device interface 160 comprises a suitable digital-to-analog conversion (DAC) electronic device. Also, in some embodiments, the controlled device interface 160 comprises a digital input / output interface. In some other embodiments, the controlled device interface 160 comprises a radio communication transceiver configured to communicate with the device. The communication includes transmission of a control signal.

As shown in FIG. 3, the controlled device interface 160 is configured to transmit the control command 161 to the vehicle actuator 162. The vehicle actuator 162 moves the wheeled robot vehicle 101 at a desired speed (ie, both speed and direction of travel) in a pedestrian-crowded environment.

The memory 130 has a memory amount 131 for storing the measurement data collected from the sensors 107 to 109. Further, the memory 130 has a memory amount 132 for storing the program code. The processor 120 executes the navigation model training function and the model-based navigation function described in the present application by executing the program code.

In some examples, the processor 120 is configured to store the digital signal generated by the sensor interface 110 in the memory 131. Further, the processor 120 is configured to read a digital signal stored in the memory 131 and transmit it to the wireless communication transceiver 150. The wireless communication transceiver 150 is configured to communicate a digital signal from the computing system 200 to an external computing device (not shown) via a wireless communication link. As shown in FIG. 3, the radio communication transceiver transmits the radio signal 152 via the antenna 151. The wireless signal 152 includes digital information indicating a digital signal transmitted from the computing system 200 to an external computing device. As an example, the image collected by the imaging device 123 and the distance measured by the distance measuring sensor system 108 and the rider sensor system 109 are communicated to an external computing system to monitor the activity of the mobile robot. ..

As another example, the wireless communication transceiver 150 is configured to communicate a digital signal from an external computing device (not shown) to the computing system 200 via a wireless communication link. As shown in FIG. 3, the radio communication transceiver receives the radio signal 153 via the antenna 151. As an example, the radio signal 153 includes digital information indicating the position of the mobile robot 100 in the service environment.

In one aspect, the computing system is configured as a navigation model training engine to train the navigation model. The navigation model is used to guide the mobile robot 100 in a pedestrian-crowded environment by mimicking the walking behavior of a pedestrian. In some examples, the computing system 200 is configured as a navigation model training engine. In some other examples, an external computing device is configured as a navigation model training engine. FIG. 4 is a diagram showing an example of the navigation model training engine 170 in one embodiment. As shown in FIG. 4, the navigation model training engine 170 includes a mobile sensor-based crowd analysis module 171, a fixed sensor-based crowd analysis module 172, and a navigation model training module 174.

The movement sensor-based crowd analysis module 171 receives a signal 175 generated by a sensor attached to a behavioral trainer who skillfully advances from one position to another through a pedestrian-crowded environment. The signal 175 is periodically collected while the behavior trainer is traveling in a crowded environment with pedestrians. As an example, the behavior trainer is a mobile robot 100 guided by a person in a pedestrian-crowded environment. In this example, the signal 175 is collected by the distance measurement sensor system 108, the image data collected by the imaging system 107 while the mobile robot 100 is traveling in a pedestrian-crowded environment under human control. The distance data and the distance data collected by the rider sensor system 109 are included. As another example, a behavioral trainer is a particular pedestrian who travels from one position to another in a pedestrian-crowded environment. In this example, one or more sensors (eg, imaging system 107, distance sensor system 108, rider sensor system 109, etc.) are attached to the pedestrian. The signal 175 is periodically collected while the pedestrian is traveling in a pedestrian-crowded environment.

The movement sensor-based crowd analysis module 171 processes the signal 175 to determine the relative velocity 177 (ie, velocity and direction of travel) of each pedestrian in the vicinity of the behavior trainer at each sampling time point with respect to the behavior trainer. In some examples, the movement sensor-based crowd analysis module 171 generates time-series images of pedestrians near the behavior trainer based on image data, distance measurement data, rider data, or a combination thereof. The speed of nearby pedestrians with respect to the behavior trainer is determined from the image sequence data at each sampling time point.

FIG. 6 is a diagram showing an environment 230 crowded with pedestrians as an example. The pedestrian-crowded environment 230 includes many fixed structures 239 (eg, building exterior walls, rotary gates, etc.) and many pedestrians traveling through the environment. The behavior trainer 237 in the environment 230 moves at the specific speed indicated by the vector 238. Similarly, the nearby pedestrians 232 to 236 each move at the speed indicated by the illustrated vector. In this example, all nearby pedestrians 232 to 236 are identified as pedestrians within distance R from the behavior trainer 237.

As an example, the mobile sensor-based crowd analysis module 171 processes the signal 175 and the speed 177 (ie, speed and direction of travel) at each additional sampling of pedestrians 232-236 to the behavior trainer 237 in the example shown in FIG. To judge.

Pedestrians in the vicinity are identified in any suitable way for training and use of the navigation model. As an example, nearby pedestrians are identified as the closest N pedestrians. Here, N is an arbitrary positive integer. As another example, nearby pedestrians are identified as behavior trainers or all pedestrians in the field of view of an autonomous mobile robot.

The fixed sensor-based crowd analysis module 172 receives a signal 176 generated by a sensor mounted on a fixed structure in a pedestrian-crowded environment. For example, FIGS. 7 and 8 are diagrams showing, as an example, a pedestrian-crowded environment 210. The pedestrian-crowded environment 210 includes many pedestrians passing through between the fixed structures 239. In this example, one or more cameras 240 are attached to the structure 239. The camera 240 produces image data 176 received by the fixed sensor based community analysis module 172. The signal 176 is periodically collected while the behavior trainer is traveling in a crowded environment with pedestrians.

In this example, the behavioral trainer may be any pedestrian in a pedestrian-crowded environment 210. As an example, signal 176 includes time-series images of a pedestrian-crowded environment 210. The fixed sensor-based crowd analysis module 172 identifies one or more behavioral trainers from among the pedestrians whose movements are captured in the image data of the signal 176, based on the signal 176. Also, the fixed sensor-based crowd analysis module 172 processes the signal 176 to determine the speed 178 (ie, speed and direction of travel) of each pedestrian in the vicinity of each behavior trainer at each sampling time point for each behavior trainer.

As an example, as shown in FIG. 7, the fixed sensor-based community analysis module 172 identifies the behavioral trainer 218 illustrated within the environment 210. The behavior trainer 218 moves at the specific speed indicated by the illustrated vector. Similarly, the nearby pedestrians 212 to 217 each move at the speed indicated by the illustrated vector. In this example, the fixed sensor-based crowd analysis module 172 identifies any nearby pedestrians 212-217 as pedestrians within distance R from the behavior trainer 218. The fixed sensor-based crowd analysis module 172 also determines the speeds of nearby pedestrians 212-217 with respect to the behavior trainer 218 based on the image sequence data at each sampling time point.

As another example, as shown in FIG. 8, the fixed sensor based community analysis module 172 identifies another behavioral trainer 226 illustrated within the environment 210. The behavior trainer 226 moves at the specific speed indicated by the illustrated vector. Similarly, the nearby pedestrians 222-225 each move at the speed indicated by the illustrated vector. In this example, the fixed sensor-based crowd analysis module 172 identifies any nearby pedestrians 222-225 as pedestrians within distance R from the behavior trainer 226. The fixed sensor-based crowd analysis module 172 also determines the speeds of nearby pedestrians 222-225 with respect to the behavior trainer 226 based on the image sequence data at each sampling time point.

Data suitable for estimating the speed of behavior trainers and nearby pedestrians with a fixed sensor-based crowd analysis module 172 by attaching one or more different types of sensors to a fixed structure in a pedestrian-crowded environment. May be obtained. As the sensor, for example, a camera, a distance measuring device, a rider device, or the like is used.

As shown in FIG. 4, the moving sensor-based crowd analysis module 171 generates velocity information 177 and the fixed sensor-based crowd analysis module 172 generates velocity information 178. These speed information is the basis for navigation model training. Alternatively, either the mobile sensor-based crowd analysis module 171 or the fixed sensor-based crowd analysis module 172 may be dedicated to the generation of speed information for navigation model training.

As shown in FIG. 4, the navigation model training module 174 trains the navigation model 184. The navigation model 184 is a method of mimicking a pedestrian walking behavior exhibited by one or more behavior trainers, allowing a mobile robot to travel from one position to another through a pedestrian-crowded environment. .. The trained navigation model 184 is based on the current position and desired destination position of the mobile robot and the desired speed of the mobile robot in each sampling period (i.e.,) based on the relative speed of one or more nearby pedestrians to the mobile robot. Judge speed and maneuvering angle).

The navigation model training module 174 receives the training data group. Each training data set pertains to a behavioral trainer's pedestrian-passing motion from a starting position to a desired destination. Each training data group has a current position 181 and a target position 182 corresponding to the movement, a measured speed 180 of the behavior trainer, and relative speed information 177, 178 related to nearby pedestrians for each sampling period of the whole movement. including. The navigation model training module 174 trains the navigation model 184 to determine the desired speed of the mobile robot based on the current position and desired target position of the mobile robot and the relative speed of nearby pedestrians. The trained navigation model 184 is stored in memory (eg, memory 133 shown in FIG. 3).

In some examples, the behavioral trainer's speed 180 is measured on the behavioral trainer. For example, in an embodiment in which a person to which a mobile robot or device is attached is used as a behavior trainer, the speed of the behavior trainer is estimated using a sensor mounted on the person to which the mobile robot or device is attached. As an example, a dead reckoning sensor mounted on a mobile robot is used to constantly estimate the speed of the mobile robot. As another example, an indoor positioning system is used to constantly estimate the speed of a person with a mobile robot or device attached.

In some other examples, the speed of the behavioral trainer is measured based on sensors installed in the environment. For example, as described with reference to FIGS. 7-8, in an embodiment in which a behavior trainer is elected from one or more participants participating in the pedestrian traffic flow, the speed information 178 is the behavior in each sampling period. Includes trainee speed.

In general, the trained navigation model 184 is a direct and functional relationship between measurements of the relative speed of the pedestrian at the current position and nearby and the speed command that causes the mobile robot to mimic the walking behavior of the pedestrian. ..

The trained navigation models described herein are implemented as neural network models, convolutional network models, support vector machine models, response phase models, polynomial models, random forest models, deep network models or other types of models. In some examples, the trained navigation model described herein is implemented as a combination of multiple models.

In a further aspect, behavioral trainers for generating teaching data for navigation model training are elected to enhance preferred pedestrian walking behavior. In the example described with reference to FIGS. 7 to 8, any of the recorded pedestrians may be used as the behavior trainer, but it is desirable that the behavior trainer group is limited to the behavior trainers who show preferable pedestrian walking behavior. For example, a pedestrian who quickly moves from one platform to another at a station, from a platform to an exit, from a storage locker to a platform, or in the opposite direction of these is preferred as a behavioral trainer. On the other hand, pedestrians who stop in the middle of the flow of people, move against the flow of people, move to the corners and stop, or move randomly with respect to the continuous flow of people are behavior trainers. Not preferable. In this way, pedestrians showing favorable pedestrian walking behavior are included in the training data group, and pedestrians showing unfavorable pedestrian walking behavior are excluded from the training data group.

In some examples, the fixed sensor-based crowd analysis module 172 is configured to filter the signal 176 to select behavioral trainers who exhibit preferred pedestrian gait behavior. As an example, a fixed sensor-based crowd analysis module 172 determines the difference in direction of movement between a candidate behavior trainer and a nearby pedestrian. If the difference is less than a predetermined threshold, the behavioral trainer candidate is selected and added to the training data group. On the other hand, when the difference exceeds a predetermined threshold value, the behavior trainer candidate is excluded from the training data group. In these examples, a predetermined threshold value is set to isolate a behavioral trainer who exhibits behavior that follows the flow (ie, moves in the same direction as nearby pedestrians).

In another aspect, the computing system 200 was configured as a navigation engine and trained the desired speed of the mobile robot based on the desired target position and the relative speed measurements of the current position and nearby pedestrians. Judgment is made directly using the navigation model. FIG. 5 is a diagram showing an example of the navigation engine 185 in one embodiment. As shown in FIG. 5, the navigation engine 185 includes a movement sensor-based crowd analysis module 171 and a navigation module 190 described with reference to FIG.

In one embodiment, the mobile robot 100 is tasked with traveling from a current position to a predetermined destination position through a pedestrian-crowded environment. The mobile sensor-based crowd analysis module 171 receives a signal 186 (eg, signals 113-115 or any combination thereof) from a sensor mounted on the mobile robot 100. The movement sensor-based crowd analysis module 171 identifies nearby pedestrians, determines the relative speed of each identified pedestrian with respect to the mobile robot 100, and communicates this speed information 189 to the navigation module 190. Further, the navigation module 190 receives a reading 187 indicating the current position of the mobile robot in the service environment and a desired destination position 188 in the service environment. The navigation module 190 uses the trained navigation model 184 to determine the desired speed 191 of the mobile robot during the current sampling period based on the current position 187, the desired target position 188, and the speed information 189. The desired speed 191 is communicated to a motion controller 192 (eg, implemented in the computing system 200) that produces a command signal 161. The command signal 161 is communicated with the vehicle actuator 162 of the mobile robot 100 to move the mobile robot 100 at a desired speed 191. The velocity information 189 from the mobile sensor-based crowd analysis module 171 and the current location information (eg from an indoor positioning system, etc.) are repeatedly sampled and the desired velocity is repeatedly updated. In this way, the navigation module 190 guides the mobile robot 100 in a pedestrian-crowded environment with minimal disruption of traffic flow.

FIG. 9 shows a flowchart of method 300 suitable for implementation by a mobile robot as described in the present application. In some embodiments, the mobile robot 100 is operable according to the method 300 shown in FIG. However, the implementation of the method 300 is not limited to the embodiment of the mobile robot 100 described with reference to FIGS. 1 to 5. These figures and the corresponding explanations are merely examples, and many other embodiments and operation examples are also conceivable.

In the block 301, a first signal group indicating the presence of one or more pedestrians in the vicinity of the mobile robot is generated by, for example, a sensor mounted on the mobile robot 100 or a sensor installed in an environment crowded with pedestrians. .. In addition, a second signal group indicating the position of the mobile robot in a pedestrian-crowded environment is generated by, for example, an indoor positioning system.

In block 302, based on the first signal group, the speed of each one or more pedestrians with respect to the mobile robot is estimated at the sampling time.

In block 303, the position of the mobile robot in the pedestrian-crowded environment at the sampling time, the speed of each of one or more pedestrians near the mobile robot at the sampling time, and the desired mobile robot in the pedestrian-crowded environment. Based on the target position of, the value of the desired speed of the mobile robot at the sampling time is estimated. The desired speed value is determined based on a trained navigation model.

At block 304, the mobile robot is moved at a desired speed value in a pedestrian-crowded environment.

The computing system 200 and the external computing system are not particularly limited, and include a personal computer system, a mainframe computer system, a workstation, an image computer, a parallel processor, or other devices known in the art. Generally, a "computing system" is broadly defined to include a device having one or more processors that execute instructions stored in a storage medium.

The program instruction 132 that performs the method as described herein may be transmitted via a transmission medium such as a wired or wireless transmission link. For example, as shown in FIG. 3 , the program instruction 132 stored in the memory 130 is transmitted to the processor 120 via the bus 140. The program instruction 132 is stored in a computer-readable medium (eg, memory 130). Examples of computer-readable media include ROMs, RAMs, magnetic disks, optical disks, magnetic tapes, and the like.

In one or more preferred embodiments, the functionality is implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functionality is stored or transmitted to a computer-readable medium as one or more instructions or codes. Computer-readable media include computer storage media and communication media. Communication media include any medium that can easily transfer a computer program from one location to another. The storage medium may be any medium available, either general purpose or accessible by a dedicated computer. Such computer-readable media are not particularly limited, and are, for example, RAM, ROM, CD-ROM, other optical disk storage, magnetic disk storage, other magnetic disk storage devices, instructions or data structures of desired program code means. Examples include other media that can be used for transmission or storage in the form and accessible by a general purpose or dedicated computer or a general purpose or dedicated processor. Also, any connection deserves to be called a computer-readable medium. For example, software is transmitted from websites, servers or other remote sources using coaxial cables, fiber optic cables, twist pairs, digital subscriber lines (DSL), or wireless technologies such as infrared, wireless communications, and microwaves. If the coaxial cable, fiber optic cable, twist pair, DSL, or wireless technology such as infrared, wireless communication, microwave, etc. are included in the definition of the medium. Examples of the "disc" used in the present invention include compact discs (CDs), laser discs, optical discs, digital multipurpose discs (DVDs), floppy discs, and Blu-ray discs. The disk described as "disk" in English usually reproduces data by magnetic action, and the disk described as "disc" optically reproduces data using a laser. The above combinations are also included in the range of computer readable media.

Although some specific embodiments have been described above for the purpose of teaching, the teachings of the present patent document have general applicability and are not limited to the above-mentioned specific embodiments. Therefore, the various features of the embodiments can be variously modified, adapted and combined without departing from the scope of the invention as described in the claims.

Claims (20)

The mobile robot equipped with one or more actuators that move the mobile robot at a desired speed, and
A first signal group provided on the mobile robot and indicating the presence of one or more pedestrians in the vicinity of the mobile robot, and a second signal group indicating the position of the mobile robot in an environment crowded with pedestrians. , With more than one sensor system to produce,
A mobile sensor-based crowd analysis module configured to estimate the speed of each of the one or more pedestrians with respect to the mobile robot at sampling time based on the first signal group.
The position of the mobile robot in the pedestrian-crowded environment at the sampling time, the speed of each of the one or more pedestrians with respect to the mobile robot at the sampling time, and the pedestrian-crowded environment. A navigation module configured to estimate the desired speed value at the sampling time based on the desired destination position of the mobile robot.
Equipped with
When the current position of the mobile robot, the speed of each of the one or more pedestrians with respect to the mobile robot, and the desired target position of the mobile robot are input, the navigation module has the desired speed of the mobile robot. A system characterized in that a value of the desired speed is determined using a navigation model that outputs a value. Further equipped with a navigation model training module configured to train the navigation model based on a group of pedestrian walking movements.
Each pedestrian gait movement involves tracking the movement of a behavioral trainer traveling from a particular starting position to a particular destination position through a pedestrian-crowded environment.
The first aspect of claim 1, wherein the positions and speeds of a plurality of pedestrians in the vicinity of the behavior trainer and the positions and speeds of the behavior trainers are measured periodically during each pedestrian walking motion. system. The system according to claim 2, wherein the behavior trainer is a mobile robot guided by a person in an environment crowded with pedestrians. The system according to claim 2, wherein the behavior trainer is a person who travels in a crowded environment with the pedestrian. The position and speed of the plurality of pedestrians in the vicinity of the behavior trainer and the position and speed of the behavior trainer shall be periodically measured by the one or more sensor systems provided on the mobile robot. 2. The system according to claim 2. Further equipped with one or more sensor systems fixed in one or more positions in the pedestrian-crowded environment.
The position and speed of the plurality of pedestrians in the vicinity of the behavior trainer and the position and speed of the behavior trainer are periodically measured by the one or more sensor systems fixed to the one or more positions. The system according to claim 2, wherein the system is characterized by the above. In the group of pedestrian walking movements, the difference between the behavior trainer related to the specific pedestrian walking movement and each pedestrian in the vicinity of the behavior trainer in the direction during the specific pedestrian walking movement is a predetermined threshold value. The system according to claim 6, wherein if less than, it is selected from a group of larger pedestrian gait movements. The system according to claim 1, wherein the one or more sensor systems include any one of an imaging system, a distance measurement sensor system, and a rider sensor system. The mobile robot equipped with one or more actuators that move the mobile robot at a desired speed, and
A first signal group provided on the mobile robot and indicating the presence of one or more pedestrians in the vicinity of the mobile robot, and a second signal group indicating the position of the mobile robot in an environment crowded with pedestrians. , With more than one sensor system to produce,
With non-temporary computer readable media, with instructions,
When the computing system executes the above instructions, the computing system becomes
Based on the first signal group, the speed of each of the one or more pedestrians with respect to the mobile robot is estimated at the sampling time.
The position of the mobile robot in the pedestrian-crowded environment at the sampling time, the speed of each of the one or more pedestrians with respect to the mobile robot at the sampling time, and the pedestrian-crowded environment. Based on the desired target position of the mobile robot, the value of the desired speed at the sampling time is estimated.
The value of the desired speed is the current position of the mobile robot, the speed of each of the one or more pedestrians with respect to the mobile robot, and the desired target position of the mobile robot. A system characterized in that a determination is made based on a navigation model that outputs a desired speed value . The non-temporary computer readable medium comprises further instructions.
When the computing system executes the instructions, the computing system trains the navigation model based on a group of pedestrian walking movements.
Each pedestrian gait movement involves tracking the movement of a behavioral trainer traveling from a particular starting position to a particular destination position through a pedestrian-crowded environment.
The ninth aspect of claim 9, wherein the positions and speeds of a plurality of pedestrians in the vicinity of the behavior trainer and the positions and speeds of the behavior trainers are measured periodically during each pedestrian walking motion. system. The system according to claim 10, wherein the behavior trainer is a mobile robot guided by a person in an environment crowded with pedestrians. The system according to claim 10, wherein the behavior trainer is a person who travels in a crowded environment with the pedestrian. The position and speed of the plurality of pedestrians in the vicinity of the behavior trainer and the position and speed of the behavior trainer shall be periodically measured by the one or more sensor systems provided on the mobile robot. 10. The system according to claim 10. Further equipped with one or more sensor systems fixed in one or more positions in the pedestrian-crowded environment.
The position and speed of the plurality of pedestrians in the vicinity of the behavior trainer and the position and speed of the behavior trainer are periodically measured by the one or more sensor systems fixed to the one or more positions. The system according to claim 10, wherein the system is characterized by the above. In the group of pedestrian walking movements, the difference between the behavior trainer related to the specific pedestrian walking movement and each pedestrian in the vicinity of the behavior trainer in the direction during the specific pedestrian walking movement is a predetermined threshold value. 14. The system of claim 14, wherein if less than, they are selected from a group of larger pedestrian gait movements. The system according to claim 9, wherein the one or more sensor systems include any one of an imaging system, a distance measurement sensor system, and a rider sensor system. A step of generating a first signal group indicating the presence of one or more pedestrians in the vicinity of the mobile robot and a second signal group indicating the position of the mobile robot in an environment crowded with pedestrians.
A step of estimating the speed of each of the one or more pedestrians with respect to the mobile robot at the sampling time based on the first signal group.
The position of the mobile robot in the pedestrian-crowded environment at the sampling time, the speed of each of the one or more pedestrians with respect to the mobile robot at the sampling time, and the pedestrian-crowded environment. A step of estimating a desired speed value at the sampling time based on a desired target position of the mobile robot, and a step of estimating the desired speed value.
Equipped with
When the current position of the mobile robot, the speed of each of the one or more pedestrians with respect to the mobile robot, and the desired target position of the mobile robot are input, the value of the desired speed is the mobile robot. Determined based on a navigation model that outputs the desired speed value ,
In addition,
A method comprising a step of moving the mobile robot in the environment crowded with the pedestrian at the value of the desired speed. Further provided with a step of training the navigation model based on a group of pedestrian walking movements,
Each pedestrian gait movement involves tracking the movement of a behavioral trainer traveling from a particular starting position to a particular destination position through a pedestrian-crowded environment.
The 17th aspect of claim 17, wherein the positions and speeds of a plurality of pedestrians in the vicinity of the behavior trainer and the positions and speeds of the behavior trainers are measured periodically during each pedestrian walking motion. Method. The method according to claim 18, wherein the behavior trainer is a mobile robot guided by a person in an environment crowded with pedestrians. The method according to claim 18, wherein the behavior trainee is a person who travels in a crowded environment with the pedestrian.

Images