[go: up one dir, main page]

WO2019173321A1 - Classification de mode de transport de robot - Google Patents

Classification de mode de transport de robot Download PDF

Info

Publication number
WO2019173321A1
WO2019173321A1 PCT/US2019/020724 US2019020724W WO2019173321A1 WO 2019173321 A1 WO2019173321 A1 WO 2019173321A1 US 2019020724 W US2019020724 W US 2019020724W WO 2019173321 A1 WO2019173321 A1 WO 2019173321A1
Authority
WO
WIPO (PCT)
Prior art keywords
robot
transportation mode
behavior
emotion state
predicted
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2019/020724
Other languages
English (en)
Inventor
Ross Peter ANDERSON
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Anki Inc
Original Assignee
Anki Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Anki Inc filed Critical Anki Inc
Publication of WO2019173321A1 publication Critical patent/WO2019173321A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J11/00—Manipulators not otherwise provided for
    • B25J11/0005—Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means
    • B25J11/001—Manipulators having means for high-level communication with users, e.g. speech generator, face recognition means with emotions simulating means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B25J13/085—Force or torque sensors
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J13/00—Controls for manipulators
    • B25J13/08—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices
    • B25J13/088—Controls for manipulators by means of sensing devices, e.g. viewing or touching devices with position, velocity or acceleration sensors
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05B—CONTROL OR REGULATING SYSTEMS IN GENERAL; FUNCTIONAL ELEMENTS OF SUCH SYSTEMS; MONITORING OR TESTING ARRANGEMENTS FOR SUCH SYSTEMS OR ELEMENTS
    • G05B2219/00—Program-control systems
    • G05B2219/30—Nc systems
    • G05B2219/39—Robotics, robotics to robotics hand
    • G05B2219/39254—Behaviour controller, robot have feelings, learns behaviour

Definitions

  • This specification relates to robots, and more particularly to robots used for consumer purposes.
  • Consumer robots have been developed that can express a variety of human-like emotions.
  • One such robot can express a chosen emotion by raising and lowering an actuated lift, moving and transforming the shape of its eyes on an OLED screen, orienting or translating its pose using continuous tracks (tank treads), and projecting noises or a voice through a speaker.
  • the robot is programmed with a set of“animations,” each of which controls these components in parallel to execute a sequence of actions.
  • the robot can also plan and execute its own actions, such as driving to a new location.
  • the choice of action or animation is informed by the current robot behavior. For instance, if the robot behavior is to laugh, the robot might play one of several laughing animations.
  • the choice of behavior is dictated by a combination of an internal state, external stimuli, and a behavior selection algorithm. In this robot and in others, this process of fusing sensor data with an internal state to choose a behavior does not fully take into account the context of the use of the robot. As a result, the emotions conveyed by the robot through animations are the same no matter how the robot is being transported.
  • Some existing robots including robots designed to express emotion, run processes that leverage the robot’s internal state, external stimuli, and a selection mechanism to choose the robot behavior or actions.
  • the mechanism is used to drive an emotion state mechanism and select actions for the robot to execute.
  • the mechanism is driven by external and internal sensors, such as a touch sensor, CCD camera, and microphone.
  • The“conditions” identified by this existing technology are limited to a those that immediately identifiable by the robot sensors, such as petting, the existence of objects in the field of view of the camera, or verbal commands detected with the microphone.
  • This specification describes how a robot can select actions, animations, speech, and/or behaviors to perform based on a transportation mode.
  • the transportation mode, other sensor data, the internal state of the robot, and the selection algorithm in concert choose the behavior of the robot and what actions or animations to perform.
  • its behavior selection engine may trigger a sequence of animations that convey“glee.”
  • Previous autonomous emotion robots do not take into account the transportation mode when selecting animations, actions, or behaviors for the robot to execute.
  • the sensor input in prior systems is used only to detect the robot’s state of interaction, e.g., whether the robot is being touched.
  • the techniques described below broaden the use of the sensors to allow for the inclusion of the general context of the robot’s use in the selection of actions, animations, and behaviors. Rather than playing an animation if the robot is touched, for instance, it can play an animation if the robot is touched while riding in a car
  • FIG. 1 A illustrates an example robot.
  • FIG. 1B is a block diagram illustrating an input and output system of an example robot.
  • FIG. 2 is a block diagram showing an example software architecture of a robot.
  • FIG. 3 is a flowchart of an example process for training a transportation mode classifier for a robot.
  • FIG. 4 is a flowchart of an example process for generating a behavior for a robot based on a predicted transportation mode.
  • FIGS. 5A-5C illustrate different robot behaviors generated as a result of the robot detecting different transportation modes.
  • This specification describes an example robotic platform having a variety of sensors and a controller that decides and directs how to execute actions using actuators, displays, and/or speakers or other components.
  • FIG. 1 A illustrates an example robot 100A.
  • the robot 100A is an example of a mobile autonomous robotic system on which the transportation mode classification techniques described in this specification can be implemented.
  • the robot 100A can use the techniques described below for use as a toy or as a personal companion.
  • robots that having components that are suitable for classifying transportation modes are described in more detail in commonly-owned ET.S. Patent Application No. 15/694,710, which is herein incorporated by reference.
  • the robot 100 A generally includes a body 105 and a number of physically moveable components.
  • the components of the robot 100A can house data processing hardware and control hardware of the robot.
  • the physically moveable components of the robot 100 A include a propulsion system 110, a lift 120, and a head 130.
  • the robot 100A also includes integrated output and input subsystems.
  • the output subsystems can include control subsystems that cause physical movements of robotic components; presentation subsystems that present visual or audio information, e.g., screen displays, lights, and speakers; and communication subsystems that communicate information across one or more communications networks, to name just a few examples.
  • the control subsystems of the robot 100A include a locomotion subsystem 110.
  • the locomotion system 110 has wheels and treads. Each wheel subsystem can be independently operated, which allows the robot to spin and perform smooth arcing maneuvers.
  • the locomotion subsystem includes sensors that provide feedback representing how quickly one or more of the wheels are turning. The robot can use this information to control its position and speed.
  • the control subsystems of the robot 100A include an effector subsystem 120 that is operable to manipulate objects in the robot’s environment.
  • the effector subsystem 120 includes a lift and one or more motors for controlling the lift.
  • the effector subsystem 120 can be used to lift and manipulate objects in the robot’s environment.
  • the effector subsystem 120 can also be used as an input subsystem, which is described in more detail below.
  • the control subsystems of the robot 100A also include a robot head 130, which has the ability to tilt up and down and optionally side to side. On the robot 100A, the tilt of the head 130 also directly affects the angle of a camera 150.
  • the presentation subsystems of the robot 100A include one or more electronic displays, e.g., electronic display 140, which can each be a color or a monochrome display or both.
  • the electronic display 140 can be used to display any appropriate information.
  • the electronic display 140 is presenting a simulated pair of eyes.
  • the presentation subsystems of the robot 100A also include one or more lights 142 that can each turn on and off, optionally in multiple different colors.
  • the presentation subsystems of the robot 100A can also include one or more speakers, which can play one or more sounds in sequence or concurrently so that the sounds are at least partially overlapping.
  • the input subsystems of the robot 100A include one or more perception subsystems, one or more audio subsystems, one or more touch detection subsystems, one or more motion detection subsystems, one or more effector input subsystems, and one or more accessory input subsystems, to name just a few examples.
  • the perception subsystems of the robot 100A are configured to sense light from an environment of the robot.
  • the perception subsystems can include a visible spectrum camera, an infrared camera, or a distance sensor, to name just a few examples.
  • the robot 100A includes an integrated camera 150.
  • the perception subsystems of the robot 100A can include one or more distance sensors. Each distance sensor generates an estimated distance to the nearest object in front of the sensor.
  • the perception subsystems of the robot 100A can include one or more light sensors.
  • the light sensors are simpler electronically than cameras and generate a signal when a sufficient amount of light is detected.
  • light sensors can be combined with light sources to implement integrated cliff detectors on the bottom of the robot. When light generated by a light source is no longer reflected back into the light sensor, the robot 100A can interpret this state as being over the edge of a table or another surface.
  • the audio subsystems of the robot 100A are configured to capture from the environment of the robot.
  • the robot 100 A can include a directional microphone subsystem having one or more microphones.
  • the directional microphone subsystem also includes post-processing functionality that generates a direction, a direction probability distribution, location, or location probability distribution in a particular coordinate system in response to receiving a sound. Each generated direction represents a most likely direction from which the sound originated.
  • the directional microphone subsystem can use various conventional beam-forming algorithms to generate the directions.
  • the touch detection subsystems of the robot 100A are configured to determine when the robot is being touched or touched in particular ways.
  • the touch detection subsystems can include touch sensors, and each touch sensor can indicate when the robot is being touched by a user, e.g., by measuring changes in capacitance.
  • the robot can include touch sensors on dedicated portions of the robot’s body, e.g., on the top, on the bottom, or both. Multiple touch sensors can also be configured to detect different touch gestures or modes, e.g., a stroke, tap, rotation, or grasp.
  • the motion detection subsystems of the robot 100A are configured to measure the movement of the robot.
  • the motion detection subsystems can include motion sensors and each motion sensor can indicate that the robot is moving in a particular way.
  • a gyroscope sensor can indicate a relative orientation of the robot.
  • an accelerometer can indicate a direction and a magnitude of an acceleration, e.g., of the Earth’s gravitational field.
  • the effector input subsystems of the robot 100A are configured to determine when a user is physically manipulating components of the robot 100A. For example, a user can physically manipulate the lift of the effector subsystem 120, which can result in an effector input subsystem generating an input signal for the robot 100A. As another example, the effector subsystem 120 can detect whether or not the lift is currently supporting the weight of any objects. The result of such a determination can also result in an input signal for the robot 100 A.
  • the robot 100A can also use inputs received from one or more integrated input subsystems.
  • the integrated input subsystems can indicate discrete user actions with the robot 100A.
  • the integrated input subsystems can indicate when the robot is being charged, when the robot has been docked in a docking station, and when a user has pushed buttons on the robot, to name just a few examples.
  • the robot 100A can also use inputs received from one or more accessory input subsystems that are configured to communicate with the robot 100A.
  • the robot 100 A can interact with one or more cubes that are configured with electronics that allow the cubes to communicate with the robot 100A wirelessly.
  • Such accessories that are configured to communicate with the robot can have embedded sensors whose outputs can be
  • a cube can be configured with a motion sensor and can communicate an indication that a user is shaking the cube.
  • the robot 100A can also use inputs received from one or more environmental sensors that each indicate a particular property of the environment of the robot.
  • Example 1 The robot 100A can also use inputs received from one or more environmental sensors that each indicate a particular property of the environment of the robot.
  • environmental sensors include temperature sensors and humidity sensors to name just a few examples.
  • One or more of the input subsystems described above may also be referred to as “sensor subsystems.”
  • the sensor subsystems allow a robot to determine when a user is interacting with the robot, e.g., for the purposes of providing user input, using a
  • the representations generated by the sensor subsystems may be referred to as“sensor inputs.”
  • the robot 100A also includes computing subsystems having data processing hardware, computer-readable media, and networking hardware. Each of these components can serve to provide the functionality of a portion or all of the input and output subsystems described above or as additional input and output subsystems of the robot 100A, as the situation or application requires.
  • one or more integrated data processing apparatus can execute computer program instructions stored on computer-readable media in order to provide some of the functionality described above.
  • the robot 100A can also be configured to communicate with a cloud-based computing system having one or more computers in one or more locations.
  • the cloud-based computing system can provide online support services for the robot.
  • the robot can offload portions of some of the operations described in this specification to the cloud- based system, e.g., for determining behaviors, computing signals, and performing natural language processing of audio streams.
  • FIG. 1B is a block diagram illustrating an input and output system 100B of an example robot, e.g., the example robot 100A.
  • the system 100B includes a controller 160 that receives inputs from sensors 172.
  • the sensors can help inform the detection of the transportation mode.
  • an accelerometer 162 and a gyroscope 168 can indicate accelerations and rotations that describe the motion of the robot;
  • a GPS receiver 164 can reveal over time the speed and absolute direction of motion, and
  • a microphone 166 can provide a spectrum of the audible vibrations caused by the hum of a vehicle.
  • the robot sensors are thus selected based on the transportation modes among which the robot should be capable of differentiating.
  • the controller 160 can process the inputs to generate outputs directed to one or more output devices.
  • the controller 160 can be coupled to output systems including actuators l74a-l74n, an electronic display 176, and a speaker 178.
  • FIG. 2 is a block diagram showing an example software architecture 200 of an example robot.
  • the software architecture 200 includes a sensor subsystem 204, a transportation mode classifier 202 (TMC), an emotion state engine 206, a behavior selection engine 208, an animation and action selection engine 210, and an animation and action generation engine 212.
  • TMC 202 is a subsystem that analyzes the current and/or historical sensor data to detect the transportation mode of the robot.
  • a transportation mode is a class of circumstances under which the robot is in transit due to the influence of external forces.
  • external forces are forces generated by entities that are not a part of the robot’s integrated subsystems.
  • Each transportation mode thus represents that the robot’s location is changing, e.g., that the robot is in transit, due to forces of an external entity.
  • Each transportation mode also classifies this movement as happening in accordance with one of an enumerated set of circumstances.
  • a transportation mode can represent that the robot is being moved in a particular way while being held, carried, or moved by a user.
  • the transportation mode can thus represent that the robot is moving with a user who is walking, running, or climbing stairs, to name just a few examples.
  • the transportation mode can also represent the robot being moved by a user, e.g., being pushed or pulled.
  • a transportation mode can also represent that the robot is being moved by a particular apparatus, which may or may not involve being held or carried by a user.
  • the transportation mode can thus also represent that the robot is being moved by a bicycle, a car, a bus, a boat, a train, or an aircraft, to name just a few examples.
  • the transportation mode can reflect and be related to human locomotion, e.g., when the robot is being carried by a user who is walking; or be independent of human locomotion, e.g., when the robot is transported in a car.
  • the TMC 202 is developed using data previously collected by the robot. Else of such “training data” ensures that the TMC 202 is tailored to the given sensor configuration of the robot. Furthermore, by collecting these data using multiple robot units, the algorithm is more robust to variation among robot units.
  • the output of the TMC 202 may include a level of certainty or confidence in the detected transportation mode, or the TMC 202 may produce a list of all possible
  • the TMC 202 may evolve over time based on the frequency of detected transportation modes or other factors. For instance, a robot that is taken for a car ride at the same time every day may use that information to simplify the subsequent detection of the current transportation mode at that time.
  • the robot can also maintains an emotion state engine 206, which is a subsystem that maintains an internal state in way that is informed by current, recent, and/or long-term external stimuli.
  • the emotion state engine 206 can maintain the current general“mood” of the robot, along with a list of emotion states and the respective intensities of those emotion states.
  • the sensor data received from the sensor subsystems 204 is used as input to the emotion state engine, and a programmed model of how the data affects the emotion state engine 206 calculates how the emotion states and“mood” should be updated.
  • the emotion state for a robot can be a single-dimensional or a multi-dimensional data structure, e.g., a vector or an array, that maintains respective values for each of one or more different aspects.
  • Each aspect can represent an enumerated value or a particular value on a simulated emotional spectrum, with each value for each aspect representing a location within that simulated emotional spectrum.
  • an example emotion state can have the following values: Happy, Calm, Brave, Confident, Excited, and Social, each of which may have a negative counterpart.
  • the emotion states need not correspond to specifically identifiable human emotions. Rather, the emotion state can also represent other, more general or more specific spectrums that characterize robot behavior.
  • the emotion state can be a Social state that represents how eager the robot is to interact with users generally, a Want-To-Play state that represents how eager the robot is to engage in gameplay with a user, and a Winning state that represents how competitive the robot is in games.
  • the emotion state can also correspond to a desired or impending change in external circumstances, such as changing the state of the user or an object, e.g., Want-to-F lighten or Want-to- Soothe.
  • Emotion states can also correspond to current physical states of the robot, such as Needs-Repair and Hungry. Such states can manifest in the same sort of character and motion constraints as other emotion states.
  • the emotion states can enhance user engagement with the robot and can improve the interface between users and the robot by making the robot’s actions and responses readily understandable.
  • the transportation mode generated by the TMC 202 is also an input to the emotion state engine 206.
  • the robot emotion state is updated based on a model of how the current transportation mode, or changes in the transportation mode, affect the emotion state.
  • the transportation mode input is also fused with current sensor data, allowing for complex robot emotional responses that reflect the context of use. For instance, a robot that experiences a large jolt while stationary may have increased“anxiety,” while a similar jolt when the robot is in a car may be considered par for the course, and it would not change the“anxiety” level of the emotion state.
  • the model underlying the emotion state engine 206 can also take into account one or more distinct robot“personality” types, which further helps to determine how the emotion state should change for a given transportation mode. For instance, the emotion state engine 206 of a“fearful” robot who is taken for a car ride may update its state with increased“anxiety.” A“carefree” robot, on the other hand, may increase the intensity of its “glee” emotion state when experiencing a car ride.
  • the behavior selection engine 206 takes as input the current sensor data from the sensor subsystems 204, the emotion state from the emotion state engine 206, and the output of the TMC 202, and from those inputs, the behavior selection engine 208 chooses a high- level behavior for the robot to perform.
  • Each behavior involves performing one or more actions.
  • the inclusion of the transportation mode alongside sensor data and the emotion state allows for the selection or exclusion of certain behaviors based on the transportation mode. For instance, if the microphone 166 recognizes the verbal command“do the dishes,” but the robot detects that the robot is riding in a car, the selected behavior may express confusion.
  • the three inputs can also be weighted, so that one source of input can have a stronger influence on the selected behavior than others, and this weighting can change over time.
  • the inputs can be weighted based on their value, so that, for instance, if the TMC 202 detects a car ride, the TMC 202 is weighted more heavily than the emotion state engine input in determining the robot behavior.
  • These weightings can also change over time.
  • the weightings can also vary based on the confidence of the measurement. For instance, if the TMC algorithm has not completed or has returned a result or results with low confidence, the emotion state or current sensor data can have a greater influence on the selected behavior.
  • the behavior selection engine 208 can also choose its behavior based on changes in the detected transportation mode. For example, the robot may trigger a behavior that expresses“excitement” when the transportation mode transitions from stationary to driving, or from walking to running. Furthermore, the behavior selection may evolve over time based on the regularity of the transportation mode or other factors. For instance, the selected behavior when a robot is first taken for a car ride could differ from the selected behavior after the robot rides in a car regularly.
  • Any unexpected transportation modes can also trigger specific behaviors. For example, if the robot expects that the robot should remain on the ground in a given area, and the robot instead detects that it is being carried for a walk or is in a car, that irregularity could trigger a specific behavior, as well as a new state of "alert" in the emotion state engine.
  • the behavior selection engine 208 uses the animation and action selection engine 210 to queue a list of actions to be performed by various components of the robot.
  • the actions can include physical actions that cause the actuation of physically moveable components, display actions that cause the presentation of visual or audio data through electronic displays or speakers.
  • These actions may include“animations” designed to convey a particular emotional response, or more basic actions like“drive forward.” These animations may be distinct to a particular transportation mode. For instance, a“riding in the car” animation might exist, but the“riding in the car” animation may be an animation that is only executed when the TMC 202 determines that the robot is riding in a car.
  • the animation and action generation engine 212 can translate the selected actions into commands for actuators, displays, speakers, and/or other components of the robot.
  • the output of the animation & action generation engine 212 is also used by the TMC 202 to improve the quality of the estimated transportation mode. Because the robot may be moving under its own accord, the TMC 202 can modify its input sensor data to account for the known motion of the robot. For instance, if the action generation engine instructs the robot to accelerate forward, the TMC 202 can ignore a corresponding acceleration in that direction, so that it considers sensor input only from unbiased external forces.
  • FIG. 3 is a flowchart of an example process for training a transportation mode classifier for a robot.
  • the example process will be described as being performed by a system of one or more computers, located in one or more locations, and programmed appropriately in accordance with this specification.
  • the example process can be performed by a system of computers in a datacenter programmed to build a transportation mode classifier for a robot.
  • the system collects training data in different transportation modes (310).
  • the system can collect training data from a single robot or from multiple robot units.
  • the training data include features indicating a robot’s status such as registered values for the accelerometer, the GPS receiver, and the gyroscope, to name just a few examples.
  • Each training example can be labeled with a mode of transportation used to generate the features.
  • the following section outlines an example technique for training a transportation mode classifications for a robots that each have n integrated an inertial measurement unit (IMU) having a 3-axis gyroscope and a 3-axis accelerometer.
  • IMU inertial measurement unit
  • Each robot is programmed to record IMEG data to a formatted log file at 200 Hz. Multiple units of the robot are produced and given to multiple individuals. Each individual is instructed to perform each of the following actions, during which time logs of IMU data are being recorded:
  • the IMU logs are then obtained from the robots.
  • the 3-axis gyroscope and 3-axis accelerometer logs are each replicated as down-sampled versions, with
  • the system uses the collected training data to train a model to predict a current transportation mode (520).
  • the system can select the model type from a number of different models. Model selection can involve creating several machine learning classifiers that can each predict the transportation mode from a feature vector, and then the classifier that performs most effectively is identified. Techniques for supervised learning are selected, including a Random Forest, Support Vector Machine (SVM), a Naive Bayes classifier, and AdaBoost.
  • SVM Support Vector Machine
  • AdaBoost AdaBoost
  • the system can use cross-validation to test the accuracy of each model.
  • the data can be randomly partitioned into ten equal groups of labeled feature vectors in order to perform k-fold cross-validation, as follows. One group of the partition is withheld, and then each classifier is trained on the remaining nine groups of labeled features. The withheld group then serves as a way to test the accuracy of the trained classifiers.
  • a single trained classifier predicts the transportation mode for each feature vector in the test data (i.e., the withheld group), which produces an accuracy score formed by dividing the number of correct predictions of transportation mode by the number of sequences in the test data. This score is calculated for each classifier.
  • each classifier will have ten accuracy scores, and the classifier with the highest average accuracy score can be selected for further use.
  • the model selection process can be repeated for each of the 48 ( T ) pairs to find the model parameters with the highest accuracy.
  • the final result is a single model trained on the entire set of labeled feature vectors that correspond to a single pair (/ T).
  • the model selection process can be repeated using all possible subsets, e.g., the power set, of the calculated statistics.
  • the system provides a trained model to predict a current transportation to a robot (330).
  • the trained model with the highest accuracy can be programmed on the robot.
  • the robot can also be programmed to translate instantaneous IMU readings into sequences of length T and frequency/ chosen through the above process, and to compute the statistics of the sequences, so that a feature vector is generated from recent IMU data on a regular interval.
  • the trained model predicts the transportation mode from the sequence’s feature vector. That prediction, and any measure of confidence, can then be used as input to the emotion state engine and the behavior selection engine. It may be advantageous to do these calculations synchronously or asynchronously.
  • FIG. 4 is a flowchart of an example process for generating a behavior for a robot based on a predicted transportation mode.
  • the example process can be performed by any appropriate system having a transportation mode classifier.
  • the example process will be described as being performed a robot having an emotion state engine and configured with a transportation mode classifier.
  • the robot receives a plurality of sensor inputs (410).
  • the system can receive sensor inputs from the sensor subsystem 204 as described above with reference to FIGS. 1-2.
  • the robot generates a predicted transportation mode based on the received sensor inputs (420).
  • the system can use a transportation mode classifier to predict a transportation mode of the robot as explained above with reference to FIG. 2
  • the robot provides the predicted transportation mode to the emotion state engine (430).
  • the emotion state engine is configured to maintain a single or multi-dimensional data structure that represents a state of the robot in one or more different simulated aspects.
  • the robot updates an internal emotion state based on the predicted transportation mode of the robot (440). For instance, a robot that experiences a large jolt while stationary may results in an increased value for an“anxiety” aspect of the emotion state while a similar jolt when the robot is in a car may be considered normal and does not change the“anxiety” aspect of the emotion state.
  • the robot provides the updated emotion state to a behavior selection engine (450).
  • the system can increase the“confusion” aspect of the emotion state and provide the updated emotion state to the behavior selection engine.
  • the robot generates a behavior for the robot to perform based on the updated emotion state (460).
  • the system can cause the animation and action selection engine 210 to select a behavior that expresses excitement when the transportation mode transitions from stationary to driving, or from walking to running.
  • the robot performs the behavior (470).
  • the selected behavior from the animation and action selection engine 210 can be passed to the animation and action generation engine 212 so that the selected actions are translated into commands for actuators, displays, speakers, and/or other components of the robot.
  • FIGS. 5A-5C illustrate different robot behaviors generated as a result of the robot detecting different transportation modes.
  • a robot is being moved by a bicycle operated by a user.
  • the robot is programmed to simulate enjoying bicycle rides. Therefore, the emotion state engine of the robot can update its emotion state when the transportation classifier determines that the robot is riding on a bicycle. As a result, a behavior engine of the robot can select a happy face 510 to output to the display of the robot.
  • the robot is being moved by a vehicle driven by a user.
  • the robot has exhibited different animations and actions.
  • the robot has produced an audio output 520 to indicate an internal emotional state of excitement.
  • the robot has exhibited an emotional state that reflects nervousness by providing an audio output 530,“WOAH! Going a little fast there buddy!” that reminds the user to use caution while driving.
  • Embodiments of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, in tangibly-embodied computer software or firmware, in computer hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them.
  • Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible non-transitory storage medium for execution by, or to control the operation of, data processing apparatus.
  • the computer storage medium can be a machine- readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of them.
  • the program instructions can be encoded on an artificially-generated propagated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus for execution by a data processing apparatus.
  • data processing apparatus refers to data processing hardware and encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers.
  • the apparatus can also be, or further include, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).
  • the apparatus can optionally include, in addition to hardware, code that creates an execution environment for computer programs, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
  • a computer program which may also be referred to or described as a program, software, a software application, an app, a module, a software module, a script, or code) can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.
  • a program may, but need not, correspond to a file in a file system.
  • a program can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub-programs, or portions of code.
  • a computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a data communication network.
  • For a system of one or more computers to be configured to perform particular operations or actions means that the system has installed on it software, firmware, hardware, or a combination of them that in operation cause the system to perform the operations or actions.
  • the one or more computer programs to be configured to perform particular operations or actions means that the one or more programs include instructions that, when executed by data processing apparatus, cause the apparatus to perform the operations or actions.
  • an“engine,” or“software engine,” refers to a software implemented input/output system that provides an output that is different from the input.
  • An engine can be an encoded block of functionality, such as a library, a platform, a software development kit (“SDK”), or an object.
  • SDK software development kit
  • Each engine can be implemented on any appropriate type of computing device, e.g., servers, mobile phones, tablet computers, notebook computers, music players, e-book readers, laptop or desktop computers, PDAs, smart phones, or other stationary or portable devices, that includes one or more processors and computer readable media. Additionally, two or more of the engines may be implemented on the same computing device, or on different computing devices.
  • the processes and logic flows described in this specification can be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output.
  • the processes and logic flows can also be performed by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and one or more programmed computers.
  • Computers suitable for the execution of a computer program can be based on general or special purpose microprocessors or both, or any other kind of central processing unit.
  • a central processing unit will receive instructions and data from a read-only memory or a random access memory or both.
  • the essential elements of a computer are a central processing unit for performing or executing instructions and one or more memory devices for storing instructions and data.
  • the central processing unit and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
  • a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto-optical disks, or optical disks.
  • a computer need not have such devices.
  • a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name just a few.
  • PDA personal digital assistant
  • GPS Global Positioning System
  • USB universal serial bus
  • Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.
  • semiconductor memory devices e.g., EPROM, EEPROM, and flash memory devices
  • magnetic disks e.g., internal hard disks or removable disks
  • magneto-optical disks e.g., CD-ROM and DVD-ROM disks.
  • embodiments of the subject matter described in this specification can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor, for displaying information to the user and a keyboard and pointing device, e.g, a mouse, trackball, or a presence sensitive display or other surface by which the user can provide input to the computer.
  • a display device e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor
  • keyboard and pointing device e.g, a mouse, trackball, or a presence sensitive display or other surface by which the user can provide input to the computer.
  • Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the user can be received in any form, including acoustic, speech, or tactile input.
  • a computer can interact with a user by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user’s device in response to requests received from the web browser.
  • a computer can interact with a user by sending text messages or other forms of message to a personal device, e.g., a smartphone, running a messaging application, and receiving responsive messages from the user in return.
  • Embodiments of the subject matter described in this specification can be implemented in a computing system that includes a back-end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front-end component, e.g., a client computer having a graphical user interface, a web browser, or an app through which a user can interact with an implementation of the subject matter described in this specification, or any combination of one or more such back-end, middleware, or front- end components.
  • the components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN) and a wide area network (WAN), e.g., the Internet.
  • LAN local area network
  • WAN wide area network
  • the computing system can include clients and servers.
  • a client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
  • a server transmits data, e.g., an HTML page, to a user device, e.g., for purposes of displaying data to and receiving user input from a user interacting with the device, which acts as a client.
  • Data generated at the user device e.g., a result of the user interaction, can be received at the server from the device.
  • one or more sensor subsystems that are configured to generate sensor inputs
  • one or more storage devices storing instructions that are operable, when executed by the one or more processors, to cause the robot to perform operations comprising:
  • Embodiment 2 is the robot of embodiment 1, wherein the predicted transportation mode represents that the robot is in motion due to the forces of an external entity and identifies one of an enumerated class of circumstances under which the motion occurs.
  • Embodiment 3 is the robot of any one of embodiments 1-2, wherein the predicted transportation mode represents that the robot is being moved in a particular way while being held or carried by a user.
  • Embodiment 4 is the robot of embodiment 3, wherein the transportation mode represents that the robot is moving with a user who is walking, running, or climbing stairs.
  • Embodiment 5 is the robot of any one of embodiments 1-4, wherein the predicted transportation mode represents that the robot is being moved by a particular apparatus.
  • Embodiment 6 is the robot of embodiment 5, wherein the predicted transportation mode represents that the robot is being moved by a bicycle, a car, a bus, a boat, a train, or an aircraft.
  • Embodiment 7 is the robot of any one of embodiments 1-6, wherein in response to a particular stimulus, the robot is configured to generate and perform a first behavior for a first predicted transportation mode and to generate and perform a different second behavior for a different second predicted transportation mode.
  • Embodiment 8 is the robot of any one of embodiments 1-7, wherein the transportation mode classifier is trained using training data generated from logs generated by integrated inertial measurement units of one or more robots being transported in each of a plurality of different transportation modes.
  • Embodiment 9 is the robot of any one of embodiments 1-8, wherein generating the predicted transportation mode of the robot comprises:
  • Embodiment 10 is the robot of embodiment 9, wherein the transportation mode features include one or more of a mean accelerometer magnitude, a standard deviation of the accelerometer magnitude, a minimum accelerometer magnitude over the sequence duration, a maximum accelerometer magnitude over the sequence duration, an autocorrelation of the accelerometer magnitude, a mean gyroscope magnitude, a standard deviation of the gyroscope magnitude, a minimum gyroscope magnitude over the sequence duration, a maximum gyroscope magnitude over the sequence duration, or an autocorrelation of the gyroscope magnitude.
  • the transportation mode features include one or more of a mean accelerometer magnitude, a standard deviation of the accelerometer magnitude, a minimum accelerometer magnitude over the sequence duration, a maximum accelerometer magnitude over the sequence duration, an autocorrelation of the gyroscope magnitude.
  • Embodiment 11 is a method comprising performing, by a robot, the operations of any one of claims 1-10.
  • Embodiment 12 is one or more computer storage media encoded with computer program instructions that when executed by a robot causes the robot to perform the operations of any one of claims 1-10.

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Toys (AREA)

Abstract

L'invention concerne des procédés, des systèmes et un appareil, comprenant des programmes informatiques codés sur des supports de mémoire informatique, pour réaliser une classification de mode de transport. Un système comprend un robot configuré pour prédire un mode de transport à partir d'entrées de capteur à l'aide d'un classificateur de mode de transport. Le mode de transport prédit sert à générer un état d'émotion mis à jour, et un comportement pour le robot est généré à l'aide de l'état d'émotion mis à jour.
PCT/US2019/020724 2018-03-06 2019-03-05 Classification de mode de transport de robot Ceased WO2019173321A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862639364P 2018-03-06 2018-03-06
US62/639,364 2018-03-06

Publications (1)

Publication Number Publication Date
WO2019173321A1 true WO2019173321A1 (fr) 2019-09-12

Family

ID=67847467

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/020724 Ceased WO2019173321A1 (fr) 2018-03-06 2019-03-05 Classification de mode de transport de robot

Country Status (1)

Country Link
WO (1) WO2019173321A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6362589B1 (en) * 1919-01-20 2002-03-26 Sony Corporation Robot apparatus
US20030060930A1 (en) * 2000-10-13 2003-03-27 Masahiro Fujita Robot device and behavior control method for robot device
US20040243281A1 (en) * 2002-03-15 2004-12-02 Masahiro Fujita Robot behavior control system, behavior control method, and robot device
US20110282620A1 (en) * 2010-05-14 2011-11-17 Casio Computer Co., Ltd. Traveling state determining device, method for determining traveling state and recording medium
WO2014118767A1 (fr) * 2013-02-03 2014-08-07 Sensogo Ltd. Classification de types de locomotion

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6362589B1 (en) * 1919-01-20 2002-03-26 Sony Corporation Robot apparatus
US20030060930A1 (en) * 2000-10-13 2003-03-27 Masahiro Fujita Robot device and behavior control method for robot device
US20040243281A1 (en) * 2002-03-15 2004-12-02 Masahiro Fujita Robot behavior control system, behavior control method, and robot device
US20110282620A1 (en) * 2010-05-14 2011-11-17 Casio Computer Co., Ltd. Traveling state determining device, method for determining traveling state and recording medium
WO2014118767A1 (fr) * 2013-02-03 2014-08-07 Sensogo Ltd. Classification de types de locomotion

Similar Documents

Publication Publication Date Title
US10970527B2 (en) Robot attention detection
US11625508B2 (en) Artificial intelligence device for guiding furniture placement and method of operating the same
US10898999B1 (en) Selective human-robot interaction
US11465294B2 (en) Artificial intelligence robot for performing cleaning using pollution log and method for same
EP4144425A1 (fr) Dispositif et procédé de commande de comportement, et programme
US20200042832A1 (en) Artificial intelligence apparatus and method for updating artificial intelligence model
US20190111563A1 (en) Custom Motion Trajectories for Robot Animation
US11583998B2 (en) Robot and method of controlling same
US20180319015A1 (en) Apparatus and methods for hierarchical training of robots
US20230241782A1 (en) Condition-Based Robot Audio Techniques
US10872438B2 (en) Artificial intelligence device capable of being controlled according to user's gaze and method of operating the same
KR20210035968A (ko) 사용자의 표정이나 발화를 고려하여 마사지 동작을 제어하는 인공 지능 마사지 장치 및 그 방법
US20190354178A1 (en) Artificial intelligence device capable of being controlled according to user action and method of operating the same
US11029803B2 (en) Robot
US20200001469A1 (en) Robot
KR20200036680A (ko) 감정 정보 획득을 위한 전자 장치 및 방법
US11548144B2 (en) Robot and controlling method thereof
US11322134B2 (en) Artificial intelligence device and operating method thereof
US11203122B2 (en) Goal-based robot animation
JPWO2020004213A1 (ja) 情報処理装置、情報処理方法、およびプログラム
US20210155262A1 (en) Electronic apparatus and operation method thereof
KR20210007965A (ko) 제어 장치, 제어 방법 및 프로그램
US11433548B2 (en) Robot system and control method thereof
US20220365588A1 (en) Information processing apparatus, information processing method, and program
US11468270B2 (en) Electronic device and feedback information acquisition method therefor

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19763831

Country of ref document: EP

Kind code of ref document: A1