CN111300412A - A method of controlling robots based on Unreal Engine - Google Patents

Abstract

The invention relates to a method for controlling a robot based on an Unreal Engine, comprising: S1, sending a control command to a robot communication module via a Socket network through a robot simulation model built in the upper computer Unreal Engine 4; S2, the robot communication module according to a preset The communication protocol parses the control commands, and sends the parsed control commands to the robot through wireless network communication; S3, the robot makes corresponding motions according to the control commands, and regularly transmits the current actual rotation angle of the steering gear through the robot communication module. Feedback to the robot simulation model on Unreal Engine 4; the present invention controls the snake robot remotely and wirelessly through the robot simulation model built in the host computer Unreal Engine 4, and feeds back the current pose state of the snake robot for display in real time, which can solve the problem of snake shape. The problem of wireless telemetry and remote control of the robot, and the visual user control interface also makes the development and debugging of the robot more convenient.

Description

A method of controlling robots based on Unreal Engine

technical field

The invention relates to the technical field of robot control technology, in particular to a method for controlling a robot based on an Unreal Engine.

Background technique

With the advancement of science and technology, the development of robotics has also entered a new stage. A large number of robots have been applied to different fields, especially in environments that are difficult for humans to explore or are likely to cause harm to personal safety. Different types of robots are required. Robots to replace humans to complete the actual work. At present, there are many forms of robot host computer development in the industry, such as creating a graphical control interface based on LabVIEW developed by National Instruments (NI), developing console programs based on Microsoft MFC architecture, or using Robot Operating System (ROS) Combining the Qt framework to develop control software, there is a problem with these methods, that is, the developed control software interface is relatively simple, only a graphic panel for sending control commands, and the current state of the robot can only be displayed by numerical values or curves, and there is no visual robot model.

Unreal Engine 4 (Unreal Engine) 4 is currently the most well-known and most widely authorized top game engine in the world. Although it was originally developed to serve game developers, with the continuous enrichment and enhancement of its functions, it can be applied The rest of the field is also growing, such as construction, automotive and transportation, film and television, training and simulation and other industry applications. In terms of physical simulation, Unreal Engine 4 adopts the PhysX physical computing engine, so that objects in the virtual world can conform to the physical motion laws of the real world, making the movement of the internal model more realistic. In terms of graphics processing, it supports the DirectX graphics rendering functions developed by Microsoft, such as the display of light sources in the scene, the material and shading rendering of the physical properties of the model surface, etc., making the objects in the whole world more vivid and gorgeous. In terms of model making, Unreal Engine 4 has a variety of basic models, such as cylinders, spheres, cubes, etc., and can make complete models for users. If you pursue more refined models, you can also import models made by software such as 3D Max. In terms of user interface, it has rich graphical interface components for users to carry out secondary development.

To sum up, there is an urgent need in the industry to develop a visual method or system for controlling robots based on Unreal Engine 4.

SUMMARY OF THE INVENTION

Aiming at the shortcomings of the existing robot's current state that can only be displayed through numerical values or curves, and does not have a visual robot model, the present invention designs a method for controlling a robot based on Unreal Engine.

The specific scheme of this application is as follows:

An Unreal Engine-based method for controlling a robot, comprising:

S1, send control commands to the robot communication module through the Socket network through the robot simulation model built in the host computer Unreal Engine 4;

S2, the robot communication module parses the control command according to the preset communication protocol, and sends the parsed control command to the robot through wireless network communication;

S3, the robot makes corresponding movements according to the control commands, and periodically feeds back the actual rotation angle of the steering gear to the robot simulation model on Unreal Engine 4 through the robot communication module;

S4, after the simulation model receives the angle feedback signal, it will sequentially assign the angle value to the corresponding simulation module, and adjust the pose of the simulation model according to the angle feedback signal, so that the pose of the simulation model is consistent with the robot entity, and through the phantom The display of the engine 4 simulation system shows the current pose state of the robot.

Preferably, the step of building a robot simulation model in the host computer Unreal Engine 4 includes:

S11, using Unreal Engine 4 to build a simulation model of the robot,

S12, making materials and textures for the simulation model, and making a scene model for the simulation model;

S13, use Unreal Engine 4 to make a visual robot graphical control interface, and bind different control commands for each key of the graphical control interface; wherein if the key is clicked, the control command will be sent to the robot communication module through the Socket network TCP protocol .

Preferably, the robot includes: a head joint module and N motion joint modules, N≥2; the head joint module is connected to the motion joint module through a bus, and the head joint module is used for receiving control commands, Analyze data and send control commands to the motion joint module through the bus; the motion joint module is used to control the steering gear to move according to the control command.

Preferably, the head joint module includes an STM32 series microprocessor, a WIFI module, an infrared sensor and a camera, the motion joint module includes a servo microprocessor, a driving state feedback module, and a servo, the WIFI module and the robot The communication module is connected, the STM32 series microprocessor and the steering gear microprocessor are connected through a bus, the steering gear microprocessor and the steering gear are both connected, and the STM32 series microprocessor is also connected with the drive state feedback module.

Preferably, step S3 includes:

S31, after receiving the control command sent by the robot communication module, the WIFI module sends the control command to the STM32 series microprocessor of the head joint module through serial communication;

S32, the STM32 series microprocessor parses the received control command, and controls the motion joint module through the bus;

S33, after the motion joint module receives the control command from the bus, the microprocessor will parse the control command, and control the steering gear to move to a corresponding angle, the robot makes a corresponding movement, and the drive status feedback module periodically reports the current steering gear to the current position. The actual rotation angle is fed back to the STM32 series microprocessor of the head joint module;

S34: After receiving the angle feedback of each motion joint module, the STM32 series microprocessor of the head joint module forwards it to the robot simulation model on Unreal Engine 4 through the robot communication module.

Preferably, the scene model made by the simulation model is a flat ground, a pipeline, a mountain or a cable.

Compared with the prior art, the present invention has the following beneficial effects:

The present invention controls the snake-shaped robot remotely and wirelessly through the robot simulation model built in the host computer Unreal Engine 4, and feeds back the current pose state of the snake-shaped robot for display in real time, which can solve the problem of wireless telemetry and remote control of the snake-shaped robot. The control interface also facilitates the development and debugging of the robot. details as follows:

(1) The present invention adopts a virtual reality engine to develop a visual interaction interface in the remote operation of the snake-shaped robot, so that the system has a better human-computer interaction function, reduces the operation difficulty, and improves the operation efficiency;

(2) The present invention uses angle feedback to visualize the pose information of the snake robot in the simulation scene of the virtual engine in real time, so that the operator can grasp the current motion state and pose of the robot in real time and accurately;

(3) The robot simulation and physical machine control can be integrated in the same software to improve the development efficiency of the staff;

(4) It can add functions such as virtual reality and augmented reality, which has good scalability and is conducive to the display and promotion of robots.

Description of drawings

FIG. 1 is a schematic flowchart of a method for controlling a robot based on Unreal Engine according to an embodiment.

FIG. 2 is a schematic block diagram of a robot controlled based on Unreal Engine according to an embodiment.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments These are some embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

Referring to Figure 1-2, a method of controlling a robot based on Unreal Engine, including:

S1, send control commands to the robot communication module through the Socket network by the robot simulation model built in the host computer Unreal Engine 4; wherein, the steps of building the robot simulation model in the host computer Unreal Engine 4 include:

S11, using Unreal Engine 4 to build a simulation model of the robot,

S12 , making materials and textures for the simulation model, and making a scene model for the simulation model; the scene model made by the simulation model is a flat ground, a pipeline, a mountain or a cable. Make materials and textures for the model to enrich the visual experience of the snake robot simulation model.

S13, use Unreal Engine 4 to make a visual robot graphical control interface, and bind different control commands for each key of the graphical control interface; wherein if the key is clicked, the control command will be sent to the robot communication module through the Socket network TCP protocol . The graphic control interface is designed for button layout according to the functions actually required by the snake-shaped robot, which simplifies the user's control operation on the robot.

In this embodiment, the robot is a snake-shaped robot. The snake-shaped robot includes: a head joint module and N motion joint modules, N≥2; the head joint module is connected with the motion joint module through a bus, and the head joint module is used for receiving control commands, analyzing data and send control commands to the motion joint module through the bus; the motion joint module is used to control the steering gear to move according to the control command. The head joint module includes an STM32 series microprocessor, a WIFI module, an infrared sensor and a camera, and the motion joint module includes a steering gear microprocessor, a drive state feedback module, and a steering gear. The WIFI module is connected to the robot communication module. , the STM32 series microprocessor and the steering gear microprocessor are connected through a bus, the steering gear microprocessor and the steering gear are both connected, and the STM32 series microprocessor is also connected with the drive state feedback module. The snake-shaped robot further includes: a power supply system, the power supply system is used to supply power to each module of the snake-shaped robot.

In this embodiment, the STM32 series microprocessors are STM32F103C8T2 microprocessors.

S2, the robot communication module parses the control command according to the preset communication protocol, and sends the parsed control command to the robot through wireless network communication; the snake-shaped robot communication module can be a software program, which uses C++ language combined with socket network For development, it is an intermediate bridge between the two-way communication between the simulation model in the host computer Unreal Engine 4 and the snake robot body. The snake robot communication module is responsible for parsing and sending the control instructions sent by the Unreal Engine 4 simulation system to the snake robot body. Responsible for transmitting the sensor information and angle feedback information on the snake robot body to the Unreal Engine 4 simulation system.

S3, the robot makes corresponding movements according to the control commands, and regularly feeds back the actual rotation angle of the steering gear to the robot simulation model on the Unreal Engine 4 through the robot communication module; specifically, step S3 includes:

S31, after receiving the control command sent by the robot communication module, the WIFI module sends the control command to the STM32 series microprocessor of the head joint module through serial communication;

S32, the STM32 series microprocessor parses the received control command, and controls the motion joint module through the bus;

S33, after the motion joint module receives the control command from the bus, the microprocessor will parse the control command, and control the steering gear to move to a corresponding angle, the robot makes a corresponding movement, and the drive status feedback module periodically reports the current steering gear to the current position. The actual rotation angle is fed back to the STM32 series microprocessor of the head joint module;

Among them, if the control command is the relevant command of infrared and camera, the robot will perform the corresponding opening/closing operation. If the control command is the relevant command of the joint module movement, the TM32 series microprocessor will parse the control command and control the protocol through the CAN bus. Control all kinematic joint modules connected to the bus, so that the snake-shaped robot entity makes corresponding movements;

S34: After receiving the angle feedback of each motion joint module, the STM32 series microprocessor of the head joint module forwards it to the robot simulation model on Unreal Engine 4 through the robot communication module.

S4, after the simulation model receives the angle feedback signal, it will sequentially assign the angle value to the corresponding simulation module, and adjust the pose of the simulation model according to the angle feedback signal, so that the pose of the simulation model is consistent with the robot entity, and through the phantom The display of the engine 4 simulation system shows the current pose state of the robot. This solution uses Unreal Engine 4 simulation software to pre-build the working scene of the snake robot and the simulation model of the snake robot. And according to the motion information of the snake-shaped robot body, the position and posture information of each joint is parsed and applied to the corresponding joints of the simulation model, and the display is used to restore and display the motion posture of the current robot.

In the host computer, the virtual engine 4 simulation system controls the joint rotation angle of the snake robot through Socket network communication and WIFI module, thereby realizing the remote control of the overall motion of the robot, and can also receive the joint angle from the snake robot in real time. Feedback and restore the motion pose of the snake-like robot in the display of the simulation system, with strong observability, high stability and good versatility.

The above-mentioned embodiments only represent several embodiments of the present invention, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can also be made, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention should be subject to the appended claims.

Claims (6)

1. A method of controlling a robot based on a ghost engine, comprising:

s1, sending a control command to the robot communication module through a Socket network by a robot simulation model built in the upper computer illusion engine 4;

s2, the robot communication module analyzes the control command according to a preset communication protocol and sends the analyzed control command to the robot through wireless network communication;

s3, the robot makes corresponding movement according to the control command, and feeds back the current actual rotation angle of the steering engine to the robot simulation model on the illusion engine 4 through the robot communication module at regular time;

and S4, after receiving the angle feedback signal, the simulation model sequentially endows the angle values to the corresponding simulation modules, adjusts the pose of the simulation model according to the angle feedback signal so as to enable the pose of the simulation model to be consistent with the robot entity, and displays the current pose state of the robot through a display of the illusion engine 4 simulation system.

2. A phantom engine based method of controlling a robot according to claim 1, wherein the step of building a robot simulation model in the upper machine phantom engine 4 comprises:

s11, building a simulation model of the robot by adopting the illusion engine 4,

s12, making materials and maps for the simulation model, and making a scene model for the simulation model;

s13, making a visual robot graphical control interface by using the illusion engine 4, and binding different control instructions for each key of the graphical control interface; and if the key is clicked, sending a control command to the robot communication module through a Socket network TCP protocol.

3. A ghost engine based method of controlling a robot according to claim 1, wherein the robot comprises: the head joint module and N kinematic joint modules, wherein N is more than or equal to 2;

the head joint module is connected with the motion joint module through a bus, and is used for receiving a control command, analyzing data and sending the control command to the motion joint module through the bus;

and the motion joint module is used for controlling the steering engine to move according to the control command.

4. The method for controlling the robot based on the illusion engine of claim 3, wherein the head joint module comprises an STM32 series microprocessor, a WIFI module, an infrared sensor and a camera, the motion joint module comprises a steering engine microprocessor, a driving state feedback module and a steering engine, the WIFI module is connected with the robot communication module, the STM32 series microprocessor is connected with the steering engine microprocessor through a bus, the steering engine microprocessor is connected with the steering engine, and the STM32 series microprocessor is further connected with the driving state feedback module.

5. The illusion-engine-based method of controlling a robot of claim 4, wherein step S3 includes:

s31, after receiving the control command sent by the robot communication module, the WIFI module sends the control command to an STM32 series microprocessor of the head joint module through serial port communication;

s32, the STM32 series microprocessor analyzes the received control command and controls the motion joint module through a bus;

s33, after the motion joint module receives a control command from the bus, the microprocessor analyzes the control command, controls the steering engine to move to a corresponding angle, the robot makes a corresponding motion, and the driving state feedback module feeds the current actual rotation angle of the steering engine back to the STM32 series of microprocessors of the head joint module at regular time;

s34: the STM32 series microprocessors of the head joint module, after receiving the angle feedback of each kinematic joint module, will forward to the robot simulation model on the illusion engine 4 through the robot communication module.

6. A phantom engine based method of controlling a robot according to claim 2, wherein said simulation model makes a scene model of level ground, pipe, mountain or cable.

Images