RU2811845C1 - Automotive self-adjusting sensors system based on environmental recognition - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: technology for optimizing performance of sensors in an automatic vehicle control system. The effect is achieved through an automotive self-adjusting sensor system based on environmental recognition, containing an Around View Monitor (AVM) sensor, an AVM controller and an Advanced Driver Assistance System (ADAS) controller. The AVM sensor includes the camera, the position adjustment mechanism, and the position sensor. The camera receives information in the form of an image and transmits it to the AVM controller. The position adjustment mechanism receives the position adjustment command from the AVM controller and adjusts the position of the camera according to the position adjustment instruction. The position sensor sends response information about the camera position to the AVM controller. The AVM controller transmits information about recognized targets to the ADAS controller.

EFFECT: improved accuracy of scene recognition while reducing computational load.

10 cl, 7 dwg

Description

Technical field

The present invention relates to the field of optimizing the performance of sensors in an automatic driving system and, in particular, to a self-tuning sensor system.

State of the art

A conventional sensor system for automatic driving and parking of a car is not capable of recognizing multiple aspects of the environment to implement appropriate hardware and software adjustments in complex driving environments. The field of view (FOV) and region of interest (ROI) of the Around View Monitor (AVM) are fixed, as is the maximum survey range, resulting in the system being unable to survey a wider area in a high-speed environment, which may result in missing information. The parking environment and the high-speed driving environment have different effective inspection areas, and the conventional 360-degree monitor sensor cannot independently adapt to different environmental “scenes” to change its characteristics and mainly relates to FOV and ROI.

In the future, hardware processing capabilities will be actively expanded, it is also possible to refine the classification of detected targets, and the requirements for sensor integration will increase, but at present, various sensors have low efficiency of synergetic processing, in which one control system usually requires a corresponding sensor. Therefore, improving the performance of the AVM controller and achieving multifunctional sensor integration are necessary for a future automated driving system.

However, the technical solution in the relevant field of technology depends on hardware upgrade or software optimization, both of these aspects need to be comprehensively improved in the future, while increasing the number of monitors requires more costs, and increasing the number of sensors makes vehicle design more complex. Since the requirements of optimal sensor installation position and attractive appearance cannot be satisfied simultaneously, it may be difficult to use more sensors when designing a vehicle. Meanwhile, each of the leading automobile manufacturers desires to create visually attractive and discreet sensor layouts, making designs that accommodate more sensors on a vehicle difficult to achieve.

The essence of the invention

At least some embodiments of the present invention provide an environmental awareness-based automotive adaptive sensing system configured to adjust expected monitor positions to suit various environmental scenarios, optimizing ROI such that target detection is achieved at the sensor stage, and the results were directly transmitted to the ADAS driver assistance system controller while optimizing scene recognition accuracy and system computational load.

As an embodiment of the present invention, there is provided an automotive self-tuning (adaptive) sensing system based on environmental scene recognition, including an AVM (Around View Monitor) sensor, an AVM controller, and an ADAS controller.

The AVM sensor (1) includes the monitor, the position adjustment mechanism and the position sensor, the monitor receives visual information and transmits this information to the AVM controller, the position adjustment mechanism receives the position adjustment command from the AVM controller and adjusts the position of the monitor according to the position adjustment instruction , the position sensor transmits information about the position of the monitor to the AVM controller.

The AVM controller is connected to the ADAS controller and transmits information about the recognized target to it.

The AVM controller, in operation mode, generates information about the current scene environment according to the visual information input from the monitor, calculates the expected position of the monitor according to various scene data, generates a position adjustment command according to the expected position, and outputs an instruction to the position adjustment mechanism for position adjustment.

After adjusting the monitor position by the position adjustment mechanism, the AVM controller receives the updated position information and updated visual information from the monitor, performs ROI optimization based on the updated visual information to obtain the current ROI, performs target recognition in the current ROI to obtain information about the recognized targets, and outputs the information about recognized targets to the ADAS controller.

As an embodiment, when the vehicle is in low light conditions, when the vehicle is traveling at low speeds and low gears, when there are multiple obstacle targets or traffic lights or pedestrian obstacles, the position adjustment mechanism adjusts the lateral and longitudinal angles of the monitor to narrow the viewing range of the monitor and reducing the range of the current ROI. When the vehicle is in conditions of good lighting, driving at high speeds and high gears, and the presence of a small number of obstacle targets, the position adjustment mechanism rotates the vertical position of the monitor to expand the viewing range of the monitor and expand the range of the current ROI.

In one embodiment, the AVM controller includes a microcontroller unit (MCU) and a system on a chip (SOC), wherein the AVM sensor monitor interfaces with the SOC via Low Voltage Differential Signaling (LVDS) and provides visual information to the SOC, the MCU is coupled to the position adjustment mechanism, and position sensor via hard wiring, the MCU is also connected to the ADAS controller via a local controller network (CAN) or Ethernet.

In another embodiment, the AVM sensor monitor optionally provides visual information to the SOC 10, and the SOC 10 generates the current scene and outputs information about that scene to the MCU. The MCU calculates the expected position of the monitor according to various scene data, generates a position adjustment instruction according to the expected position of the monitor, and outputs the position adjustment instruction to the position adjustment mechanism to adjust the position of the monitor. Once the monitor position is adjusted, the position sensor returns updated monitor position information to the MCU, and the MCU returns updated position information to the SOC. The monitor inputs the updated visual information into the SOC, the SOC performs ROI optimization according to the updated position information and the updated video information of the monitor to obtain the current ROI, performs target recognition in the current ROI to obtain information about the recognized targets, and sends the recognized target information to the MCU. The MCU transmits information about recognized targets to the ADAS controller.

Another embodiment takes into account factors that influence scene formation, such as lighting, the number of obstacles, the speed and gear in which the vehicle is traveling, and obstacles.

In another version of the technical solution, the environment is classified as: entrance to a parking lot with dim lighting, narrow space for movement, numerous obstacle targets, low speed and transmission of vehicle motion, rails and fences; an expressway for travel at high speeds and gears with sufficient lighting, good visibility, wide passage space, few obstacles and cases of rapid vehicle convergence; a toll station with narrow traffic channels, restrictions on the throughput system, traffic at low speeds and gears, and numerous obstacles when passing through the toll station; and a city road with traffic at low speeds and gears, a complex environment, numerous traffic lights, vehicles and pedestrians, with a large number of obstacles of various types.

In an embodiment, ROI optimization may include the following: after adjusting the position of the monitor, the AVM controller receives the updated position information and updated video information of the monitor, recognizes the current scene, and generates an ROI corresponding to various scene data.

In one embodiment, target recognition includes the following: after dividing the ROI, target recognition is performed on images in the divided ROIs, and information about the recognized target is output to the ADAS controller.

In another embodiment of the invention, the AVM controller performs image processing and stitching in accordance with the video information input by the monitor, and generates the current scene in combination with a training model based on deep analysis.

As an option in the embodiment of the invention, the automotive self-tuning scene recognition sensor system includes four AVM sensors mounted on the front bumper, rear door, and under the left and right rear view mirrors, respectively.

At least some embodiments of the present invention have the following beneficial technical effects.

By optimizing scene recognition, the ROI is adapted to the current scene and the system's image processing is improved.

The AVM sensor performs target recognition in the image, after which the target and scene output can be transmitted to the ADAS controller, thus the performance of the image processing chip expected by the ADAS controller is reduced, while the power consumption of the ADAS system is reduced, and the configuration cost is greatly reduced.

Automotive self-adjusting sensor system based on situation recognition provides the ability to optimize sensor operation and energy consumption in the field of sensor automation of driving and parking and improve recognition accuracy.

Brief description of drawings

In fig. 1 is a schematic block diagram of an automotive self-tuning sensor system based on scene recognition as an embodiment of the present invention.

In fig. 2 is a schematic diagram of a control sequence of an Around View Monitor (AVM) controller as an embodiment of the present invention.

In fig. 3 is a flowchart of an image processing process in a system-on-chip (SOC) in accordance with an embodiment of the present invention.

In fig. 4 is a schematic depiction of a region of interest (ROI) of a high-speed environment scene before adjustment in accordance with an embodiment of the invention.

In fig. 5 schematically depicts the ROI of a high-speed environment scene after adjustment in accordance with an embodiment of the invention.

In fig. 6 schematically depicts the ROI of a low-speed environment scene before adjustment in accordance with an embodiment of the invention.

In fig. 7 is a schematic illustration of the ROI of a low-speed environment scene after adjustment in accordance with an embodiment of the invention.

In the drawings, the numbers indicate: 1 - AVM sensor, 2 - LVDS low-voltage differential signaling, 13 - AVM controller, 4 - CAN or Ethernet bus, 5 - ADAS controller, 6 - MCU microcontroller unit, 7 - SOC system-on-chip, 11 - mechanism position adjustment, 12 - position sensor.

Detailed description

For a better understanding of the solutions of the present invention by those skilled in the art, the design solutions in the embodiments of the present invention are clearly and in detail elaborated and set forth below with the accompanying drawings. It is obvious that the described design options are part of the embodiments of the present invention, but not all. If the embodiments of the present invention are taken as a basis, all other design solutions obtained by those skilled in the art without the application of creative efforts should be included within the scope of legal protection of the present invention.

It should be noted that terms such as “first” and “second” in the description, claims and accompanying drawings of the present invention are used to distinguish similar objects, but do not describe a specific order or sequence. It should be understood that objects can be exchanged under appropriate circumstances such that the embodiments of the present invention described herein may be implemented in a manner different from that described or shown herein. In addition, the terms “includes” and “has,” as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, for processes, methods, systems, products or devices containing a series of steps or elements, there is no restriction on clearly listing those steps or elements, but instead may include other steps or elements that are not clearly listed or are inherent in those processes, methods , products or devices.

As shown in FIG. 1, as an embodiment of the present invention, there is provided an automotive scene recognition-based plug-and-play sensing system including an Around View Monitor (AVM) sensor 1, an AVM controller 3, and an ADAS controller 5.

The AVM sensor 1 includes a monitor, a position adjustment mechanism 11 and a position sensor 12. The AVM controller 3 contains a microcontroller unit (MCU) 6 and a system on chip (SOC) 7.

The AVM sensor monitor is connected to the SOC 7 via Low Voltage Differential Signaling (LVDS2) and transmits video information to the SOC 7. The SOC 7 preprocesses the video information to produce a stitched image and generates a scene of the current environment through in-depth analysis. SOC 7 outputs current scene information to MCU 6.

MCU 6 is connected to the position adjustment mechanism 11 and the position sensor 12 via hard wiring. MCU 6 calculates the expected position of the monitor according to various scenes, generates a position adjustment command (such as an angle adjustment pulse width modulation (PWM) signal) according to the expected position of the monitor, and outputs the position adjustment instruction to the position adjustment mechanism 11 to control position sensor to adjust the position of the monitor. After adjusting the monitor position, position sensor 12 feeds back updated monitor position information to MCU 6, and MCU 6 feeds back updated position information to SOC 7. At the same time, the monitor inputs updated video information to SOC 7, SOC 7 performs ROI optimization according to updated position information and updated monitor video information to obtain the current ROI.

Thus, when the car is in a favorable environment of a scene such as an expressway, good road conditions with a wide view, the ROI expands; when the car is in an unfavorable scene such as a traffic jam, low light or tight space, the ROI narrows.

SOC 7 performs target recognition in the current ROI to obtain information about recognized targets, and sends information about recognized targets to MCU 6.

MCU 6 interfaces with Driver Assistance System (ADAS) controller 5 via a Controller Area Network (CAN) or Ethernet 4 and transmits information about recognized targets to the ADAS 5 controller.

In an embodiment, ROI optimization may include the following principles: After the monitor position is adjusted, the SOC 7 in the AVM controller 3 receives the updated position data and the updated video information of the monitor, recognizes the current scene, and generates an ROI region corresponding to the various scenes. That is, when the car is in an environment of expressway, good road conditions with a wide view, the ROI expands, and when it is in an unfavorable environment of traffic jams, low light or limited space, the ROI narrows.

In an embodiment, target recognition may include the following principles: after dividing the ROI, recognition of targets in images in the divided ROI is performed through deep analysis, and information about the recognized targets is output to the ADAS 5 controller.

In an embodiment, factors influencing scene formation include: lighting, number of obstacles, speeds and gears at which the vehicle is traveling, and specific obstacles.

In an embodiment, the scenes are classified as: dimly lit parking lot entrance, narrow traffic space, multiple obstacle targets, low speed and vehicle motion transmission, rails and guardrails; an expressway for travel at high speeds and gears with sufficient lighting, good visibility, wide passage space, few obstacles and cases of rapid vehicle convergence; a toll collection point with narrow traffic channels, restrictions on the throughput system, traffic at low speeds and gears, and numerous obstacles when passing through the toll point; and a city road with traffic at low speeds and gears, a complex environment, numerous traffic lights, vehicles and pedestrians, with a large number of obstacles of various types.

In an embodiment, factors affecting the system's scene recognition generally include lighting, number of obstacles, vehicle speeds, vehicle gears, specific obstacles, and the like. For example, when a car is in conditions of insufficient lighting, driving at low speed and low gear, and the presence of many specific obstacles (traffic lights and pedestrians), this means that the movement occurs in the transport environment of the city. At this moment, the position adjustment mechanism 11 must adjust the transverse and longitudinal angles of the monitor to reduce the field of view. Or, for example, when the vehicle is in conditions of good lighting, few obstacles, driving at high speed and high gear and no specific obstacles, it is considered to be driving on an expressway, and at this moment the vertical position of the monitor should be used to increase the viewing range sensor

In one embodiment, the automotive scene recognition-based self-tuning sensing system includes four AVM sensors 1 that are mounted on the front bumper, rear door, and under the left and right rear view mirrors, respectively.

As shown in FIG. 4 - 7, in a high-speed environment scene, after adjusting the monitor, a wider working field of view can be obtained.

The shaded whitish part of the frame in Fig. 4 represents the ROI before optimization, the shaded whitish part in FIG. 5 displays the ROI after optimization based on a high-speed scene, from which it can be seen that more video information can be extracted through ROI optimization.

The shaded whitish area in Fig. 6 represents the ROI before optimization, the shaded whitish area in FIG. 7 shows the ROI after optimization, from which it can be seen that in a narrow and low-speed scene, the ROI area is enlarged, while the actual ROI where the car itself is located is removed so that more visual information can be extracted.

The target object information obtained through SOC 7 processing can be applied to the ADAS 5 controller, and can replace the functions of image processing chips in the ADAS control system, and the hardware cost and power consumption of the ADAS system can be effectively reduced.

The serial numbers of the embodiments of the present invention are used for description and do not reflect the advantages or disadvantages of technical solutions.

The descriptions of the above embodiments of the present invention focus on various aspects. A component that is not described in detail in one embodiment may be relevantly described in other embodiments.

It should be understood that in several embodiments presented in the application, the disclosed technical content may be implemented in other ways. The description of the above examples of design solutions for the device is schematic; thus, the separation of elements represents the separation of logical functions, while in actual implementation other methods of separation may be used, for example, several elements or components may be combined or integrated into another system, or some functions may be neglected or not performed. In addition, the connection, direct connection or communication connection shown or described may be implemented by indirect connection or communication connection of certain interfaces, elements or components and may be in electrical or other forms.

Items described as separate parts may or may not be physically separable. The part shown as an element may or may not be a physical element, that is, it may be located in a specific location, or it may be distributed across multiple network elements. All or part of the elements may be selected to achieve technical solutions of the embodiments in accordance with practical requirements.

Moreover, all functional elements in embodiments of the present invention may be integrated in a processing element; or the elements may exist separately and physically; or at least two elements may be integrated into one element. The integrated element may be implemented as hardware or as a functional software element.

When such an integrated element is implemented by software functional components, and the software functional components are sold or used as independent products, they may also be stored on a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention in whole or in part, which is a contribution to the prior art, can be embodied in the form of a software product; the computer program product is stored in a storage medium and includes a series of instructions for a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method in each embodiment of the present invention. Storage media include, but are not limited to, USB flash drive, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, CD-ROM, and other media capable of storing program codes.

The above are representative embodiments of the present invention; It should be noted that, without departing from the principles of the present invention, a number of improvements and additions may also be made by those skilled in the art, and these improvements and additions shall fall within the scope of protection of the present invention.

Claims (10)

1. An automotive self-tuning sensor system based on environmental awareness, comprising an Around View Monitor (AVM) sensor (1), an AVM controller (3) and a Driver Assistance System (ADAS) controller (5), characterized in that the AVM sensor (1) contains a camera, a position adjustment mechanism (11) and a position sensor (12); in this case, the camera is configured to receive information in the form of an image and transmit it to the AVM controller (3); the position adjustment mechanism (11) is configured to receive a position adjustment command from the AVM controller (3) and adjust the camera position in accordance with the position adjustment instructions; the position sensor is configured to transmit information about the camera position to the AVM controller (3); the AVM controller (3) is interfaced with the ADAS controller (5) and transmits recognized information about targets to the ADAS controller (5); The AVM controller (3) in operating mode generates the current environment data according to the image information received from the camera, calculates the expected position of the camera according to various environmental data, generates a position adjustment command according to the expected position, and outputs the adjustment instruction position on the position adjustment mechanism (11); and after adjusting the camera position by the position adjustment mechanism (11), the AVM controller (3) receives the updated position information and the updated camera, performs region of interest (ROI) optimization based on the updated information in the form of images to form the current region of interest, performs target recognition in the current areas of interest to obtain information about recognized targets and outputs information about recognized targets to the ADAS controller (5), and when the vehicle is in an environment with good lighting conditions, a small number of obstacle targets, a wide view, moving at high speed and high gear, ROI is expanding; and when the vehicle is in a congested, poorly lit or narrow area, the ROI is reduced. 2. Automotive self-adjusting sensor system according to claim 1, characterized in that when the car is in conditions of poor lighting, driving at low speed and low gear, the presence of numerous obstacle targets or obstacles in the form of traffic lights and pedestrians, the position adjustment mechanism is made adjustable transverse and longitudinal camera angles to reduce the camera's field of view and reduce the range of the current ROI; When the car is in conditions of good lighting, driving at high speed and high gear, and the presence of few obstacle targets, the position adjustment mechanism is designed to increase the vertical position of the camera to expand the camera's field of view and expand the range of the current ROI area. 3. Automotive self-tuning sensor system according to claim 2, characterized in that the AVM controller contains a microcontroller unit (MCU) (6) and a system on a chip (SOC) (7); The AVM sensor camera interfaces with the SOC (7) via Low Voltage Differential Signaling (LVDS) (2) to transmit information in the form of images to the SOC (7); The MCU (6) is connected to the position adjustment mechanism and the position sensor via hard wiring; The MCU (6) is also connected to the ADAS controller (5) via a local controller network (CAN) or Ethernet (4). 4. Automotive self-adjusting sensor system according to claim 3, characterized in that the AVM sensor camera (1) is configured to transmit information in the form of images to the SOC (7), and the SOC (7) determines the current environment and displays data about it on MCU(6)); The MCU (6) calculates the expected position of the camera according to various situation data, generates a position adjustment instruction according to the expected position of the camera, and outputs the position adjustment instruction to the position adjustment mechanism to adjust the camera position; after adjusting the camera position, the position sensor (12) returns the updated camera position information to the MCU (6), and the MCU (6) returns the updated position information to the SOC (7); The camera inputs the updated information in the form of images into the SOC (7), the SOC (7) performs ROI optimization according to the updated position information and the updated information in the form of images, the camera performs target recognition in the current ROI to generate the information to form the current ROI about recognized targets and sending information about recognized targets to the MCU (6); and the MCU (6) transmits the recognized target information to the ADAS controller (5). 5. Automotive self-adjusting sensor system according to claim 4, characterized in that it is designed to take into account factors that influence the determination of the environment, including lighting, the number of obstacles, vehicle speed, vehicle gear and obstacles. 6. Automotive self-adjusting sensor system according to claim 5, characterized in that it is designed to classify the environment: entrance to a parking lot with dim lighting, narrow space for movement, numerous obstacle targets, low speed and transmission of vehicle motion, rails and fences ; as an expressway for driving at high speeds and gears with sufficient lighting, good visibility, wide passage space, few obstacles and cases of rapid vehicle approach; as a toll station with narrow traffic channels, restrictions on the throughput system, traffic at low speeds and gears, and numerous obstacles when passing through the toll station; and as an urban road with traffic at low speeds and gears, a complex environment, numerous traffic lights, vehicles and pedestrians, and a large number of different types of obstacles. 7. Automotive self-adjusting sensor system according to claim 6, characterized in that ORI optimization includes the following: after adjusting the camera position, the AVM controller (3) receives updated position information and updated information in the form of camera images, recognizes the current scene and generates an ROI corresponding various scene data. 8. Automotive self-adjusting sensor system according to claim 7, characterized in that environmental recognition includes the following: after dividing the ROI, performing target recognition on images in the divided ROI through deep analysis and outputting the recognized target information to the ADAS controller (5). 9. The automotive self-tuning sensor system according to claim 8, characterized in that the AVM controller (5) performs image processing and stitching in accordance with image information input by the camera, and generates a current scene in combination with a training model based on in-depth analysis. 10. Automotive self-adjusting sensor system according to claim 9, characterized in that it includes four AVM sensors (1) installed, respectively, on the front bumper, on the rear door and under the left and right rear view mirrors.