CN115035357A - Object detection model construction method, object detection method, device and computing device - Google Patents

Abstract

The disclosure provides a construction method of a target detection model, a target detection method, a target detection device and computing equipment, which are used for solving the problem that a single-stage target detector in the existing scheme has poor detection performance on a small object. The construction method of the target detection model comprises the following steps: constructing a feature extraction network, wherein the feature extraction network is used for carrying out feature extraction on an input picture to obtain a multilayer feature map, and the multilayer feature map comprises a first feature map and a second feature map; constructing a target detection network, wherein the target detection network comprises a plurality of network layers corresponding to the multilayer characteristic diagram, and the plurality of network layers comprise a first network layer and a second network layer; the first network layer is used for carrying out query operation on the first feature map and transmitting the obtained query result to the second network layer, and the query result comprises a query point of a specific target in the first feature map; the second network layer is used for determining a mapping area of the query point in the second feature map, and performing detection operation in the mapping area to obtain a detection result.

Description

Object detection model construction method, object detection method, device and computing device

technical field

The present disclosure relates to the field of target detection, and in particular, to a method for constructing a target detection model, a target detection method, an apparatus and a computing device.

Background technique

Object detectors based on deep neural networks have achieved great success in recent years. Common object detectors can be divided into single-stage detectors and two-stage detectors. Among them, the single-stage detector has the advantages of simple structure and fast inference speed. For example, the anchor-based RetinaNet object detector pre-defines some candidate boxes of specific size and shape in the picture, and then uses the neural network to classify and regress these candidate boxes to perform object detection. However, the detection effect of the existing single-stage object detectors for small objects is not satisfactory. In order to improve the performance of small object detection, high-resolution input images and features are usually used, but this will bring huge the amount of computation, which seriously slows down the inference speed. Therefore, it is necessary to provide a more efficient and fast target detection method.

SUMMARY OF THE INVENTION

Embodiments of the present disclosure provide a method for constructing a target detection model, a method for target detection, an apparatus, and a computing device, so as to improve the detection effect rate of a single-stage target detector for small objects.

In order to achieve the above object, the embodiments of the present disclosure adopt the following technical solutions:

A first aspect of the embodiments of the present disclosure provides a feature extraction network for constructing a feature extraction network for performing feature extraction on an input image to obtain multi-layer feature maps with different sizes, where the multi-layer feature maps include a first feature map and a second feature map. Two feature maps; construct a target detection network, the target detection network includes multiple network layers corresponding to the multi-layer feature maps, and the multiple network layers include a first network layer and a second network layer; the first network layer corresponds to the first feature map , which is used to query the first feature map, and transmit the obtained query result to the second network layer, where the query result includes the query point of a specific target in the first feature map; the second network layer and the second feature map The corresponding map is used to determine the mapping area of the query point in the second feature map, and perform a detection operation in the mapping area to obtain a detection result.

A second aspect of the embodiments of the present disclosure provides a target detection method, including: inputting a to-be-detected picture into a target detection model, where the target detection model includes a feature extraction network and a target detection network; using the feature extraction network to characterize the to-be-detected picture Extraction to obtain multi-layer feature maps with different sizes, the multi-layer feature maps include a first feature map and a second feature map; and a target detection network is used to output the detection results of the multi-layer feature maps, and the target detection network includes and multi-layer features. The multiple network layers corresponding to the map, the multiple network layers include a first network layer and a second network layer; wherein, the first network layer corresponds to the first feature map, and is used to perform a query operation on the first feature map, and The obtained query result is transmitted to the second network layer, and the query result includes the query point of the specific target in the first feature map; the second network layer corresponds to the second feature map and is used to determine the query point in the second feature map. Map the area, and perform the detection operation in the mapped area to obtain the detection result.

In a third aspect of the embodiments of the present disclosure, there is provided an apparatus for constructing a target detection model, including: a first constructing unit for constructing a feature extraction network, and the feature extraction network is configured to perform feature extraction on an input picture to obtain images with different sizes. A multi-layer feature map, the multi-layer feature map includes a first feature map and a second feature map; the second construction unit is used to construct a target detection network, and the target detection network includes multiple network layers corresponding to the multi-layer feature maps. The network layer includes a first network layer and a second network layer; the first network layer corresponds to the first feature map, and is used to perform a query operation on the first feature map, and transmit the obtained query result to the second network layer. The result includes the query point of the specific target in the first feature map; the second network layer corresponds to the second feature map, and is used to determine the mapping area of the query point in the second feature map, and perform a detection operation in the mapped area to obtain the detection result.

A fourth aspect of the embodiments of the present disclosure provides a target detection device, comprising: an input unit for inputting a picture to be detected into a target detection model, where the target detection model includes a feature extraction network and a target detection network; a feature extraction unit, It is used for extracting features from the image to be detected by using a feature extraction network to obtain multi-layer feature maps with different sizes, and the multi-layer feature maps include a first feature map and a second feature map; and a target detection unit, used for using the target detection network output For the detection results of the multi-layer feature maps, the target detection network includes multiple network layers corresponding to the multi-layer feature maps, and the multiple network layers include a first network layer and a second network layer; wherein, the first network layer and the first network layer. Corresponding to a feature map, it is used to query the first feature map, and transmit the obtained query result to the second network layer, where the query result includes the query point of a specific target in the first feature map; The two feature maps correspond to, and are used to determine the mapping area of the query point in the second feature map, and perform a detection operation in the mapped area to obtain a detection result.

In a fifth aspect of the embodiments of the present disclosure, a computing device is provided, including: a processor, a memory, and a computer program stored in the memory and executable on the processor; wherein, when the processor runs the computer program, the processor executes the above The method for constructing the target detection model and/or the method for target detection.

In a sixth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided on which a computer program is stored, and when the computer program is run by a processor, the above-described method and/or target for constructing a target detection model are implemented Detection method.

A seventh aspect of the embodiments of the present disclosure provides a vehicle, including the computing device as described above.

According to the technical solution of the present disclosure, a novel single-stage object detector based on a query mechanism is proposed, which can quickly detect small objects on high-resolution features. The basic idea is to first find the approximate position of the small object on the low-resolution feature map, that is, the query point of a specific target, so as to map and detect in the high-resolution feature map according to the query point. In the present disclosure, a query network may also be added to the detection head, and the query network is used to determine whether there is an object whose size is smaller than the size threshold of the layer at a certain position. At the same time, the present disclosure can also cooperate with sparse convolution to improve the inference speed, construct a sparse feature tensor with high-resolution features based on these query points, and use sparse convolution to calculate the result.

Description of drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only These are some embodiments of the present disclosure, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

FIG. 1 is a structural diagram of a vehicle 100 according to an embodiment of the present disclosure;

FIG. 2 is a flowchart of a method 200 for constructing a target detection model according to an embodiment of the present disclosure;

3 is a schematic diagram of a target detection model provided by an embodiment of the present disclosure;

4 is a schematic diagram of another target detection model provided by an embodiment of the present disclosure;

FIG. 5 is a flowchart of another target detection method 500 provided by an embodiment of the present disclosure;

6 is a schematic diagram of a detection effect of a target detection method provided by an embodiment of the present disclosure;

FIG. 7 is a structural diagram of an apparatus 700 for constructing a target detection model according to an embodiment of the present disclosure;

FIG. 8 is a structural diagram of another target detection model apparatus 800 provided by an embodiment of the present disclosure;

FIG. 9 is a structural diagram of a computing device 900 according to an embodiment of the present disclosure.

Detailed ways

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, but not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

It should be noted that the terms "first", "second" and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances for the embodiments of the present disclosure described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

FIG. 1 is a schematic diagram of a vehicle 100 in which the various techniques disclosed herein may be implemented. Vehicle 100 may be a car, truck, motorcycle, bus, boat, airplane, helicopter, lawn mower, backhoe, snowmobile, aircraft, touring vehicle, amusement park vehicle, farm installation, construction installation, streetcar , golf cart, train, trolleybus, or other vehicle. The vehicle 100 may operate fully or partially in an autonomous driving mode. The vehicle 100 may control itself in an autonomous driving mode, for example, the vehicle 100 may determine the current state of the vehicle and the current state of the environment in which the vehicle is located, determine the predicted behavior of at least one other vehicle in the environment, determine the at least one other vehicle The confidence level corresponds to the likelihood of performing the predicted behavior, and the vehicle 100 itself is controlled based on the determined information. When in the autonomous driving mode, the vehicle 100 may operate without human interaction.

Vehicle 100 may include various vehicle systems, such as drive system 142 , sensor system 144 , control system 146 , user interface system 148 , control computer system 150 , and communication system 152 . Vehicle 100 may include more or fewer systems, and each system may include multiple units. Further, each system and unit of the vehicle 100 may be interconnected. For example, the control computer system 150 can be in data communication with one or more of the vehicle systems 142 - 148 and 152 . Thus, one or more of the described functions of the vehicle 100 may be divided into additional functional or physical components, or combined into a smaller number of functional or physical components. In a further example, additional functional components or physical components may be added to the example shown in FIG. 1 .

The drive system 142 may include a number of operable components (or units) that provide kinetic energy to the vehicle 100 . In one embodiment, the drive system 142 may include an engine or electric motor, wheels, a transmission, electronic systems, and power (or power source). The engine or electric motor may be any combination of an internal combustion engine, an electric motor, a steam engine, a fuel cell engine, a propane engine, or other forms of engine or electric motor. In some embodiments, the engine may convert a power source into mechanical energy. In some embodiments, the drive system 142 may include various motors or electric motors. For example, a gasoline-electric hybrid vehicle may include a gasoline engine and an electric motor, among other things.

The wheels of the vehicle 100 may be standard wheels. The wheels of the vehicle 100 may be various types of wheels, including single-wheel, double-wheel, three-wheel, or four-wheel forms, such as those on a car or truck. Other numbers of wheels are possible, such as six or more wheels. One or more wheels of the vehicle 100 may be manipulated to rotate in a different direction than the other wheels. The wheel may be at least one wheel that is fixedly connected to the transmission. Wheels can include a combination of metal and rubber, or a combination of other substances. The transmission may include a unit operable to transmit the mechanical power of the engine to the wheels. For this purpose, the transmission may include a gearbox, a clutch, a differential gear and a drive shaft. The transmission may also include other units. The driveshaft may include one or more axles that mate with the wheels. The electronic system may include a unit for transmitting or controlling electronic signals of the vehicle 100 . These electronic signals may be used to activate lights, servos, motors, and other electronic drive or control devices in the vehicle 100 . The power source may be an energy source that powers the engine or electric motor in whole or in part. That is, the engine or electric motor can convert the power source into mechanical energy. Illustratively, the power source may include gasoline, petroleum, petroleum-based fuels, propane, other compressed gas fuels, ethanol, fuel cells, solar panels, batteries, and other electrical energy sources. Power sources may additionally or alternatively include fuel tanks, batteries, capacitors, or any combination of flywheels. The power source may also power other systems of the vehicle 100 .

The sensor system 144 may include a plurality of sensors for sensing information about the environment and conditions of the vehicle 100 . For example, sensor system 144 may include an inertial measurement unit (IMU), a global positioning system (GPS) transceiver, a radar (RADAR) unit, a laser rangefinder/LIDAR unit (or other distance measurement device), acoustic sensors, and cameras or image capture device. The sensor system 144 may include a number of sensors for monitoring the vehicle 100 (eg, an oxygen (O 2 ) monitor, a fuel gauge sensor, an engine oil pressure sensor, etc.). Other sensors can also be configured. One or more sensors included in sensor system 144 may be driven individually or collectively to update the position, orientation, or both of the one or more sensors.

The IMU may include a combination of sensors (eg, accelerometers and gyroscopes) for sensing changes in position and orientation of the vehicle 100 based on inertial acceleration. The GPS transceiver may be any sensor used to estimate the geographic location of the vehicle 100 . For this purpose, the GPS transceiver may include a receiver/transmitter to provide position information of the vehicle 100 relative to the earth. It should be noted that GPS is an example of a global navigation satellite system, therefore, in some embodiments, the GPS transceiver may be replaced by a Beidou satellite navigation system transceiver or a Galileo satellite navigation system transceiver. The radar unit may use radio signals to sense objects in the environment in which the vehicle 100 is located. In some embodiments, in addition to sensing objects, the radar unit may also be used to sense the speed and heading of objects approaching the vehicle 100 . A laser rangefinder or LIDAR unit (or other distance measuring device) may be any sensor that uses a laser to sense objects in the environment in which the vehicle 100 is located. In one embodiment, a laser rangefinder/LIDAR unit may include a laser source, a laser scanner, and a detector. Laser rangefinder/LIDAR units are designed to operate in continuous (eg using heterodyne detection) or discontinuous detection modes. The camera may include means for capturing multiple images of the environment in which the vehicle 100 is located. The camera may be a still image camera or a motion video camera.

The control system 146 is used to control the operation of the vehicle 100 and its components (or units). Accordingly, the control system 146 may include various units, such as a steering unit, a power control unit, a braking unit, and a navigation unit.

The steering unit may be a combination of machinery that adjusts the direction of travel of the vehicle 100 . A power control unit (which may be an accelerator, for example) may be used to control the running speed of the engine, and thus the speed of the vehicle 100 , for example. The braking unit may include a combination of mechanisms for decelerating the vehicle 100 . The braking unit can use friction to decelerate the vehicle in a standard manner. In other embodiments, the braking unit may convert the kinetic energy of the wheels into electrical current. The braking unit may also take other forms. The navigation unit may be any system that determines a driving path or route for the vehicle 100 . The navigation unit can also dynamically update the driving path as the vehicle 100 travels. The control system 146 may additionally or alternatively include other components (or units) not shown or described.

User interface system 148 may be used to allow interaction between vehicle 100 and external sensors, other vehicles, other computer systems, and/or a user of vehicle 100 . For example, user interface system 148 may include a standard visual display device (eg, a plasma display, liquid crystal display (LCD), touch screen display, head-mounted display, or other similar display), speakers or other audio output devices, microphones or other audio input device. For example, the user interface system 148 may also include a navigation interface and an interface to control the interior environment of the vehicle 100 (eg, temperature, fans, etc.).

)。在一些实施例中，通信系统152可以直接与一个或多个设备或者周围其它车辆进行通信，例如，使用红外线，

或者ZIGBEE。其它无线协议，例如各种车载通信系统，也在本申请公开的范围之内。例如，通信系统可以包括一个或多个专用短程通信(DSRC)装置、V2V装置或者V2X装置，这些装置会与车辆和/或路边站进行公开或私密的数据通信。The communication system 152 may provide a means for the vehicle 100 to communicate with one or more devices or other surrounding vehicles. In an exemplary embodiment, the communication system 152 may communicate with one or more devices directly or through a communication network. Communication system 152 may be, for example, a wireless communication system. For example, the communication system may use 3G cellular communications (eg, CDMA, EVDO, GSM/GPRS) or 4G cellular communications (eg, WiMAX or LTE), and may also use 5G cellular communications. Alternatively, the communication system may communicate with a wireless local area network (WLAN) (eg, using

). In some embodiments, the communication system 152 may communicate directly with one or more devices or other surrounding vehicles, eg, using infrared,

Or ZIGBEE. Other wireless protocols, such as various in-vehicle communication systems, are also within the scope of this disclosure. For example, the communication system may include one or more Dedicated Short Range Communication (DSRC) devices, V2V devices, or V2X devices that may conduct public or private data communications with vehicles and/or roadside stations.

The control computer system 150 can control some or all of the functions of the vehicle 100 . The autonomous driving control unit in the control computer system 150 may be used to identify, evaluate, and avoid or overcome potential obstacles in the environment in which the vehicle 100 is located. Typically, an automated driving control unit may be used to control the vehicle 100 without a driver, or to provide assistance for the driver to control the vehicle. In some embodiments, the autonomous driving control unit is used to combine data from GPS transceivers, radar data, LIDAR data, camera data, and data from other vehicle systems to determine the travel path or trajectory of the vehicle 100 . The autopilot control unit may be activated to enable the vehicle 100 to be driven in an autopilot mode.

Control computer system 150 may include at least one processor (which may include at least one microprocessor) that executes processing instructions (ie, machine-readable) stored in a non-volatile computer-readable medium (eg, data storage or memory). execute command). The memory stores at least one machine-executable instruction, and the processor executes the at least one machine-executable instruction to implement functions including a map engine, a positioning module, a perception module, a navigation or path module, and an automatic control module. The map engine and positioning module are used to provide map information and positioning information. The perception module is used to perceive things in the environment where the vehicle is located according to the information obtained by the sensor system and the map information provided by the map engine. The navigation or route module is used to plan the driving route for the vehicle according to the processing results of the map engine, the positioning module and the perception module. The automatic control module converts the decision information input and analysis of modules such as navigation or route modules into control command output for the vehicle control system, and implements the vehicle through the in-vehicle network (for example, through CAN bus, local area interconnection network, multimedia directional system transmission, etc.). The internal electronic network system) sends control commands to the corresponding components in the vehicle control system to realize automatic control of the vehicle; the automatic control module can also obtain the information of each component in the vehicle through the vehicle network.

Control computer system 150 may also be a plurality of computing devices that control components or systems of vehicle 100 in a distributed manner. In some embodiments, the memory may contain processing instructions (eg, program logic) that are executed by the processor to implement various functions of the vehicle 100 . In one embodiment, control computer system 150 is capable of data communication with systems 142 , 144 , 146 , 148 and/or 152 . Interfaces in the control computer system are used to facilitate data communication between control computer system 150 and systems 142 , 144 , 146 , 148 , and 152 .

The memory may also include other instructions, including instructions for data transmission, instructions for data reception, instructions for interaction, or instructions for controlling drive system 140 , sensor system 144 , or control system 146 or user interface system 148 instruction.

In addition to storing processing instructions, the memory may store various information or data, such as image processing parameters, road maps, and route information. This information may be used by the vehicle 100 and the control computer system 150 during operation of the vehicle 100 in automatic, semi-automatic, and/or manual modes.

Although the autopilot control unit is shown as being separate from the processor and memory, it should be understood that in some embodiments some or all of the autopilot control unit's functionality may utilize memory (or data storage devices) residing in one or more of the ) and executed by one or more processors, and the autonomous driving control unit may in some cases be implemented using the same processors and/or memory (or data storage). In some embodiments, the autonomous driving control unit may use, at least in part, various special purpose circuit logic, various processors, various field programmable gate arrays ("FPGA"), various application specific integrated circuits ("ASIC"), Various real-time controllers and hardware are implemented.

Control computer system 150 may control the functions of vehicle 100 based on input received from various vehicle systems (eg, drive system 142 , sensor system 144 , and control system 146 ), or input received from user interface system 148 . For example, control computer system 150 may use input from control system 146 to control the steering unit to avoid obstacles detected by sensor system 144 . In one embodiment, the control computer system 150 may be used to control various aspects of the vehicle 100 and its systems.

Although various components (or units) are shown integrated into the vehicle 100 in FIG. 1 , one or more of these components (or units) may be onboard or individually associated with the vehicle 100 . For example, the control computer system may exist partially or fully independent of the vehicle 100 . Thus, the vehicle 100 can exist in the form of separate or integrated equipment units. The equipment units constituting the vehicle 105 can communicate with each other in the form of wired communication or wireless communication. In some embodiments, additional components or units may be added to the various systems or one or more of the above components or units (eg, the LiDAR or radar shown in FIG. 1 ) may be added or removed from the system.

As mentioned above, existing single-stage object detectors have poor performance for small object detection. Although the introduction of the feature pyramid has greatly improved the performance of small objects, the results are still unsatisfactory: the highest resolution feature of the usual feature pyramid is 1/8 of the input resolution, but for very small objects this is still is not enough. Although the introduction of higher resolution features can further alleviate this problem, it will bring huge computational load. For example, in RetinaNet, if a feature with a resolution of 1/4 of the input resolution is introduced, the computational load on the detection head will be would be 300% more, which would seriously slow down inference. To this end, the embodiments of the present disclosure aim to propose a more efficient target detection solution.

As shown in FIG. 2 , a method 200 for constructing a target detection model provided by an embodiment of the present disclosure includes:

Step S201 , constructing a feature extraction network, the feature extraction network is used to perform feature extraction on the input picture to obtain multi-layer feature maps with different sizes, and the multi-layer feature maps include a first feature map and a second feature map.

Step S202 , constructing a target detection network, where the target detection network includes multiple network layers corresponding to the multi-layer feature maps, and the multiple network layers include a first network layer and a second network layer.

The first network layer corresponds to the first feature map, and is used to perform a query operation on the first feature map, and transmit the obtained query result to the second network layer, where the query result includes the specific information in the first feature map. The query point for the target. The second network layer corresponds to the second feature map, and is used to determine the mapping area of the query point in the second feature map, and perform a detection operation in the mapped area to obtain a detection result.

In order for those skilled in the art to better understand the present disclosure, the embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings and examples.

In an embodiment of the present disclosure, the target detection model, as shown in FIG. 3 and FIG. 4 , includes a feature extraction network and a target detection network. Among them, the feature extraction network includes a backbone network and a feature pyramid network.

The backbone network is used to perform feature extraction on the input image to obtain the initial multi-layer feature map. That is, the backbone network obtains the initial multi-layer feature map by down-sampling the input image. The feature maps obtained by the downsampling operation have a corresponding scaling relationship. As shown in Figure 3, the feature map of the P2 layer in the backbone network layer is 1/4 of the resolution of the input image, and the feature map of the P3 layer is 1/4 of the feature map of the P2 layer. 2 resolution.

The feature pyramid network is used to upsample and fuse the initial feature map to obtain an improved multi-layer feature map. The feature maps in the feature pyramid network as shown in Figure 3 are obtained based on the fusion of feature maps in the backbone network, and the resulting improved multi-layer feature maps still have a corresponding scaling relationship. Here, a first feature map is set, a plurality of additional feature maps are obtained by downsampling from the first feature map, up-sampling is performed from the first feature map, and fused with the feature maps of the same level in the backbone network, and sequentially Obtain multiple high-resolution feature maps. In Figure 3, layers P7 and P8 in the feature pyramid network are additional feature maps for traditional target detection, and layers P1-P6 are query feature maps for target detection based on query operations. The target is a target object in the image data, which may include static objects and dynamic objects, such as pedestrians, vehicles, animals, obstacles, signal lights, road signs, and the like. Target detection is to find the position of the target object in the raw sensor data through an algorithm. Generally, a rectangle or cuboid is used to represent the position occupied by the object in 2D or 3D space.

It should be noted that the present disclosure can perform target detection on the initial multi-layer feature map obtained by the backbone network, and can also perform target detection on the improved multi-layer feature map obtained by the feature pyramid network. The source of the feature map is not limited.

Each layer of feature maps has a corresponding size threshold, such as the minimum target size that can be detected by each layer of feature maps, or the target size range that can be detected by each layer of feature maps. For feature maps with anchors, the size threshold includes the minimum anchor size and/or the maximum anchor size of the feature map of this layer. In this way, the feature maps of different layers output targets of different sizes, and the output targets of different sizes can be combined to obtain the target detection result of the input image.

The target detection network includes a plurality of network layers, each network layer corresponds to a layer of feature maps, the additional network layer corresponds to the additional feature map, and the query layer corresponds to the query feature map. The object detection network in Figure 3 includes two additional network layers and four query layers.

The additional network layer is a traditional network layer, which may also be called a non-query layer, and is used to perform detection operations on the corresponding additional feature maps to obtain detection results. The detection network of the additional network layer includes a classification network and/or a regression network. The classification network and the regression network are respectively used to perform the classification detection and regression detection of the target on the corresponding additional feature map, so as to obtain the classification result and the regression result. The classification result is, for example, Classification category, the regression result is, for example, the regression box.

The query layer is a network layer based on query operations. Each network layer in the query layer can implement any one or more of the following functions: Detect query points of specific targets in the feature map of this layer, and transmit the detected query points. To the low-level network layer; receive the query point transmitted by the high-level network layer, and map the query point to the mapping area in the feature map of the layer. Here, the layer where the low-resolution feature map is located is the high-level layer, and the layer where the high-resolution feature map is located is the low-level layer. As shown in Figure 3, the level of the P2 layer is lower than that of the P3 layer. Therefore, the resolution of the multi-level feature map of the present disclosure increases from high level to low level, and there is a corresponding proportional relationship between the upper and lower level feature maps.

According to one embodiment, the query layer includes a first network layer corresponding to the first feature map, a second network layer corresponding to the second feature map, a third network layer corresponding to the third feature map, ..., and the i-th network layer. The i-th network layer corresponding to the network layer, ..., the n-th network layer corresponding to the n-th feature map.

The first network layer is the starting layer of the query layer, and is used to perform a query operation on the first feature map to obtain a query result. The query results of the query layer include query points corresponding to specific targets in the feature map of the layer. For example, the query result of the first network layer includes query points of a specific target in the first feature map. Optionally, the query result includes a query result graph, the query result graph of a certain network layer has the same size as the feature graph corresponding to the network layer, and the query result graph includes the probability that a specific target exists at each position point in the corresponding feature graph. Correspondingly, the query point is a position point whose probability value is greater than or equal to the preset threshold in the feature map. The preset threshold can be set according to needs, for example, it is set to 0.5, which is not limited in the present disclosure.

The query operation may be performed by a query network, which is also a type of detection network (ie, detection head). The specific target is the target whose size is less than or equal to the size threshold of the corresponding layer feature map. Taking the first feature map as an example, assuming that the minimum size threshold of the first feature map is a, the specific target is a target whose size is less than or equal to a. Of course, the specific target can also be a target whose size is less than or equal to a predetermined multiple b (b>1) of the size threshold of the feature map of the layer. In this case, the specific target of the first feature map is a target whose size is less than or equal to a*b. The present disclosure sets such an expansion interval for the specific target size of the feature map of each layer, which is conducive to generating a larger range of query points and improving the detection accuracy of the lower network layer.

The query point may be the position point contained in the feature target, or the position point within the detection frame where the specific target is located, or the point within the preset range of the center point of the specific target. The range of query points is not specifically limited. The query point can be represented by coordinates, that is, the coordinate position of the query point in the first feature map.

The first network layer transmits the query result to the second network layer, and the second network layer extracts the query point in the transmission result, maps the query point to a plurality of mapping points in the first feature map, and obtains a mapping area. Generally, the second feature map is obtained by performing up-sampling and feature fusion on the first feature map, and the size of the second feature map is m times that of the first feature map, and m>1. Therefore, the second network layer maps the query point to a plurality of points in the second feature map according to the magnification m between the first feature map and the second feature map to obtain a mapped region. If the second feature map is twice as large as the first feature map, the second network layer maps each received query point to four adjacent points in the second feature map according to the coordinate mapping relationship between images; The feature map is 3 times larger than the first feature map, then the second network layer maps each received query point to the eight adjacent points in the second feature map, and so on. Here, the mapping operation of the query point by the second network layer may be performed by the mapping module. A query operation is performed on the mapping area in the second feature map to obtain a corresponding query result.

The obtained query result is transmitted to the third network layer, so that the third network layer maps the query point in the query result to the mapping area in the corresponding third feature map, and performs the detection operation in the mapping area to obtain Test results.

Afterwards, the second network layer performs a detection operation on the determined mapping area, and the detection operation is also performed by a detection network, such as a classification network and/or a regression network, to perform classification detection and regression detection, respectively. Generally, if a certain network layer receives the query result transmitted from the high-level network layer, the network layer usually has a mapping module and a detection network, wherein the mapping module is used to map the query points in the query result, and the detection module is used to map the query points in the query result. Detect the mapped area. Similarly, if a network layer performs a query operation, the query result will be passed to the lower layer.

Here, the first network layer mainly includes a query network for performing a query operation, and the second network layer mainly includes a detection network for performing a detection operation. Optionally, the first network layer may also include a detection network for performing a detection operation on the first feature map to obtain a detection result. The second network layer may also include a query network for performing a query operation on the corresponding second feature map, and transmitting the query query to the third network layer. The query result output by each layer of the network layer is the query point of the specific target in the feature map of this layer. At this time, the third network layer must have a mapping module and a detection network, so as to map the query point received from the second network layer into a query area, and perform detection operations in the mapped area.

Similarly, in addition to the mapping and detection network, the third network layer may also have a query network to perform a query operation on the corresponding third feature map, and transmit the query result to the fourth network layer, so that the fourth network layer After the query result is mapped into the mapping area, target detection is performed, and so on, until the bottom layer of the query layer is reached. The bottom layer of the query layer only includes the mapping and detection network, and the bottom layer only needs to output the detection result after obtaining the mapping area, and no longer needs to query the network.

It should be noted that the present disclosure considers the network layers that receive the same group of query points to be the same group of network layers, that is, the second network layer, the third network layer, the i-th network layer, etc. in the target detection network can be multiple. (Two are taken as an example in FIG. 4, and there may actually be more). The feature maps at the same level as the second network layer are called second feature maps, those at the same level as the third network layer are called third feature maps, and the feature maps at the same level as the i-th network layer are called is the i-th feature map. Wherein, each second network layer receives the query result from the first network layer, and maps the query result to the mapping area in the corresponding second feature map; each third network layer receives the query result from the second network layer query results, and map the query results to the corresponding mapping regions in the third feature map; and so on.

Specifically, the first network layer obtains a first group of query points after performing a query operation on the first feature map, and transmits the first group of query points to one or more second network layers. That is, the one or more second network layers receive the same set of query points, and respectively map the first query result to a mapping area in the feature map of the corresponding level, so as to perform detection operations in the mapping area.

Similarly, after a certain second network layer performs a query operation on the second feature map of the same level, a second set of query points are obtained, and are passed to one or more third network layers. At this time, the one or more third network layers respectively map the second group of query points to a mapping area in the corresponding hierarchical feature map, so as to perform a detection operation in the mapping area.

Here, multiple second network layers can perform query operations to obtain corresponding query results, and then the size scaling relationship between feature maps at different levels can be combined to obtain a final merged result by synthesizing the multiple query results obtained. For example, according to the query point coordinates of each second network layer and the mapping relationship between the coordinates, the comprehensive query points of the plurality of second network layers are obtained and transmitted to the third network layer.

Similarly, after the i-th network layer performs a query operation on the i-th feature map at the same level, it obtains the i-th group of query points and transmits it to one or more i+1-th network layers. At this time, the one or more i+1 th network layers respectively map the i th group of query points to a mapping area in the corresponding hierarchical feature map, so as to perform detection operations in the mapping area.

Preferably, each query layer has a query network and a detection network at the same time, and sequentially transmits the query results to the next lower-level network layer. At this time, the top layer of the query layer, that is, the first network layer, outputs the query result of the first feature map and transmits it to the second network layer, so that the second network layer can perform mapping and detection operations. At the same time, the second network layer performs a query operation on the second feature map, and transmits the query result to the third network layer, so that the third network layer can perform mapping and detection operations. That is, the middle layer of the query layer not only obtains the query results from the high-level network layer for mapping and detection operations, but also performs query operations on the feature maps of this layer, and transmits the query results to the lower-level network layer. And so on, until the bottom layer of the network layer is reached, the bottom layer only performs mapping and detection operations, and no longer performs query operations. That is, for the top and middle layers of the query layer, the detection network includes a classification network, a regression network, and a query network, which are used to output classification results, regression results, and query results; while the bottom layer of the query layer, the detection network only includes the classification network and the query network. Query the network to output classification and regression results.

According to an embodiment of the present disclosure, in order to improve the overall detection efficiency of the target detection network, the present disclosure uses a sparse convolution method to perform a detection operation on a feature map. Specifically, each network layer (eg, the second network layer) performing detection operations extracts image features of the mapped region to construct a sparse tensor, and uses sparse convolution to perform detection operations and/or query operations in the mapped region.

For a convolutional neural network, the input is usually a set of four-dimensional tensors, corresponding to the four dimensions of batch, channel, height, and width, respectively. The convolution kernel will be in the height and width plane. Calculation results for each position of . However, sometimes only the calculation result of a specific position in the plane is required, so it is not cost-effective to calculate the entire plane. For this reason, a sparse convolution method is proposed to reduce the amount of calculation and speed up the inference speed. In addition to the usual four-dimensional tensor input, the location of the calculation result is additionally required, so that the convolution kernel is only calculated at the specified location.

When sparse convolution is applied in the query layer, for the highest layer of the query layer (ie, the first network layer), since it has no input from the upper query point, ordinary convolution is used to calculate the result. For the features of the subsequent layers, the input query points are first mapped to the multiple adjacent positions of the layer, then the features are extracted from all positions and sparse features are constructed, and finally the results of the queried positions are calculated.

Optionally, the sparse convolution can also be replaced with a cropping operation, that is, feature cropping is performed on the queried mapping region, and a convolution result is calculated on the cropped features.

As can be seen from the foregoing, the target detection network includes multiple network layers capable of outputting detection results, such as an additional network layer, a second network layer, a third network layer, and the like. Based on this, the method 200 may further include the step of: constructing an output network, where the output network is used to combine and output the detection results obtained after each network layer performs the detection operation. There are various ways to combine outputs, for example, a weighted non-maximum suppression method is used to combine multiple regression boxes, which is not limited in the present disclosure.

In addition, as shown in Figure 3, in terms of detection types, the target detection network includes three network branches: classification network, regression network, and query network, which are used to perform classification operations, regression operations, and query operations, respectively. In order to improve the overall efficiency of model training, the classification network, regression network, and query network of the present disclosure share parameters among multiple network layers, that is, all classification networks have the same parameter types and parameter values, and all regression networks have the same parameters Type and parameter value, all query networks have the same parameter type and parameter value to achieve synchronous training of the same network branch.

Further, the present disclosure further includes the step of training the target detection model: generating a training sample set, and training the constructed target detection model according to the training sample set to obtain a trained target detection model. Wherein, the training sample set includes a plurality of training images and a label of each training image, and the label includes at least one of a classification label and a position label. At this time, the classification network branch can be iteratively updated according to the classification label of each training image and the predicted classification result, and the regression network branch can be iteratively updated according to the regression label of each training image and the predicted regression result.

In addition, the annotation of each training image may also include the query point annotation of the feature map of each layer generated by the training image. In this case, the query point annotation of the feature map of each layer and the predicted query point of the feature map of this layer can be , calculate the loss function, and train the constructed target detection network according to the loss function, so as to iteratively update the query network branch. When marking the query point of a training image, for any layer of feature maps; when the distance from a position point to the center of a specific target in the layer's feature map is less than a preset threshold, the position point is marked as a query point; Or when the degree of coincidence between the anchor box corresponding to a position point and the specific target in the feature map of the layer is greater than a preset threshold, the position point is marked as a query point. The degree of coincidence may be an intersection-union ratio, that is, the ratio of the intersection area and the union area.

Optionally, the present disclosure employs the focused loss to train the classification network branch and the query network branch, and the smooth L1 _loss to train the regression network branch.

After the target detection model is trained, the trained target detection model can be used for target detection. FIG. 5 shows a target detection method 500 according to an embodiment of the present disclosure. As shown in FIG. 5 , the target detection method 500 includes:

Step S501, input the image to be detected into the trained target detection model, the target detection model includes a feature extraction network and a target detection network, the specific network structure is shown in Figure 3 and Figure 4, and will not be repeated here.

Step S502, using a feature extraction network to perform feature extraction on the image to be detected, to obtain multi-layer feature maps with different sizes, and the multi-layer feature maps include a first feature map and a second feature map.

Step S503, using a target detection network to output the detection result of the multi-layer feature map, the target detection network includes a plurality of network layers corresponding to the multi-layer feature maps, and the plurality of network layers include a first network layer and a second network layer. The first network layer corresponds to the first feature map, and is used to perform a query operation on the first feature map, and transmit the obtained query result to the second network layer, where the query result includes the query point of a specific target in the first feature map. The second network layer corresponds to the second feature map, and is used to determine the mapping area of the query point in the second feature map, and perform a detection operation in the mapped area to obtain a detection result.

As shown in Fig. 3 and Fig. 4, the target detection network also includes an additional network layer, a third network layer, a fourth network layer, ..., an nth network layer, and each network layer has a detection network for obtaining the corresponding Test results. Therefore, in step S530, using the target detection network to output the detection result of the multi-layer feature map includes: combining the detection results of each network layer to obtain the detection result of the input picture.

FIG. 6 shows a schematic diagram of the effect of using a target detection model for target detection according to an embodiment of the present disclosure, wherein a low-resolution feature map is mainly used to detect large objects, and query points are transmitted to a high-resolution feature map, so as to After the high-resolution feature map determines the mapping area corresponding to the query point, small objects in the mapping area are detected. Using the target detection model of the present disclosure to perform target detection has the advantages of high robustness, fast reasoning speed, and high efficiency.

FIG. 7 shows an apparatus 700 for constructing a target detection model according to an embodiment of the present disclosure. As shown in FIG. 7 , the apparatus 700 includes:

The first construction unit 701 is used to construct a feature extraction network, the feature extraction network is used to perform feature extraction on the input picture to obtain multi-layer feature maps with different sizes, and the multi-layer feature maps include a first feature map and a second feature picture.

The second construction unit 702 is configured to construct a target detection network, where the target detection network includes multiple network layers corresponding to the multi-layer feature maps, and the multiple network layers include a first network layer and a second network layer. The first network layer corresponds to the first feature map, and is used to perform a query operation on the first feature map, and transmit the obtained query result to the second network layer, where the query result includes the query point of a specific target in the first feature map. The second network layer corresponds to the second feature map, and is used to determine the mapping area of the query point in the second feature map, and perform a detection operation in the mapped area to obtain a detection result.

Optionally, the apparatus 700 for constructing a target detection model may further include a third constructing unit and a model training unit (neither are shown in the figures). The third construction unit is used to construct an output network, and the output network is used to combine and output the detection results obtained after each network layer performs detection operations. The model training unit is used to generate a training sample set, and train the constructed target detection model according to the training sample set to obtain a trained target detection model. The training sample set includes a plurality of training images and a label for each training image, and the label includes at least one of a classification label and a position label.

In addition, the annotation of each training image also includes the query point annotation of the feature map of each layer generated by the training image. At this time, the model training unit is based on the query point annotation of the feature map of each layer and the predicted query point of the feature map of this layer. point, calculate the loss function, and train the constructed object detection network according to the loss function. Generally, for any feature map of any layer; when the distance between a certain position point and the center of a specific target in the feature map of this layer is less than a preset threshold, the position point is marked as a query point; or, when the anchor corresponding to a certain position point is When the intersection ratio between the box and the specific target in the feature map of this layer is greater than the preset threshold, the location point is marked as a query point.

FIG. 8 shows a target detection apparatus 800 according to an embodiment of the present disclosure. As shown in FIG. 8 , the apparatus 800 includes:

The input unit 801 is configured to input the to-be-detected picture into a target detection model, where the target detection model includes a feature extraction network and a target detection network.

The feature extraction unit 802 is configured to perform feature extraction on the image to be detected by using a feature extraction network to obtain multi-layer feature maps with different sizes, and the multi-layer feature maps include a first feature map and a second feature map.

The target detection unit 803 is used to output the detection result of the multi-layer feature map by using a target detection network, the target detection network includes a plurality of network layers corresponding to the multi-layer feature maps, and the plurality of network layers include a first network layer and a first network layer. Two network layers. The first network layer corresponds to the first feature map, and is used to perform a query operation on the first feature map, and transmit the obtained query result to the second network layer, and the query result includes the query of a specific target in the first feature map. point. The second network layer corresponds to the second feature map, and is used to determine the mapping area of the query point in the second feature map, and perform a detection operation in the mapped area to obtain a detection result.

Optionally, the target detection unit 803 is configured to combine and output the detection results obtained after the detection operations are performed on each network layer (including the non-query layer and the query layer with the detection network).

It should be noted that the specific implementation manners of the target detection model building apparatus 700 and the target detection apparatus 800 provided by the embodiments of the present disclosure have been disclosed in detail in the description based on FIG. 1 to FIG. 6 , and will not be repeated here.

In addition, an embodiment of the present disclosure further provides a computer-readable storage medium, including a program or an instruction, when the program or instruction is run on a computer, the object state estimation method as described above is implemented.

In addition, an embodiment of the present disclosure further provides a computing device 900 as shown in FIG. 9 , including a memory 901 and one or more processors 902 communicatively connected to the memory. The memory 901 stores instructions executable by the one or more processors 902 to cause the one or more processors 902 to implement the object state estimation method as previously described. Computing device 900 may further include a communication interface 903 that may implement one or more communication protocols (LTE, Wi-Fi, etc.).

According to the technical solution of the present disclosure, the detection performance of small objects is improved by introducing features of higher resolution, and a query head is added to the detection head, thereby realizing a target detection method based on query and sparse convolution, and solving the problem of introducing high-resolution The problem of slower inference after features. By deploying the object detection method of the present disclosure to an autonomous vehicle, the performance of small object detection can be made fast and good.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In this disclosure, specific embodiments are used to illustrate the principles and implementations of the present disclosure. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present disclosure; There will be changes in the disclosed ideas in terms of specific implementations and application scopes. To sum up, the contents of this specification should not be construed as limiting the present disclosure.

Claims (23)

1. A construction method of an object detection model comprises the following steps:

constructing a feature extraction network, wherein the feature extraction network is used for carrying out feature extraction on an input picture to obtain multilayer feature maps with different sizes, and the multilayer feature maps comprise a first feature map and a second feature map;

constructing an object detection network, wherein the object detection network comprises a plurality of network layers corresponding to the multilayer characteristic diagram, and the plurality of network layers comprise a first network layer and a second network layer;

the first network layer corresponds to the first feature map and is used for carrying out query operation on the first feature map and transmitting an obtained query result to the second network layer, wherein the query result comprises a query point of a specific target in the first feature map;

the second network layer corresponds to the second feature map and is used for determining a mapping area of the query point in the second feature map and performing detection operation in the mapping area to obtain a detection result.

2. The method of claim 1, wherein the first network layer is further configured to perform a detection operation on the first feature map to obtain a detection result.

3. The method of claim 1, wherein the target detection network has a plurality of second network layers, each second network layer receiving the query result from the first network layer and mapping the query result to a mapping region in a corresponding second feature map.

4. The method of claim 1, wherein the multi-layer signature further comprises a third signature, the object detection network further comprises a third network layer corresponding to the third signature, the second network layer further configured to:

inquiring the mapping area in the second characteristic diagram to obtain a corresponding inquiry result;

and transmitting the obtained query result to the third network layer so that the third network layer maps the query point in the query result to a mapping area in the corresponding third feature map, and performing detection operation in the mapping area to obtain a detection result.

5. The method of claim 1, wherein the second network layer is further configured to:

extracting image features of the mapping region to construct a sparse tensor;

and carrying out detection operation and/or query operation in the mapping region by adopting sparse convolution.

6. The method of claim 1, wherein,

the multi-layer feature map further comprises an additional feature map;

the target detection network further comprises an additional network layer corresponding to the additional feature map;

and the additional network layer is used for carrying out detection operation on the additional characteristic diagram to obtain a detection result.

7. The method of any of claims 1-6, further comprising:

and constructing an output network, wherein the output network is used for merging and outputting detection results obtained after detection operation is carried out on each network layer.

8. The method of claim 1, wherein the feature extraction network comprises:

the system comprises a backbone network, a data processing network and a data processing network, wherein the backbone network is used for extracting features of an input picture to obtain an initial multilayer feature map; and

and the characteristic pyramid network is used for performing up-sampling and characteristic fusion on the initial characteristic diagram to obtain an improved multilayer characteristic diagram.

9. The method according to claim 1, wherein the second feature map is obtained by upsampling and feature fusing the first feature map, the size of the second feature map is m times of that of the first feature map, and m > 1.

10. The method of claim 9, wherein the second network layer maps the query point to a plurality of points in the second feature map according to a magnification between the first feature map and the second feature map, resulting in the mapped region.

11. The method of claim 1, wherein,

the detecting operation comprises at least one of a regression operation and a classification operation;

the detection result includes at least one of a regression result and a classification result.

12. The method of claim 11, wherein,

the target detection network comprises a classification network, a regression network and an inquiry network;

the classification operation, the regression operation and the query operation are respectively executed by a classification network, a regression network and a query network.

13. The method of claim 1, wherein,

the query result comprises a query result graph;

the query result graph comprises the probability of the specific target existing at each position point in the corresponding feature graph;

the query point is a position point with the probability value being greater than or equal to a preset threshold value.

14. The method of claim 1, wherein each layer of feature map has a corresponding size threshold, and the particular target of each layer of feature map is a target having a size equal to or less than the size threshold of the layer of feature map.

15. The method of claim 1, further comprising:

generating a training sample set, wherein the training sample set comprises a plurality of training images and labels of each training image, and the labels comprise at least one of classification labels and position labels;

and training the constructed target detection model according to the training sample set to obtain the trained target detection model.

16. The method of claim 15, wherein the label of each training image further includes a query point label of each layer of feature map generated by the training image, and the training of the constructed target detection model according to the training sample set includes:

and calculating a loss function according to the query point label of each layer of feature diagram and the predicted query point of the layer of feature diagram, and training the constructed target detection network according to the loss function.

17. The method of claim 16, wherein, for any layer profile;

when the distance from a certain position point to the center of a specific target in the layer characteristic diagram is smaller than a preset threshold value, marking the position point as a query point; or

And when the intersection ratio of the anchor frame corresponding to a certain position point and a specific target in the layer of feature graph is greater than a preset threshold value, marking the position point as a query point.

18. A method of target detection, comprising:

inputting a picture to be detected into a target detection model, wherein the target detection model comprises a feature extraction network and a target detection network;

performing feature extraction on the picture to be detected by adopting the feature extraction network to obtain multilayer feature maps with different sizes, wherein the multilayer feature maps comprise a first feature map and a second feature map; and

outputting a detection result of the multilayer characteristic diagram by adopting the target detection network, wherein the target detection network comprises a plurality of network layers corresponding to the multilayer characteristic diagram, and the plurality of network layers comprise a first network layer and a second network layer; wherein,

the first network layer corresponds to the first feature map and is used for carrying out query operation on the first feature map and transmitting an obtained query result to the second network layer, wherein the query result comprises a query point of a specific target in the first feature map;

the second network layer corresponds to the second feature map and is used for determining a mapping area of the query point in the second feature map and performing detection operation in the mapping area to obtain a detection result.

19. An apparatus for constructing an object detection model, comprising:

the device comprises a first construction unit, a second construction unit and a third construction unit, wherein the first construction unit is used for constructing a feature extraction network, the feature extraction network is used for performing feature extraction on an input picture to obtain multilayer feature maps with different sizes, and the multilayer feature maps comprise a first feature map and a second feature map;

a second constructing unit, configured to construct an object detection network, where the object detection network includes multiple network layers corresponding to the multiple layers of feature maps, and the multiple network layers include a first network layer and a second network layer;

the first network layer corresponds to the first feature map and is used for carrying out query operation on the first feature map and transmitting an obtained query result to the second network layer, wherein the query result comprises a query point of a specific target in the first feature map;

the second network layer corresponds to the second feature map and is used for determining a mapping area of the query point in the second feature map and performing detection operation in the mapping area to obtain a detection result.

20. An object detection device comprising:

the image detection device comprises an input unit, a detection unit and a processing unit, wherein the input unit is used for inputting an image to be detected into a target detection model, and the target detection model comprises a feature extraction network and a target detection network;

the characteristic extraction unit is used for extracting the characteristics of the picture to be detected by adopting the characteristic extraction network to obtain multilayer characteristic diagrams with different sizes, and the multilayer characteristic diagrams comprise a first characteristic diagram and a second characteristic diagram; and

the target detection unit is used for outputting a detection result of the multilayer characteristic diagram by adopting the target detection network, the target detection network comprises a plurality of network layers corresponding to the multilayer characteristic diagram, and the plurality of network layers comprise a first network layer and a second network layer; wherein,

the first network layer corresponds to the first feature diagram and is used for carrying out query operation on the first feature diagram and transmitting an obtained query result to the second network layer, wherein the query result comprises a query point of a specific target in the first feature diagram;

the second network layer corresponds to the second feature map and is used for determining a mapping area of the query point in the second feature map and performing detection operation in the mapping area to obtain a detection result.

21. A computing device comprising a memory, and one or more processors communicatively connected with the memory;

the memory has stored therein instructions executable by the one or more processors to cause the one or more processors to implement a method according to any one of claims 1 to 18.

22. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the method of any one of claims 1-18.

23. A vehicle comprising the computing device of claim 21.

Images