WO2025241439A1 - Image processing method for image acquisition device, electronic device and storage medium - Google Patents

Abstract

Disclosed in the present application are an image processing method for an image acquisition device, an electronic device and a storage medium. The image processing method for an image acquisition device comprises: in a low-frequency mode, a processor receives image data from an image sensor, at least some of functional modules in the low-frequency mode being in an unstarted state; in response to the received image data meeting a first image processing condition, adjusting the processor to a high-frequency mode to start each functional module, the first image processing condition comprising that the image data contains related information of a target object and/or the amount of the image data has reached a preset amount; and controlling each functional module to perform image processing on the image data. The solution can shorten the time of awakening each functional module in the image processing process of the image acquisition device, thereby reducing the consumption of battery power of the image acquisition device.

Description

Image processing methods, electronic devices, and storage media of image acquisition equipment

Cross-referencing

This application claims priority to Chinese Patent Application No. 2024106321211, filed on May 21, 2024, entitled “Image Processing Method, Electronic Device and Storage Medium for Image Acquisition Device”, the entire contents of which are incorporated herein by reference.

[Technical Field]

This application relates to the field of computer technology, and in particular to an image processing method, electronic device, and storage medium for an image acquisition device.

[Background Technology]

With the rapid development of image acquisition technology, image acquisition devices capable of real-time image acquisition are becoming increasingly widely used. Existing image acquisition devices are in a startup state during operation, allowing each functional module to perform corresponding image processing after receiving the image data. However, since these modules are constantly awake, they operate in a high-power mode when image processing is not required, resulting in significant energy consumption.

Given the shortcomings of existing technologies, how to provide an effective image processing solution for image acquisition devices is a technical problem that urgently needs to be solved by those skilled in the art.

[Summary of the Invention]

This application provides at least one image processing method for an image acquisition device, an electronic device, and a storage medium.

This application provides an image processing method for an image acquisition device, comprising: a processor receiving image data from an image sensor in a low-frequency mode, wherein at least some functional modules in the low-frequency mode are in an inactive state; in response to the received image data satisfying a first image processing condition, adjusting the processor to a high-frequency mode to activate each functional module, the first image processing condition including that the image data contains relevant information about a target object and/or the number of image data reaches a preset number; and controlling each functional module to perform image processing on the image data.

This application provides an image processing apparatus for an image acquisition device, comprising: a receiving module, an adjustment module, and a control module; the receiving module is configured to receive image data from an image sensor in a low-frequency mode, wherein at least some functional modules are inactive in the low-frequency mode; the adjustment module is configured to adjust the processor to a high-frequency mode to activate each functional module in response to the received image data satisfying a first image processing condition, wherein the first image processing condition includes that the image data contains relevant information about a target object and/or the number of image data reaches a preset number; and the control module is configured to control each functional module to perform image processing on the image data.

This application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the image processing method of the image acquisition device described above.

This application provides a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the image processing method of the aforementioned image acquisition device.

The above solution, in response to the image data received by the processor in low-frequency mode meeting the first image processing condition, adjusts the processor to high-frequency mode to start each functional module. The first image processing condition includes that the image data contains relevant information about the target object and/or the number of image data reaches a preset number. Control each functional module to perform image processing on the image data. Compared with the prior art where each functional module is always in a wake-up state waiting to send image data and perform image processing, this application can reduce the wake-up time of each functional module during the image processing of the image acquisition device, thereby reducing the battery power consumption of the image acquisition device.

It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application.

[Attached Image Description]

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

Figure 1 is a flowchart illustrating an embodiment of the image processing method of the image acquisition device of this application;

Figure 2 is another flowchart illustrating an embodiment of the image processing method of the image acquisition device of this application;

Figure 3 is another schematic flowchart of an embodiment of the image processing method of the image acquisition device of this application;

Figure 4 is a schematic diagram of the structure of an embodiment of the image processing device of the image acquisition equipment of this application;

Figure 5 is a schematic diagram of the structure of an embodiment of the electronic device of this application;

Figure 6 is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application.

【Detailed Implementation Methods】

The embodiments of this application will now be described in detail with reference to the accompanying drawings.

In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

In this document, the term "and/or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and/or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character "/" generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

This application provides image processing methods and devices for image acquisition devices. The application scenarios of the image processing methods include, but are not limited to, the startup process of multi-camera devices. The execution entity of the image processing methods can be the image processing device or the main controller of the image acquisition device. For example, the image processing device can be located in a terminal device, server, or other processing device. The terminal device can be a device for image processing, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, etc. In some possible implementations, the image processing methods of the image acquisition device can be implemented by a processor calling computer-readable instructions stored in memory. This can be achieved through commands.

Please refer to Figure 1, which is a schematic flowchart of an embodiment of the image processing method of the image acquisition device of this application. Specifically, the image acquisition device includes an image sensor, a processor, and several functional modules. The image processing method of the image acquisition device may include the following steps:

Step S11: The processor receives image data from the image sensor in low-frequency mode.

In the low-frequency mode, at least some functional modules are in an inactive state.

The processor can be the central processing unit (CPU) of the image acquisition device. Specifically, the processor can be a processor with computational and control capabilities, and can operate at different frequencies.

The processor can have a low-frequency mode. In low-frequency mode, the processor has weak processing power and low power consumption. In some applications, the processor in low-frequency mode can still receive and transmit image data. In some applications, when the processor is in low-frequency mode, at least some functional modules in the image acquisition device are in an inactive state. Each functional module in the image acquisition device can be a module capable of image processing of image data. In some applications, each functional module in the image acquisition device can be a hardware encoder. The hardware encoder can encode the received data into video files of different formats for storage. Specifically, the hardware encoder can be a video encoder (ENC). The video encoder can be a video encoder of different formats. For example, the video encoder can be a first encoder (Video EncoderH264), a second encoder (Video EncoderH265), or a third encoder (Video EncoderMotionJpeg). Each functional module in the image acquisition device can also be an image signal processor (ISP). An image sensor can be an image sensor (Sensor). An image sensor can send image data to a processor. The image sensor may have a motion detection mode. Specifically, the motion detection mode can be the motion detection function inherent in the image sensor. In some applications, the motion detection function can be an image sensor adapted for motion detection, outputting a control signal when the image changes. In some applications, the aforementioned processor may refer to a main control chip connected to the image sensor.

Step S12: In response to the received image data satisfying the first image processing condition, adjust the processor to high-frequency mode to start each functional module.

The first image processing condition includes that the image data contains relevant information about the target object and/or the amount of image data reaches a preset quantity.

The processor can have a high-frequency mode. Processors in high-frequency mode have strong processing power but also high power consumption. In some applications, processors in high-frequency mode can have processing capabilities other than receiving and sending image data. These other capabilities could include sending corresponding data to various functional modules to activate them and complete image data encoding. In some applications, when the processor is in high-frequency mode, all functional modules in the image acquisition device are in the activated state. It is understandable that after determining that the received image data meets the first image processing conditions, the processor in low-frequency mode can be switched to high-frequency mode. After switching from low-frequency to high-frequency mode, the processor in high-frequency mode can activate all functional modules. In some applications, the main control chip enters high-frequency mode and simultaneously starts the target software operating system to implement personalized settings related to image processing. The requirement is to simultaneously boot the target software operating system. Specifically, this process can involve recovering from a power management mode to enter the operating system. The power management mode can be the target power mode STR (Suspend to RAM or Sleep, STR). The target power mode STR manages the operating system's permissions. Depending on the different requirement stages, the target power mode STR manages the operating system's permissions. In some application scenarios, the first requirement stage can be when all operations of the operating system are stopped, but the memory still receives power and retains its contents. In this first requirement stage, the operating system saves energy when inactive, conserving the battery power of the image acquisition device. In other application scenarios, the second requirement stage allows the system to quickly return to full-power operation when the required conditions are met. These required conditions can be the steps mentioned above, such as the received image data meeting the first image processing conditions, or the steps mentioned above, such as adjusting the processor to a high-frequency mode. Dividing the system into first and second requirement stages can improve battery power efficiency.

In some application scenarios, it's necessary to determine whether image data contains information about a target object. The first image processing condition can be that the image data contains information about the target object. The target object can be set according to requirements. Specifically, the target object can be a predetermined biological type, a predetermined object type, or a predetermined vehicle type. In some application scenarios, it's necessary to determine whether the number of images corresponding to the image data reaches a preset number. The preset number can be 10 images corresponding to the image data. The first image processing condition can be that the number of images corresponding to the image data reaches the preset number. The images corresponding to the image data can be images acquired by an image acquisition device from the target area. In other application scenarios, it's necessary to simultaneously determine whether the image data contains information about the target object and whether the number of images corresponding to the image data reaches a preset number. The first image processing condition can be that the image data contains information about the target object and the number of images related to the target object in the image data reaches the preset number.

Step S13: Control each functional module to perform image processing on the image data.

After activating each functional module, the system controls each module to process the image data. In some applications, image processing can involve sending the image data to each functional module all at once to obtain video data. For example, the image data can be in the form of ImageRaw data. Alternatively, image processing can involve sending the image data to a hardware encoder all at once, waiting for encoding to complete to obtain video data. Specifically, the image data can be sent to both an image signal processor and a video encoder all at once, waiting for encoding to complete to obtain video data. After obtaining the video data, this video data is used as the first target storage data and stored. After the first target storage data is fully stored, the system controls the processor to power down. In some applications, powering down the processor can also mean powering down the main control chip.

In some embodiments, the image acquisition device further includes a memory. In some applications, video data can be stored by writing the video data to the memory in a high-power mode. In response to the completion of video data storage, the memory is switched to a low-power mode, and the image sensor is controlled to enter a sleep state. After the memory is in low-power mode, the processor is powered down. Controlling the image sensor to enter a sleep state can be achieved by the processor notifying the image sensor to enter a sleep state.

The above solution, in response to the image data received by the processor in low-frequency mode meeting the first image processing condition, adjusts the processor to high-frequency mode to start each functional module. The first image processing condition includes the image data containing relevant information about the target object and/or the number of image data reaching a preset quantity. It then controls each functional module to perform image processing on the image data. Compared to the prior art where each functional module is constantly awake, waiting for image data to be sent and performing image processing, this application can reduce the need for image acquisition equipment. This reduces the wake-up time of each functional module during image processing, thereby reducing the battery consumption of the image acquisition device.

In some embodiments, the image acquisition device further includes a memory, and the image processing method of the image acquisition device may further include the following steps: First, in response to the received image data satisfying a second image processing condition, the image data is written to the memory in a high-power mode. The second image processing condition includes that the image data does not contain relevant information about the target object and the number of image data does not reach a preset number. Subsequently, in response to the image data storage being completed, the memory is adjusted to a low-power mode, and the image sensor is controlled to enter a sleep state.

The memory in an image acquisition device can be random access memory (RAM). For example, the memory can be DDR (Double Data Rate) RAM. Specifically, DDR RAM can be DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory). DDR RAM can be a memory technology primarily used in computer RAM. The memory has low-power and high-power modes. In some applications, memory in high-power mode executes read/write commands immediately upon receipt, resulting in faster speed. Memory in high-power mode consumes more power. Memory in high-power mode can be in an active or normal mode, capable of reading and writing the data to be stored. In other applications, memory in low-power mode automatically refreshes the data to be stored; this mode is slower but can operate at low voltage. Memory in low-power mode consumes less power. Memory in low-power mode can also be in self-refresh mode, unable to read or write the data to be stored. It is understood that memory in low-power mode has the function of retaining historical stored data but does not have the function of reading and writing the data to be stored. Retaining historical stored data can be either video data stored in memory that was in high-power mode before the mode adjustment, or image data stored in memory that was in high-power mode before the mode adjustment.

The second image processing condition includes that the image data does not contain information related to the target object and the number of image data points does not reach a preset quantity. In response to the received image data meeting the second image processing condition, the image data is written to the memory in high-power mode. At this time, the image data can be raw image data that has not been processed by the image signal processor and video encoder. In response to the received image data meeting the second image processing condition, the image data is used as second target storage data and written to the memory in high-power mode. After the second target storage data is fully stored, the memory is switched to low-power mode, and the image sensor is controlled to enter sleep mode. After the memory is in low-power mode, the processor is powered down. Controlling the image sensor to enter sleep mode can be achieved by the processor notifying the image sensor to enter sleep mode.

It can be assumed that when the received image data meets the second image processing conditions, the image data is written to the memory in high-power mode, the memory is adjusted to low-power mode, and the image sensor is put into sleep mode and the processor is powered off. This can cache image data when the device is in motion and there is a non-target object, and save the power consumption of each module in the image acquisition device, thereby improving the battery efficiency of the image acquisition device.

In some embodiments, prior to the step of adjusting the processor to a high-frequency mode in response to the received image data satisfying the first image processing condition, the image processing method of the image acquisition device may further include the following steps: First, cropping the image data to obtain data to be processed. Then, performing object detection processing and/or quantity statistics processing on the data to be processed to obtain a data processing result. Image processing result package This includes object detection results and/or quantity statistics results. The object detection results include information about whether the data to be processed contains the target object or information about whether the data to be processed does not contain the target object. The quantity statistics results include the total number of images in the data to be processed. Next, based on the data processing results, it is determined whether the received image data meets the first image processing condition.

The image acquisition device also includes a video cropping module. This video cropping module can be a Video Input Interface (VIF).

In some application scenarios, the video cropping module can transmit received image data to the main control unit. Object detection and/or quantity statistics processing are then performed on the image data to obtain the data processing result. In some application scenarios, object detection processing on the image data yields an object detection result. In other application scenarios, quantity statistics processing on the image data yields a quantity statistics result. In still other application scenarios, the above steps of object detection and quantity statistics processing on the image data are performed simultaneously, resulting in a data processing result that includes both object detection and quantity statistics.

In other application scenarios, the video cropping module can crop image data to obtain data to be processed. The video cropping module can then transmit the data to be processed to the main control unit. Object detection and/or quantity statistics processing are performed on the data to be processed to obtain the data processing result. In some application scenarios, object detection processing is performed on the data to be processed, and the resulting data processing result can be the object detection result. In some application scenarios, quantity statistics processing is performed on the data to be processed, and the resulting data processing result can be the quantity statistics result. In other application scenarios, the above steps of object detection processing and quantity statistics processing are performed simultaneously, and the resulting data processing result can be both the object detection result and the quantity statistics result. The relationship between the data to be processed and the image data can be such that the data to be processed is cropped image data, where the number of images in the data to be processed is the same as the number of images in the image data.

In some application scenarios, the steps described above for performing quantity statistics on the data to be processed, or for performing quantity statistics on image data, can be equivalent to counting the frame rate in the data to be processed or the image data. The frame rate counting can be achieved by adding the historical statistical count to the number of images in the image data, or by adding the historical statistical count to the number of images in the data to be processed.

Specifically, the frame rate counting process can be referred to formula (1):

RawCount1＝RawCount0+N Formula (1);

Here, RawCount0 can represent the historical statistical count mentioned above. The historical statistical count can be the number of images in historical image data stored in memory at a historical time. N can represent the number of images in the data to be processed or the image data at the current time. For example, N can be 1. RawCount1 can represent the statistical result.

The image processing results include object detection results and/or quantity statistics results. Object detection results include information about whether the data to be processed contains the target object or information about whether the data to be processed does not contain the target object. Quantity statistics results include the total number of images in the data to be processed. Based on the data processing results, it is determined whether the received image data meets the first image processing condition.

In some application scenarios, the above steps of object detection processing on the data to be processed are performed first, and the data processing result can be the object detection result. If the object detection result indicates that the data to be processed does not contain the target object... If the relevant information of the target object is obtained, the above steps of performing quantity statistics processing on the image data are then performed. The quantity statistics result is the total number of images corresponding to each image in the data to be processed. If the total number of images corresponding to each image in the data to be processed corresponding to the quantity statistics result does not reach the preset number, the image processing result is determined to be that the received image data meets the second image processing condition. The data to be processed is used as the second target storage data and written to the memory in high-power mode. After the second target storage data is fully stored, the memory is adjusted to low-power mode, and the image sensor is controlled to enter sleep mode.

It is understood that the steps for object detection processing and quantity statistics processing of the data to be processed described above can be executed serially or in parallel. The steps for object detection processing of image data described above can refer to the steps for object detection processing of the data to be processed described above, and will not be repeated here. Similarly, the steps for quantity statistics processing of image data described above can refer to the steps for quantity statistics processing of the data to be processed described above, and will not be repeated here.

In some application scenarios, in a serial execution mode, the above-mentioned steps for object detection processing of the data to be processed are executed first, and the data processing result can be the object detection result. If the object detection result indicates that the data to be processed does not contain information related to the target object, then the above-mentioned steps for quantity statistics processing of the data to be processed are executed. When the total number of images corresponding to the quantity statistics in the data to be processed reaches a preset number, the image processing result is determined, confirming that the received data to be processed meets the first image processing condition.

In some application scenarios, in a serial execution mode, the above-mentioned steps for object detection processing of the data to be processed are executed first, and the data processing result can be the object detection result. If the object detection result indicates that the data to be processed does not contain information related to the target object, then the above-mentioned steps for quantity statistics processing of the data to be processed are executed. If the total number of images corresponding to the quantity statistics result in the data to be processed does not reach the preset number, the image processing result is determined to indicate that the received data to be processed meets the second image processing condition.

In some application scenarios, in a serial execution mode, the above-mentioned steps for object detection processing of the data to be processed are executed first, and the data processing result can be the object detection result. If the object detection result shows that the data to be processed contains relevant information about the target object, then the above-mentioned steps for quantity statistics processing of the data to be processed are executed. When the total number of images corresponding to the quantity statistics in the data to be processed reaches a preset number, the image processing result is determined, confirming that the received data to be processed meets the first image processing condition.

In some application scenarios, in the serial execution mode, the above-mentioned steps for object detection processing of the data to be processed are performed first, and the data processing result can be the object detection result. If the object detection result indicates that the data to be processed contains relevant information about the target object, the above-mentioned steps for quantity statistics processing of the data to be processed do not need to be performed. The image processing result is then determined to ensure that the received data to be processed meets the first image processing condition.

In other application scenarios, in parallel execution mode, the steps of object detection processing and quantity statistics processing of the data to be processed are executed in parallel. The resulting data processing results include object detection results and quantity statistics results. If the object detection result indicates that the data to be processed does not contain information related to the target object, and the total number of images in the data to be processed corresponding to the quantity statistics results does not reach a preset number, then the data to be processed is determined to meet the second image processing condition. If the object detection result indicates that the data to be processed does not contain the target object... If, in the case of relevant information, the total number of images in the data to be processed corresponding to the quantity statistics results reaches a preset number, then the data to be processed is determined to meet the first image processing condition. If, in response to the case where the object detection result indicates that the data to be processed contains relevant information about the target object, and the total number of images in the data to be processed corresponding to the quantity statistics results reaches or does not reach a preset number, then the data to be processed is determined to meet the first image processing condition.

In some application scenarios, in response to the received data to be processed meeting the first image processing condition, the processor is adjusted to a high-frequency mode to activate each functional module. Each functional module is controlled to perform image processing on the data to be processed, obtaining video data. After obtaining the video data, this video data is used as the first target storage data and stored. After the first target storage data is fully stored, the memory is adjusted to a low-power mode, and the image sensor is controlled to enter a sleep state. After the first target storage data is fully stored, the processor is powered down.

In some application scenarios, in response to the received data to be processed meeting the second image processing conditions, the data to be processed is used as the second target storage data and written to the memory in high-power mode. After the second target storage data is stored, the memory is switched to low-power mode, and the image sensor is controlled to enter sleep mode. After the memory is in low-power mode, the processor is powered down.

It can be assumed that when the received image data meets the second image processing conditions, the image data is written to the memory in high-power mode, the memory is adjusted to low-power mode, and the image sensor is put into sleep mode and the processor is powered off. This can cache image data when the device is in motion and there is a non-target object, and save the power consumption of each module in the image acquisition device, thereby improving the battery efficiency of the image acquisition device.

It can be assumed that when the received image data meets the first image processing conditions, video data is written to the memory in high-power mode, and the memory is adjusted to low-power mode, and the image sensor is put into sleep mode and the processor is powered off. This allows the storage of video data when the data is in motion and the total number of images in the data to be processed corresponding to the target object and/or the quantity statistics results reaches a preset number. This saves the power consumption of each module in the image acquisition device, thereby improving the battery efficiency of the image acquisition device.

In some embodiments, the operating states of the image sensor include a motion detection mode and an image output mode. Before step S11 above, the image processing method of the image acquisition device may further include the following steps: First, controlling the image sensor in motion detection mode to detect image data to obtain a motion detection result. The motion detection result includes a first motion detection result regarding information about a moving object contained in the image data. Next, in response to the motion detection result being the first motion detection result, adjusting the image sensor to an image output mode so that the processor receives image data from the image sensor in a low-frequency mode.

Image sensors have an active state and a sleep state. In the active state, an image sensor can have a motion detection mode and an image output mode. Specifically, the motion detection mode can be the motion detection function inherent in the image sensor. In some applications, the motion detection function can be an image sensor adapted to a motion detection function, outputting a control signal when the image changes. For example, the motion detection mode can be a motion detection (MD) function. Specifically, the image output mode allows the image sensor to send the acquired image data to the processor.

The moving object can be any object to be verified, including the target object. The objects to be verified can be pre-defined objects. The object to be identified can be a specific biological type, a predetermined object type, or a predetermined vehicle type. Specifically, the object to be identified can be an object type in the acquired image data that causes changes in the corresponding image area. The image sensor, in motion detection mode, is controlled to detect the image data and obtain a motion detection result. The motion detection result is a first motion detection result regarding information about moving objects contained in the image data. If the motion detection result is the first motion detection result, the image sensor is adjusted to an image output mode so that the processor receives image data from the image sensor in a low-frequency mode.

In some embodiments, the motion detection result includes a second motion detection result regarding the image data that does not contain information about a moving object. The image processing method of the image acquisition device may further include the following steps: in response to the motion detection result being the second motion detection result, controlling the image sensor to enter a sleep state.

The image sensor, in motion detection mode, is controlled to detect image data and obtain motion detection results. These results are considered a first motion detection result, containing information about moving objects within the image data. If the motion detection result is a second motion detection result, the image sensor is controlled to enter a sleep state.

In some embodiments, the first motion detection result includes a plurality of current motion detection events, each representing a plurality of objects to be confirmed in motion. The step of adjusting the image sensor to an image output mode in response to the motion detection result being the first motion detection result may include the following steps: adjusting the image sensor to an image output mode in response to the first motion detection result satisfying a first preset condition, wherein the first preset condition includes that there are no historical motion detection events prior to the current timestamp corresponding to the current motion detection event in the first motion detection result. Historical motion detection events represent objects to be confirmed in motion prior to the current timestamp. Alternatively, adjusting the image sensor to an image output mode in response to the first motion detection result satisfying a second preset condition, wherein the second preset condition includes that there are historical motion detection events prior to the current timestamp corresponding to the current motion detection event in the first motion detection result, and the time interval between the current timestamps corresponding to at least one current motion detection event in the first motion detection result is greater than or equal to the second preset time.

The first motion detection result includes several current motion detection events. These current motion detection events can be a single event or multiple events. Each current motion detection event represents several objects to be verified as being in motion. A single current motion detection event represents one object to be verified as being in motion. The objects to be verified across multiple current motion detection events can be different objects or the same object.

For example, when the image sensor acquires image data for the first time after power-on, it is determined that there are no historical motion detection events prior to the first acquisition of image data. Alternatively, if the image sensor acquires image data multiple times, no historical motion detection events are detected in the other image data before the last image data, but a motion detection event is detected in the last image data, and there are no historical motion detection events prior to the current motion detection event corresponding to the last image data. The absence of historical motion detection events may be due to the absence of historical image data in the memory or the absence of historical data to be processed.

Historical motion detection events represent the state of motion of the object to be confirmed before the current timestamp. Each current or historical motion detection event corresponds to a timestamp. That is, the current timestamp can be the start time of the object being confirmed being in motion within the current motion detection event; the current timestamp can be the end time of the object being confirmed being in motion within the current motion detection event; or the current time period of the object being confirmed being in motion within the current motion detection event. The current time period includes the start and end times of the object being confirmed being in motion. Similarly, a historical timestamp can be the start time of the object being confirmed being in motion within a historical motion detection event; or the end time of the object being confirmed being in motion within a historical motion detection event. The historical time period of the motion state. The historical time period includes the start and end times of the object being checked being in motion.

In some application scenarios, the first preset condition includes that there are no historical motion detection events before the current timestamp corresponding to the current motion detection event in the first motion detection result. It is understandable that if there are no historical motion detection events before the current timestamp of several current motion detection events corresponding to the image data acquired by the image sensor, the image sensor is directly adjusted to the image output mode.

In some application scenarios, the second preset condition includes the following: if there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the time interval between the current timestamps corresponding to at least one current motion detection event in the first motion detection result is greater than or equal to the second preset time. It can be understood that, in the case where there are historical motion detection events before the current timestamps of several current motion detection events corresponding to image data acquired by the image sensor, the relationship between the current timestamps of several current motion detection events and the historical timestamps of historical motion detection events is determined. In response to the relationship between the current timestamps of several current motion detection events and the historical timestamps of historical motion detection events satisfying the second preset time, the image sensor is adjusted to the image output mode. In some application scenarios, this is in response to the time interval between the current timestamp of at least one current motion detection event and the historical timestamp of a historical motion detection event being greater than or equal to the second preset time. In other application scenarios, this is in response to the time interval between the current timestamp of the first current motion detection event and the historical timestamp of a historical motion detection event being greater than or equal to the second preset time. The current timestamp of the first current motion detection event can be the time of the first motion detection event among the several current timestamps corresponding to several current motion detection events. A historical animal detection event can be an animal detection event that is adjacent to the first current animal detection event and is located before the current timestamp corresponding to the first current animal detection event.

It is understandable that the first preset time and the second preset time can be the same time or different times.

It can be assumed that if there is no current motion detection event in the image processing data, the image sensor directly enters a sleep state and waits for a timed wake-up to re-detect the motion detection event. This can reduce the wake-up time of the image sensor, so that the image sensor is not always in a wake-up state. This can save the power consumption of each module in the image acquisition device, thereby improving the battery efficiency of the image acquisition device.

In some embodiments, the image processing method of the image acquisition device may further include the following steps: in response to the first motion detection result satisfying a third preset condition, controlling the image sensor to enter a sleep state, wherein the third preset condition includes the case that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the time interval between the current timestamp corresponding to all current motion detection events in the first motion detection result and the historical timestamp of the historical motion detection event is less than the third preset time.

In some application scenarios, the third preset condition includes the following: if there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the time interval between the current timestamp corresponding to all current motion detection events in the first motion detection result and the historical timestamp of the historical motion detection events is less than the third preset time. It can be understood that, in the case where there are historical motion detection events before the current timestamp of several current motion detection events corresponding to image data acquired by the image sensor, the relationship between the current timestamps of several current motion detection events and the historical timestamps of historical motion detection events is determined. In response to the relationship between the current timestamps of several current motion detection events and the historical timestamps of historical motion detection events satisfying the third preset time, the image sensor is controlled to enter a sleep state. In some application scenarios, this is in response to the time interval between the current timestamp of each current motion detection event and the historical timestamp of each historical motion detection event being less than the third preset time. In other application scenarios, this is in response to the current timestamp of the last current motion detection event... The time interval between the current timestamp of a current motion detection event and a previous one is less than a third preset time. The current timestamp of the last current motion detection event can be the time of the last motion detection event among several current timestamps corresponding to several current motion detection events. A historical motion detection event can be a motion detection event that is adjacent to the first current motion detection event and precedes the current timestamp corresponding to the first current motion detection event among several historical motion detection events.

It is understandable that the third preset time can be the same as or different from the first and second preset times.

It can be assumed that when there are current motion detection events in the image processing data, the relationship between several current motion detection events and historical motion detection events can be determined, or the relationship between several current motion detection events can be determined, thereby determining whether the image sensor is in the image output mode or the sleep mode. This ensures that the image sensor is not always in the wake-up state or always in the motion detection mode, which can save the power consumption of each image sensor in the image acquisition device, thereby improving the battery efficiency of the image acquisition device.

In some embodiments, prior to step S11 described above, the image processing method of the image acquisition device may further include the following steps: First, in response to the image sensor being powered on, a wake-up condition is set for the image sensor. Then, the image sensor is controlled to switch states at a predetermined frequency. The wake-up condition includes adjusting the image sensor from a dormant state to a working state after a target wake-up time, wherein the image sensor in the working state can be used to acquire image data.

In response to the image sensor being powered on, a wake-up condition is set for the image sensor. Subsequently, the image sensor is controlled to switch states at a predetermined frequency. Setting the wake-up condition for the image sensor can mean that the image sensor switches from a sleep state to an active state within a target wake-up time. Specifically, after the image sensor switches from a sleep state to an active state within the target wake-up time, the image sensor enters motion detection mode. The wake-up condition includes adjusting the image sensor from a sleep state to an active state after a first preset time, wherein the image sensor in the active state can be used to acquire image data. The target wake-up time can be dynamically set according to the needs of image processing.

It can be assumed that setting wake-up conditions allows the image sensor to be woken up periodically, preventing it from remaining in a dormant state. This improves the efficiency of the image sensor in acquiring image data and detecting motion, thereby enhancing the image processing efficiency of the image acquisition device.

In some embodiments, before the processor receives image data from the image sensor in low-frequency mode, the image processing method of the image acquisition device may further include the following steps: adjusting the exposure parameters of the image acquisition device; and setting a wake-up condition for the adjusted image sensor so that the adjusted image sensor switches states at a predetermined frequency.

Exposure parameter adjustment can be described as exposure convergence. FastAe (Fast Auto Exposure) is a technique used in camera systems to quickly adjust exposure settings. The adjusted image sensor can find the optimal exposure settings as quickly as possible to adapt to changes in ambient lighting conditions. This ensures that the exposure of the image captured by the image sensor stabilizes within a short period of time.

It can be assumed that the adjusted image sensor switches states at a predetermined frequency, and the acquired image data is output stable data.

The above solution, in response to the image data received by the processor in low-frequency mode meeting the first image processing condition, adjusts the processor to high-frequency mode to start each functional module. The first image processing condition includes the image data containing relevant information about the target object and/or the number of image data reaching a preset quantity. It then controls each functional module to perform image processing on the image data. This is different from the prior art where each functional module is always in a low-frequency mode. By waiting for image data to be sent and performing image processing in a wake-up state, this application can reduce the wake-up time of each functional module during the image processing of the image acquisition device, thereby reducing the battery power consumption of the image acquisition device.

Please refer to Figure 2, which is another schematic flowchart of an embodiment of the image processing method of the image acquisition device of this application.

Steps S201, S202, S203, and S204 are executed sequentially. Step S201: The image sensor is powered on. Step S202: The exposure parameters of the image acquisition device are adjusted. Step S203: A wake-up condition is set for the image sensor. Step S204: The image sensor enters motion detection mode, detects the acquired image data, and obtains a motion detection result. The motion detection result is determined to be either a first motion detection result or a second motion detection result. If the motion detection result is the first motion detection result, step S205 is executed. Step S205: In response to the first motion detection result meeting the first preset condition, the image sensor is adjusted to the image output mode. If the motion detection result is the second motion detection result, steps S206, S207, and S208 are executed sequentially. Step S206: In response to the motion detection result being the second motion detection result, the image sensor is controlled to enter a sleep state. Step S207: It is determined whether the image sensor meets the wake-up condition. Step S208: The image sensor remains in sleep mode.

It can be argued that controlling the image sensor to enter sleep mode and working mode under different conditions can reduce the number of times the image sensor is falsely woken up, thereby improving the battery efficiency of the image acquisition device.

Please refer to Figure 3, which is another schematic flowchart of an embodiment of the image processing method of the image acquisition device of this application. As shown in Figure 3, the image processing flow of the image acquisition device may include the following steps:

Steps S301 and S302 are executed sequentially. Step S301: The processor receives image data from the image sensor in image output mode in low-frequency mode. Step S302: The video cropping module crops the image data to obtain data to be processed. It is determined whether the data to be processed meets the first image condition and whether it meets the second image processing condition. If the data to be processed meets the first image processing condition, steps S303, S304, and S305 are executed sequentially. Step S303: In response to the received data to be processed meeting the first image processing condition, the processor is adjusted to high-frequency mode. Step S304: Each functional module is controlled to perform image processing on the data to be processed to obtain video data. Step S305: In response to the completion of video data storage, the memory is adjusted to low-power mode, and the image sensor is controlled to enter sleep mode. If the data to be processed meets the second image processing condition, steps S306 and S307 are executed sequentially. Step S306: In response to the received image data meeting the second image processing condition, the data to be processed is written to the memory in high-power mode. Step S307: In response to the completion of data storage, adjust the memory to a low-power mode and control the image sensor to enter a sleep state.

It can be argued that controlling the activation of various functional modules in an image acquisition device under different image processing conditions can reduce the activation of these modules by non-target objects, thereby improving the battery efficiency of the image acquisition device. Furthermore, switching between different power consumption modes for each module in the image acquisition device under varying usage requirements can further improve the battery efficiency of the image acquisition device.

The above solution, in response to the image data received by the processor in low-frequency mode meeting the first image processing condition, adjusts the processor to high-frequency mode to start each functional module. The first image processing condition includes the image data containing relevant information about the target object and/or the number of image data reaching a preset quantity. It then controls each functional module to perform image processing on the image data. Compared to the prior art where each functional module is constantly awake, waiting for image data to be sent and performing image processing, this application can reduce the need for image acquisition equipment. This reduces the wake-up time of each functional module during image processing, thereby reducing the battery consumption of the image acquisition device.

Please refer to Figure 4, which is a schematic diagram of the structure of an embodiment of the image processing device of the image acquisition equipment of this application. The image processing device 40 of the image acquisition equipment includes a receiving module 41, an adjustment module 42, and a control module 43. The receiving module 41 is used to receive image data from the image sensor in a low-frequency mode, wherein at least some functional modules in the low-frequency mode are in an inactive state; the adjustment module 42 is used to adjust the processor to a high-frequency mode to start each functional module in response to the received image data satisfying a first image processing condition, wherein the first image processing condition includes that the image data contains relevant information about the target object and/or the number of image data reaches a preset number; the control module 43 is used to control each functional module to perform image processing on the image data.

The above solution, in response to the image data received by the processor in low-frequency mode meeting the first image processing condition, adjusts the processor to high-frequency mode to start each functional module. The first image processing condition includes that the image data contains relevant information about the target object and/or the number of image data reaches a preset number. Control each functional module to perform image processing on the image data. Compared with the prior art where each functional module is always in a wake-up state waiting to send image data and perform image processing, this application can reduce the wake-up time of each functional module during the image processing of the image acquisition device, thereby reducing the battery power consumption of the image acquisition device.

For details on the functions performed by each module, please refer to the image processing methods of the image acquisition device; they will not be elaborated here.

Please refer to Figure 5, which is a schematic diagram of the structure of an embodiment of the electronic device of this application. The electronic device 50 includes a memory 51 and a processor 52. The processor 52 is used to execute program instructions stored in the memory 51 to implement the steps in the image processing method embodiment of the image acquisition device described above. In a specific implementation scenario, the electronic device 50 may include, but is not limited to, a multi-camera device, a microcomputer, or a server. In addition, the electronic device 50 may also include mobile devices such as laptops and tablets, which are not limited here.

Specifically, processor 52 controls itself and memory 51 to implement the steps in the image processing method embodiment of the image acquisition device described above. Processor 52 can also be called a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor can be a microprocessor or any conventional processor. Furthermore, processor 52 can be implemented using integrated circuit chips.

The above solution, in response to the image data received by the processor in low-frequency mode meeting the first image processing condition, adjusts the processor to high-frequency mode to start each functional module. The first image processing condition includes that the image data contains relevant information about the target object and/or the number of image data reaches a preset number. Control each functional module to perform image processing on the image data. Compared with the prior art where each functional module is always in a wake-up state waiting to send image data and perform image processing, this application can reduce the wake-up time of each functional module during the image processing of the image acquisition device, thereby reducing the battery power consumption of the image acquisition device.

Please refer to Figure 6, which is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. The computer-readable storage medium 60 stores program instructions 601 thereon, which, when executed by a processor, implement the steps in the image processing method embodiment of any of the above-described image acquisition devices.

The above solution, in response to the image data received by the processor in low-frequency mode meeting the first image processing condition, adjusts the processor to high-frequency mode to start each functional module. The first image processing condition includes that the image data contains relevant information about the target object and/or the number of image data reaches a preset number. Control each functional module to perform image processing on the image data. Compared with the prior art where each functional module is always in a wake-up state waiting to send image data and perform image processing, this application can reduce the wake-up time of each functional module during the image processing of the image acquisition device, thereby reducing the battery power consumption of the image acquisition device.

In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims (10)

An image processing method for an image acquisition device, characterized in that the image acquisition device includes an image sensor, a processor, and several functional modules, and the method includes: The processor receives image data from the image sensor in a low-frequency mode, wherein at least some functional modules in the low-frequency mode are in an inactive state; In response to the received image data satisfying a first image processing condition, the processor is adjusted to a high-frequency mode to start each of the functional modules. The first image processing condition includes the image data containing relevant information about the target object and/or the number of image data reaching a preset number. Control each of the aforementioned functional modules to perform image processing on the image data. According to the method of claim 1, the image acquisition device further includes a memory, and the method further includes: In response to the received image data satisfying a second image processing condition, the image data is written to the memory in a high-power mode. The second image processing condition includes that the image data does not contain information related to the target object and the number of image data does not reach the preset number. In response to the completion of image data storage, the memory is adjusted to a low-power mode, and the image sensor is controlled to enter a sleep state. The method according to claim 2, characterized in that, before adjusting the processor to a high-frequency mode in response to the received image data satisfying the first image processing condition, the method further includes: The image data is cropped to obtain the data to be processed; The data to be processed is subjected to object detection processing and/or quantity statistics processing to obtain data processing results. The data processing results include object detection results and/or quantity statistics results. The object detection results include information related to the detection of the target object in the image data or information related to the detection of the non-containment of the target object in the image data. The quantity statistics results include the total number of each image in the data to be processed. Based on the data processing results, determine whether the received image data meets the first image processing condition. The method according to any one of claims 1 to 3, characterized in that the operating states of the image sensor include a motion detection mode and an image output mode, and before the processor receives image data from the image sensor in low-frequency mode, the method further includes: The image sensor, which is in the motion detection mode, is controlled to detect the image data to obtain a motion detection result, which includes a first motion detection result about the image data containing information about a moving object. In response to the motion detection result being the first motion detection result, the image sensor is adjusted to the output mode so that the processor receives image data from the image sensor in a low-frequency mode. According to the method of claim 4, the first motion detection result includes a plurality of current motion detection events, the plurality of current motion detection events representing a plurality of objects to be confirmed in motion, and the step of adjusting the image sensor to the output mode in response to the motion detection result being the first motion detection result includes: In response to the first motion detection result satisfying a first preset condition, the image sensor is adjusted to the image output mode. The first preset condition includes that there are no historical motion detection events before the current timestamp corresponding to the current motion detection event in the first motion detection result, wherein the historical motion detection events indicate that the object to be confirmed was in motion before the current timestamp; or, In response to the first motion detection result satisfying the second preset condition, the image sensor is adjusted to the image output mode. The second preset condition includes the case that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the time interval between the current timestamps corresponding to at least one of the current motion detection events in the first motion detection result is greater than or equal to the second preset time. The method according to claim 5, characterized in that the method further comprises: In response to the first motion detection result satisfying a third preset condition, the image sensor is controlled to enter a sleep state. The third preset condition includes the condition that there is a historical motion detection event before the current timestamp corresponding to the current motion detection event in the first motion detection result, and the time interval between the current timestamp corresponding to all current motion detection events in the first motion detection result and the historical timestamp of the historical motion detection event is less than the third preset time. The method according to claim 4, wherein the motion detection result includes a second motion detection result regarding the image data not containing information related to a moving object, and the method further includes: In response to the motion detection result being the second motion detection result, the image sensor is controlled to enter the sleep state. The method according to any one of claims 1 to 3, characterized in that, before the processor receives image data from the image sensor in low-frequency mode, the method further comprises: In response to the power-on of the image sensor, a wake-up condition is set for the image sensor; The image sensor is controlled to switch states at a predetermined frequency. The wake-up condition includes adjusting the image sensor from a dormant state to a working state after a target wake-up time. The image sensor in the working state is capable of acquiring the image data. An electronic device, characterized in that it comprises: a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to perform the method as described in any one of claims 1-8. A computer-readable storage medium, characterized in that it comprises: storing a program file, which, when executed by a processor, is used to implement the method as described in any one of claims 1-8.