WO2024212750A1 - Image signal processing method and apparatus, device, and computer-readable storage medium - Google Patents

Abstract

Embodiments of the present disclosure relate to the technical field of image processing, and provide an image signal processing method and apparatus, a device, and a computer-readable storage medium. The method of the embodiments of the present disclosure comprises: acquiring a first image, and performing multi-channel feature extraction on the first image to obtain a multi-scale shared feature parameter; inputting the multi-scale shared feature parameter into each sub-task functional unit for multi-channel multi-task joint processing to obtain a multi-scale target feature parameter; and performing feature fusion on the multi-scale target feature parameter to obtain a fused feature parameter, and performing image construction according to the fused feature parameter, to obtain a second image for output.

Description

Image signal processing method, device, equipment and computer readable storage medium

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure is based on Chinese patent application CN202310387238.3 filed on April 12, 2023, entitled “Image Signal Processing Method, Device, Equipment and Computer-readable Storage Medium”, and claims the priority of the patent application, and all the contents disclosed therein are incorporated into the present disclosure by reference.

Technical Field

The present disclosure relates to the field of image processing technology, and in particular to an image signal processing method, apparatus, device and computer-readable storage medium.

Background Art

Image signal processing (ISP) is responsible for optimizing the output signal of the image sensor through a pipeline structure composed of a series of image processing modules in series. Traditional ISP technology is limited by image quality and the difficulty of adjusting ISP to generate high-quality results.

In recent years, some ISPs based on artificial intelligence (AI) have been proposed. The development of AI has promoted deep learning to replace the overall process or some module functions of traditional ISPs, breaking through the "ceiling" of traditional ISP technology. The progress of deep learning has brought the most advanced new methods to many image processing methods of traditional ISPs, such as image noise reduction and image enhancement.

However, in some cases, the various deep learning-based functional units (such as image denoising, tone mapping, image enhancement, etc.) in AI-ISP (i.e., artificial intelligence-based image signal processing) technology are independent of each other, resulting in a very large overall neural network structure. The large amount of computing and memory resource requirements restrict its deployment in electronic devices with limited hardware resources.

Summary of the invention

The main purpose of the embodiments of the present disclosure is to provide an image signal processing method, device, equipment and computer-readable storage medium, aiming to solve the technical problems of the current AI-ISP's huge neural network structure and high demand for computing resources.

To achieve the above-mentioned purpose, the present disclosure provides an image signal processing device, comprising: The end backbone network module is configured to obtain a first image, perform feature extraction on the first image, and obtain shared feature parameters; the core network module is configured to input the shared feature parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain target feature parameters, wherein the shared feature parameters are feature parameters with task correlation between the various subtask functional units; the back-end backbone network module is configured to construct an image according to the target feature parameters, and construct a second image for output, wherein the quality of the second image is higher than that of the first image.

In addition, to achieve the above-mentioned purpose, the embodiment of the present disclosure also provides an image signal processing method, including: acquiring a first image, performing feature extraction on the first image, and obtaining shared feature parameters; inputting the shared feature parameters into multiple preset sub-task function units for multi-task joint processing to obtain target feature parameters, wherein the shared feature parameters are feature parameters with task correlation between the various sub-task function units; constructing an image according to the target feature parameters, constructing a second image for output, wherein the quality of the second image is higher than that of the first image.

In addition, to achieve the above-mentioned purpose, an embodiment of the present disclosure also provides an image signal processing device, which includes: a memory, a processor, and an image signal processing program stored in the memory and executable on the processor, and when the image signal processing program is executed by the processor, it implements the image signal processing method as described above.

In addition, to achieve the above-mentioned purpose, an embodiment of the present disclosure further provides a computer-readable storage medium, on which an image signal processing program is stored, and when the image signal processing program is executed by a processor, the image signal processing method as described above is implemented.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

FIG1 is a schematic diagram of the structure of an image signal processing device according to an embodiment of the present disclosure;

FIG2 is a schematic diagram of the current end-to-end AI-ISP network architecture;

FIG3 is a schematic diagram of a feature-sharing AI-ISP network architecture according to an embodiment of the present disclosure;

FIG4 is a diagram of a neural network architecture for joint processing of noise reduction and enhancement according to an embodiment of the present disclosure;

FIG5 is a flowchart of a specific embodiment of the image signal processing method disclosed in the present invention;

FIG. 6 is a schematic diagram of the hardware structure of an image signal processing device involved in an embodiment of the present disclosure.

The realization of the objectives, functional features and advantages of the present disclosure will be further described in conjunction with embodiments and with reference to the accompanying drawings.

DETAILED DESCRIPTION

It should be understood that the specific embodiments described herein are only used to explain the present disclosure, and are not used to limit the present disclosure.

The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

It should be noted that all directional indications in the embodiments of the present disclosure (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

In the present disclosure, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

In addition, in the present disclosure, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present disclosure.

In recent years, some ISPs based on artificial intelligence (AI) have been proposed. The development of AI has promoted deep learning to replace the entire process or part of the traditional ISP. The module function breaks through the "ceiling" of traditional ISP technology. Advances in deep learning have brought state-of-the-art new methods to many image processing methods of traditional ISP, such as image denoising and image enhancement.

However, the various deep learning-based functional units (such as image denoising, tone mapping, image enhancement, etc.) in the existing AI-ISP (i.e., artificial intelligence-based image signal processing) technology are independent of each other, resulting in a very large overall neural network structure. The large amount of computing and memory resource requirements restrict its deployment in electronic devices with limited hardware resources.

Based on this, an embodiment of the present disclosure provides an image signal processing device, referring to FIG1 , which is a schematic diagram of the structure of the image signal processing device of the embodiment of the present disclosure. In this embodiment, the image signal processing device includes:

The front-end backbone network module 10 is configured to obtain a first image, perform feature extraction on the first image, and obtain shared feature parameters;

The core network module 20 is configured to input the shared characteristic parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain target characteristic parameters, wherein the shared characteristic parameters are characteristic parameters having task correlation between the subtask functional units;

The back-end backbone network module 30 is configured to construct an image according to the target feature parameters, and construct a second image for output, wherein the quality of the second image is higher than that of the first image.

In this embodiment, the front-end backbone network module 10 may be a backbone network module that uses a pyramid network as a feature extraction module, and the back-end backbone network module 30 may be a backbone network module that uses a pyramid network as a feature recovery module.

It should be noted that the core network module 20 is configured to input the shared feature parameters into a plurality of preset subtask function units for multi-task joint processing to obtain the target feature parameters. Among them, the types of the subtask function units include but are not limited to at least one of an image denoising unit, an image enhancement unit and a tone mapping unit. It is easy to understand that when the subtask function unit is an image denoising unit and an image mapping unit, the first image is the original image to be processed by image denoising and image mapping, and the second image is the output image after the image denoising and image mapping. For another example, when the subtask function unit is an image denoising unit and an image enhancement unit, the first image is the original image to be processed by image denoising and image enhancement, and the second image is the output image after the image denoising and image enhancement. For another example, when the subtask function unit is an image enhancement unit and a tone mapping unit, the first image is the original image to be processed by image enhancement and tone mapping, and the second image is the output image after the image enhancement and tone mapping. This embodiment The embodiment does not make any specific limitation to this. It is known that the quality of the second image is higher than that of the first image.

It is known to those skilled in the art that the shared characteristic parameter refers to a network characteristic parameter having similarity/correlation between different subtask functional units of the core network module 20, that is, the shared characteristic parameter is a characteristic parameter having task correlation between each subtask functional unit. The target characteristic parameter refers to a characteristic parameter output after the shared characteristic parameter is input into a plurality of preset subtask functional units for multi-task joint processing. To facilitate understanding, an example is given, where the preset subtask functional units are an image denoising unit and an image enhancement unit, and the target characteristic parameter refers to a characteristic parameter output after the shared characteristic parameter (the shared characteristic parameter is a characteristic parameter having task correlation between the image denoising unit and the image enhancement unit) is input into the image denoising unit for denoising task processing, and input into the image enhancement unit for enhancement task processing.

In the face of neural networks of different functional units (such as image denoising and image enhancement and other functional units) of the AI-ISP (artificial intelligence-Image Signal Process) architecture, the embodiments of the present disclosure achieve the reuse of similar network features among various sub-task functional units by extracting features (i.e., shared feature parameters) that are similar and relevant to the initial image, thereby sharing some neural network parameters, thereby reducing the network model and computing power requirements when performing multi-task joint processing in the embodiments of the present disclosure.

At present, the mainstream AI-ISP network architecture is to isolate sub-task functional units such as image denoising and image enhancement from each other, and use independent neural network models to implement each function. That is, the neural network model corresponding to each functional unit needs to start from the pixel level, perform feature extraction, task processing and feature recovery on the image, and then complete the end-to-end functional implementation of each module. As shown in Figure 2, Figure 2 is a schematic diagram of the current mainstream end-to-end AI-ISP network architecture. Among them, "Input" represents the input of the first image, "Output" represents the output of the second image, and task1, task2...task N represents each sub-task functional unit (such as image denoising unit, image enhancement unit and/or tone mapping unit).

The AI-ISP network architecture of the embodiment of the present disclosure is an AI-ISP network architecture that can share features in various sub-task functional units. As shown in Figure 3, Figure 3 is a schematic diagram of the feature-sharing AI-ISP network architecture of the embodiment of the present disclosure. Correspondingly, "Input" represents the input of the first image, "Output" represents the output of the second image, and task1, task2...task N represent various sub-task functional units (such as image denoising units, image enhancement units and/or tone mapping units). Compared to the current mainstream AI-ISP technology with completely independent modules, the embodiment of the present disclosure extracts data features through the backbone network, and the core networks of each task module reuse the same feature space, which reduces the movement of neural network parameters and feature data. This saves computing resources and improves the inference rate.

The disclosed embodiment extracts the features of the first image to obtain shared feature parameters, and inputs the shared feature parameters into a plurality of preset subtask function units for multi-task joint processing, thereby sharing some neural network parameters in the AI-ISP architecture, and each subtask function unit reuses the same feature space, which reduces the network structure and reduces the movement of feature data, thereby saving computing resources and improving the reasoning rate. That is, the disclosed embodiment extracts the data features of the initial image based on the front-end backbone network, and reduces the movement of neural network parameters and feature data by reusing the same feature space for each task core network, and then constructs the output image through the back-end backbone network module 30, thereby saving computing resources and improving the reasoning rate. In addition, the disclosed embodiment integrates the neural network architecture to a certain extent by using the subtask function units to share the same backbone network for feature extraction (i.e., feature extraction of the first image) and feature recovery (i.e., image construction according to the target feature parameters), which can avoid repeated feature calculations and data migration and transmission between different subtask function units, effectively improving the overall network operation efficiency, thereby enabling the disclosed embodiment to solve the technical problems of the current AI-ISP's huge neural network structure and high demand for computing resources.

In this embodiment, the ISP processes the image signal in the following aspects:

1. Correction and compensation: defective pixel correction (DPC), black level compensation (BLC), lens distortion correction (LSC), geometric correction for distortion, stretching, offset, etc., gamma correction, correction related to perspective principle, etc.

2. Denoising and image enhancement: time domain, spatial domain filtering, hierarchical compensation filtering, various noise removal and sharpening, suppression of ringing effect and banding artifacts, edge enhancement, brightness enhancement and contrast enhancement.

3. Color and format conversion: color interpolation, color space conversion, tone mapping, chroma adjustment, color correction, saturation adjustment, scaling and rotation, etc.

4. Adaptive processing: automatic white balance, automatic exposure, automatic focus and strobe detection, etc.

5. Visual recognition (face and gesture recognition) and image processing in extreme environments. Extreme environments include vibration, fast movement, darkness, and excessive brightness. The processing involved generally includes deblurring, point spread function estimation, brightness compensation, motion detection, dynamic capture, image stabilization, and high-dynamic range image (HDR) processing.

There are two mainstream implementation ideas for existing AI-ISP technology: one is to rely entirely on neural network computing. The mapping relationship between the front-end image sensor signal and the output image simulates the ISP full process to achieve end-to-end signal processing. This method is highly dependent on computing power and data sets, is highly blind and has poor interpretability, and cannot form a point-to-point solution for local color and contrast, texture, noise and other issues that specific modules are concerned about. The second is to AI the functional modules that best reflect the algorithm improvement effect in the ISP process (such as noise reduction, high dynamic lighting rendering and image enhancement, etc.), in order to focus limited computing power on the most critical and most visible functions to the human eye. This method only uses deep learning to process single-module tasks that are relatively complex and difficult in ISP and have obvious effect improvements, and the network design is more targeted and interpretable. However, in the existing AI-ISP technology, the functional modules based on deep learning are independent of each other, resulting in a very large overall neural network structure, and a large amount of computing and memory resource requirements limit its deployment in mobile devices with limited hardware resources. In fact, most of the existing research on image denoising, image enhancement and tone mapping, the neural network models designed use similar structures or methods, and the only difference between different tasks is often the error function. Therefore, in the neural networks of different functional modules of AI-ISP, the features extracted by convolutional networks are similar and relevant. By reusing similar network features and sharing some neural network parameters, it is possible to reduce network model and computing power requirements while achieving joint processing of multiple module tasks.

That is, the disclosed embodiment proposes a new method for joint multi-task processing of image signals. By sharing some neural network parameters, each task module reuses the same feature space, reduces the network structure, reduces the movement of feature data, and thus saves computing resources and improves the inference rate. Most AI-ISP technologies isolate tasks such as image denoising, tone mapping, and image enhancement from each other, and use independent neural network models to implement each function. Each neural network model starts from the pixel level, performs feature extraction, task processing, and feature recovery on the image, and then completes the end-to-end function implementation of each module. The disclosed embodiment realizes the joint processing of multiple module tasks based on balancing computing power and performance. On the one hand, in the disclosed embodiment, each task module shares the same backbone network for feature extraction and feature recovery, and to a certain extent, the neural network architecture is integrated, which can avoid repeated calculation of features and data migration and transmission between different tasks, and effectively improve the overall operation efficiency of the network. On the other hand, different subtask types can be decoupled at the feature level, and each task module has an independent neural network structure. During the training process, the parameters of the front-end backbone network and the forward task module neural network are fixed, and the separate training of each neural network can be completed.

In a possible implementation manner, the front-end backbone network module 10 includes a preset number of feature extraction network layers of different scales, the feature extraction network layer is configured to obtain a first image, perform Perform multi-channel feature extraction to obtain multi-scale shared feature parameters;

The core network module 20 includes a feature processing network layer connected to each feature extraction network layer in a one-to-one correspondence, each feature processing network layer includes a plurality of subtask functional units, and the feature processing network layer is configured to input multi-scale shared feature parameters into each subtask functional unit for multi-channel multi-task joint processing to obtain multi-scale target feature parameters;

The back-end backbone network module 30 includes an image restoration network layer connected to each feature processing network layer in a one-to-one correspondence. The image restoration network layer is configured to perform feature fusion on multi-scale target feature parameters to obtain fused feature parameters, and construct an image based on the fused feature parameters to construct a second image for output.

In this embodiment, the front-end backbone network module 10 may include a preset number of feature extraction network layers of different scales. For example, the front-end backbone network module 10 may include feature extraction network layers of 1/4 size, 1/16 size and 1/64 size, and the number of channels thereof are 16, 64 and 256, respectively. Among them, the feature extraction network layer of 1/4 size can be used to extract features of the first image at 1/4 size. The feature extraction network layer of 1/16 size can be used to extract features of the first image at 1/16 size. The feature extraction network layer of 1/64 size can be used to extract features of the first image at 1/64 size. Each feature extraction network layer can be composed of a 2D convolutional network with a convolution kernel size of 2*2 and a step size of 2.

Correspondingly, the core network module 20 includes a feature processing network layer connected to each feature extraction network layer in a one-to-one correspondence, each feature processing network layer includes a plurality of subtask functional units, and the feature processing network layer is configured to input multi-scale shared feature parameters into each subtask functional unit for multi-channel multi-task joint processing to obtain multi-scale target feature parameters. To help understand, an example is given, in which the front-end backbone network module 10 includes feature extraction network layers of 1/4 size, 1/16 size and 1/64 size, and the core network module 20 also includes three feature processing network layers. Among them, the feature extraction network layer of 1/4 size is connected to a feature processing network layer, the feature extraction network layer of 1/16 size is connected to a feature processing network layer, and the feature extraction network layer of 1/64 size is connected to a feature processing network layer. And each feature processing network layer contains multiple subtask functional units with the same structure. For example, each feature processing network layer includes an image denoising unit and an image enhancement unit.

Correspondingly, the back-end backbone network module 30 includes an image restoration network layer connected to each feature processing network layer in a one-to-one correspondence. The image restoration network layer is configured to perform feature fusion on multi-scale target feature parameters to obtain fused feature parameters, and to construct an image based on the fused feature parameters to construct a second image for output. To facilitate understanding, an example is given. In this example, the front-end backbone network module 30 includes an image restoration network layer connected to each feature processing network layer in a one-to-one correspondence. The image restoration network layer is configured to perform feature fusion on multi-scale target feature parameters to obtain fused feature parameters, and to construct an image based on the fused feature parameters to obtain a second image for output. The backbone network module 10 includes feature extraction network layers of 1/4 size, 1/16 size and 1/64 size, and the core network module 20 includes three feature processing network layers (respectively, the first feature processing network layer, the second feature processing network layer and the third feature processing network layer). Among them, the feature extraction network layer of 1/4 size is connected to the first feature processing network layer, the feature extraction network layer of 1/16 size is connected to the second feature processing network layer, and the feature extraction network layer of 1/64 size is connected to the third feature processing network layer. At this time, the back-end backbone network module 30 also includes three image restoration network layers, and the scale of each image restoration network layer corresponds one-to-one to the scale of each feature extraction network layer, that is, the three image restoration network layers also correspond to image restoration network layers of 1/4 size, 1/16 size and 1/64 size respectively. In this example, three connection links are specifically included, one of which is a 1/4 size feature extraction network layer, a first feature network layer and a 1/4 size feature restoration network layer connected in sequence. Another connection link is a 1/16-sized feature extraction network layer, a second feature network layer, and a 1/16-sized feature recovery network layer connected in sequence. Another connection link is a 1/64-sized feature extraction network layer, a third feature network layer, and a 1/64-sized feature recovery network layer connected in sequence.

In the embodiment of the present disclosure, the front-end backbone network module 10 is configured to include a preset number of feature extraction network layers of different scales, the feature extraction network layer is configured to obtain a first image, perform multi-channel feature extraction on the first image, and obtain multi-scale shared feature parameters, thereby performing multi-scale downsampling processing on the first image to extract image features of multiple numbers of channels, i.e., multi-scale shared feature parameters, and then the core network module 20 is configured to be a feature processing network layer connected to each feature extraction network layer in a one-to-one correspondence, each feature processing network layer includes multiple sub-task functional units, and the feature processing network layer is configured to input the multi-scale shared feature parameters into each sub-task functional unit for multi-channel multi-task joint processing, thereby obtaining image features after multi-task joint processing of multiple numbers of channels, i.e., multi-scale target feature parameters. Then, this embodiment sets the back-end backbone network module 30 as an image restoration network layer connected to each feature processing network layer in a one-to-one correspondence. The image restoration network layer is set to fuse the multi-scale target feature parameters to obtain fused feature parameters, and construct an image based on the fused feature parameters to construct a second image for output, thereby upsampling the fused feature parameters to obtain a second image that is consistent with the scale of the first image (i.e., restored to the size of the original image) and has undergone multi-task joint processing (such as image denoising and image enhancement).

Exemplarily, the types of subtask functional units include image denoising units and image enhancement units, wherein each feature processing network layer includes an image denoising unit and an image enhancement unit connected in sequence.

It is known that image noise reduction refers to the process of reducing noise in digital images, sometimes also called image denoising. Image enhancement refers to an image processing method that makes an originally unclear image clear or emphasizes certain interesting features, suppresses uninteresting features, improves image quality, enriches information, and enhances image interpretation and recognition effects.

This embodiment extracts image features with multiple numbers of channels by performing downsampling processing on the first image at multiple scales, thereby facilitating improving the image processing effect of performing multi-task joint processing (such as image denoising processing and image enhancement processing) on the first image.

In a possible implementation, the image signal processing apparatus further includes a training module (not shown), and the training module is configured to:

Acquire first training data, wherein the first training data includes a third image of first noise and a fourth image of second noise obtained from the third image, and the second noise is greater than the first noise;

The image denoising unit in the first training link is trained using the first training data to perform supervised learning on the first error function that performs denoising task processing in the image denoising unit until the first error function converges, thereby obtaining a trained image denoising unit, wherein the first training link is a front-end backbone network module 10, an image denoising unit, and a back-end backbone network module 30 that are sequentially connected;

After the image denoising unit training is completed, the first error function is fixed, and second training data is obtained, wherein the second training data includes a fifth image with a first resolution and a sixth image with a second resolution obtained from the fifth image, and the first resolution is greater than the second resolution;

The image enhancement unit in the second training link is trained using the second training data to perform supervised learning on the second error function that performs the enhancement task processing in the image enhancement unit until the second error function converges, thereby obtaining a trained image enhancement unit, wherein the second training link is a front-end backbone network module 10, an image denoising unit, an image enhancement unit, and a back-end backbone network module 30 connected in sequence.

In this embodiment, the first error function refers to the error function of the image noise reduction unit in performing noise reduction task processing, and the second error function refers to the error function of the image enhancement unit in performing enhancement task processing.

In this embodiment, the image denoising unit in the first training link is trained by using the first training data to supervise the first error function of the image denoising unit for denoising task processing until the first error function converges, thereby obtaining a trained image denoising unit, wherein the first training link is the front-end backbone network module 10, the image denoising unit and the back-end backbone network module 30 connected in sequence. Then, after the image denoising unit training is completed, the first error function is fixed and the image denoising unit is obtained. The second training data includes a fifth image of a first resolution and a sixth image of a second resolution obtained from the fifth image, wherein the first resolution is greater than the second resolution, and the second training data is used to train the image enhancement unit in the second training link to supervise the learning of the second error function for the enhancement task processing in the image enhancement unit until the second error function converges to obtain a trained image enhancement unit, wherein the second training link is a front-end backbone network module 10, an image denoising unit, an image enhancement unit and a back-end backbone network module 30 connected in sequence, so that the image denoising unit and the image enhancement unit can be decoupled at the feature level, and the parameters of the neural networks of the front-end backbone network and the forward task module are fixed during the training process to complete the separate training of the respective neural networks, that is, the image denoising unit can be trained by fixing the network parameters of the front-end backbone network module 10, wherein the data flow for training the image denoising unit does not pass through the network layer of the image enhancement unit, and then after the image denoising unit is trained, the image enhancement unit can be trained by fixing the network parameters of the front-end backbone network module 10 and the image denoising unit until the network parameters of the image enhancement unit converge.

Compared with the fully end-to-end AI-ISP technology, the embodiment of the present disclosure decouples different functional modules at the feature level. Each module has an independent neural network and can be trained on the basis of a fixed forward network, thereby optimizing the overall ISP performance from the module level.

That is, the mainstream AI-ISP technology isolates tasks such as image denoising and image enhancement from each other, and uses independent neural network models to implement each function. Each neural network model starts from the pixel level, performs feature extraction, task processing and feature recovery on the image, and then completes the end-to-end functional implementation of each module. In the disclosed embodiment, each task module shares the same backbone network for feature extraction and feature recovery, which can avoid repeated calculation of features and data migration and transmission between different tasks, and effectively improve the overall operation efficiency of the network. Different subtask types can be decoupled at the feature level. By fixing the parameters of the front-end backbone network and the forward task module neural network during training, the separate training of each neural network can be completed.

In order to help understand the technical concept or technical principle of the embodiment of the present disclosure, a specific embodiment is listed, with reference to FIG. 4 , which is a neural network architecture diagram of the joint processing of noise reduction and enhancement in the embodiment of the present disclosure, including:

In this specific embodiment, a pyramid network is used as the backbone network for feature extraction and restoration (i.e., the front-end backbone network module 10), and image denoising and image enhancement are used as two subtasks (i.e., subtask functional units) as examples to illustrate the specific implementation process of the disclosed embodiment. The neural network architecture (i.e., AI-ISP network architecture) shown in FIG4 includes a front-end backbone network (i.e., the front-end backbone network module 10), Core network (i.e. core network module 20) and back-end backbone network (i.e. back-end backbone network module 30). Among them, the front-end backbone network includes three layers of feature extraction networks, which respectively perform feature extraction on the original image (i.e. first image) at sizes of 1/4, 1/16 and 1/64 (i.e. perform multi-channel feature extraction on the first image), and the number of channels are 16, 64 and 256 respectively, and each layer of feature extraction network is composed of a 2D convolution network with a convolution kernel size of 2*2 and a step size of 2. The front-end backbone network processes the input image to obtain multi-scale features as the input of the core network. Similarly, the back-end backbone network includes three layers of feature recovery networks, and the number of channels are 256, 64 and 16 respectively, and each layer of feature extraction network is composed of a pixel_shuffle network, which is used to restore the features output by the task core network to the original size image. The denoising module (i.e., image denoising unit) and enhancement module (i.e., image enhancement unit) in the core network both adopt the UNET network structure, and also include 3 network blocks of the same scale as the backbone network. The specific number of network layers in each network block can be adjusted according to the specific situation. Here, each network block contains three layers of networks, and each layer of the network is composed of a depth-separable convolution and a relu activation function.

In this specific embodiment, the features of the forward network at 1/4, 1/16 and 1/64 scales are used as inputs for the core network subtask. The training process is divided into two steps:

In the first step, the input image (i.e., the first image) is processed by the front-end backbone network, the core network of the denoising module, and the back-end backbone network in turn, and supervised learning is performed using the error function L1loss (i.e., the first error function) of the denoising task, and the relevant network parameters are updated by the gradient descent method during back propagation. Note that in the first step, the data flow does not pass through the network layer of the core network of the enhancement module, but only through the front-end backbone network, the core network of the denoising module, and the back-end backbone network (i.e., the first training link).

In the second step, the network parameters of the front-end backbone network and the core network of the denoising module in the first step are fixed. The input image is processed by the front-end backbone network, the core network of the denoising module, the core network of the enhancement module, and the back-end backbone network in sequence (i.e., the second training link). The weighted parameters of the local contrast function and the color saturation function in the error function L1loss (i.e., the first error function) of the denoising task and the error function of the enhancement network (i.e., the second error function) are used for supervised learning, thereby updating the parameters of the core network of the enhancement module and fine-tuning the parameters of the back-end backbone network. Here, the enhancement core network directly processes the data features output by the denoising core network, rather than re-extracting features at the pixel level.

Among them, there are other subtasks in the multi-task neural network model shown in Figure 4 (for example, a tone mapping task unit can also be set in the core network, etc.), and the training method is the same as the second step. The parameters of the front-end backbone network and the trained forward core network are fixed, and the weighted error function of each task is used for supervised learning.

During the testing and reasoning phases, the input image will pass through the front-end backbone network, each task core network, and the back-end backbone network in sequence to obtain the final output, realizing multi-task joint processing in ISP.

It should be noted that the above specific embodiments are only used to help understand the technical concept of the embodiments of the present disclosure, and do not constitute a limitation on the image signal processing device of the present disclosure. More simple transformations based on the technical concept should all be within the protection scope of the present disclosure.

In an exemplary embodiment, the type of subtask functional unit also includes a tone mapping unit, wherein each feature processing network layer includes an image denoising unit, an image enhancement unit, and a tone mapping unit connected in sequence.

Those skilled in the art will know that tone mapping is a computer graphics technique that approximates the display of high dynamic range images on a limited dynamic range medium. Essentially, the problem that tone mapping solves is to perform a large contrast reduction to transform the scene brightness to a displayable range while maintaining image details and colors, which are very important for representing the original scene.

In this embodiment, the types of sub-task function units are further arranged to include a tone mapping unit, so that the image signal processing device of the disclosed embodiment can not only perform image noise reduction processing and image enhancement processing on the first image, but also perform tone mapping processing on the first image, thereby effectively expanding the image processing task function of the image signal processing device.

In a possible implementation, the type of subtask function module also includes a tone mapping unit, wherein each feature processing network layer includes an image denoising unit, an image enhancement unit, and a tone mapping unit connected in sequence, and the training module is set to:

After the image enhancement unit training is completed, the second error function is fixed, and third training data is obtained, wherein the third training data includes a seventh image with a first contrast and an eighth image with a second contrast obtained from the seventh image, and the first contrast is greater than the second contrast;

The tone mapping unit in the third training link is trained using the third training data to perform supervised learning on the third error function that performs mapping task processing in the tone mapping unit until the third error function converges to obtain a trained tone mapping unit, wherein the third training link is a front-end backbone network, an image denoising unit, an image enhancement unit, a tone mapping unit, and a back-end backbone network processing connected in sequence.

In this embodiment, the third error function refers to the error function used by the tone mapping unit to perform mapping task processing.

The present disclosure implements the method of further configuring the subtask function module to include a tone mapping unit. In the method, each feature processing network layer includes an image denoising unit, an image enhancement unit and a tone mapping unit connected in sequence, and the training module is set to fix the second error function after the image enhancement unit is trained, and obtain the third training data, wherein the third training data includes a seventh image of a first contrast and an eighth image of a second contrast obtained from the seventh image, and the first contrast is greater than the second contrast, and then the tone mapping unit in the third training link is trained using the third training data to supervise the learning of the third error function of the mapping task processing in the tone mapping unit until the third error function converges to obtain the trained tone mapping unit, wherein the third training link is a front-end backbone network, an image denoising unit, an image enhancement unit connected in sequence, , tone mapping unit and back-end backbone network processing, so that the image denoising unit, image enhancement unit and tone mapping unit can be decoupled at the feature level. During the training process, by fixing the parameters of the neural network of the front-end backbone network and the forward task module, the separate training of each neural network can be completed, that is, the image denoising unit can be trained by fixing the network parameters of the front-end backbone network module 10, wherein the data flow of training the image denoising unit does not pass through the network layer of the image enhancement unit, and then after the image denoising unit is trained, the image enhancement unit can be trained by fixing the network parameters of the front-end backbone network module 10 and the image denoising unit until the network parameters of the image enhancement unit converge, wherein the data flow of training the image enhancement unit does not pass through the network layer of the tone mapping unit. Then, the tone mapping unit can be trained by fixing the network parameters of the front-end backbone network module 10, the image denoising unit and the image enhancement unit until the network parameters of the tone mapping unit converge.

Compared with the fully end-to-end AI-ISP technology, the embodiment of the present disclosure decouples different functional modules at the feature level. Each module has an independent neural network and can be trained on the basis of a fixed forward network, thereby optimizing the overall ISP performance from the module level.

That is, the mainstream AI-ISP technology isolates tasks such as image denoising, image enhancement, and tone mapping from each other, and implements each function with independent neural network models. Each neural network model starts from the pixel level, performs feature extraction, task processing, and feature recovery on the image, and then completes the end-to-end functional implementation of each module. In the disclosed embodiment, each task module shares the same backbone network for feature extraction and feature recovery, which can avoid repeated calculation of features and data migration and transmission between different tasks, and effectively improve the overall operation efficiency of the network. Different subtask types can be decoupled at the feature level. By fixing the parameters of the front-end backbone network and the forward task module neural network during training, separate training of each neural network can be completed.

The device embodiments described above are merely illustrative, and the individual components described as separate components are The elements may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In various embodiments, some modules in the image processing device may be omitted, or additional modules may be included. In addition, the modules/elements according to various embodiments of the present disclosure may be combined to form a single entity, and thus may equivalently perform the functions of the corresponding modules/elements before the combination.

In addition, the present disclosure also provides an image signal processing method, with reference to FIG5 , which is a flowchart of a specific embodiment of the image signal processing method of the present disclosure. In this embodiment, the image signal processing method includes:

Step S10, acquiring a first image, performing feature extraction on the first image, and obtaining shared feature parameters;

Step S20, inputting the shared characteristic parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain target characteristic parameters, wherein the shared characteristic parameters are characteristic parameters having task correlation between the subtask functional units;

Step S30, constructing an image according to the target feature parameters, and constructing a second image for output, wherein the quality of the second image is higher than that of the first image.

In the face of the neural networks of different functional units of the AI-ISP architecture (such as image denoising and image enhancement functional units), the embodiments of the present disclosure achieve the reuse of similar network features among various sub-task functional units by extracting features of the initial image (i.e., sharing feature parameters) that are similar and relevant, thereby sharing some neural network parameters, thereby achieving the embodiment of the present disclosure when performing multi-task joint processing. Reduced network model and computing power requirements.

Compared with the current AI-ISP technology, which isolates sub-task functional units such as image denoising and image enhancement from each other and implements each function with their own independent neural network models, that is, the neural network model corresponding to each functional unit currently needs to start from the pixel level, perform feature extraction, task processing and feature recovery on the image, and then complete the end-to-end functional implementation of each module. The embodiment of the present disclosure extracts features from the first image to obtain shared feature parameters, and inputs the shared feature parameters to multiple preset sub-task functional units for multi-task joint processing, thereby realizing the sharing of some neural network parameters in the AI-ISP architecture. Each sub-task functional unit reuses the same feature space, which reduces the network structure and the movement of feature data, thereby indirectly saving computing resources and improving the inference rate. That is, the embodiment of the present disclosure extracts the data features of the initial image, and by integrating each task core The core network reuses the same feature space, reducing the movement of neural network parameters and feature data, and then constructs the output image through the back-end backbone network module, thereby saving computing resources and improving the reasoning rate. In addition, the disclosed embodiment integrates the neural network architecture to a certain extent by using the subtask functional units to share the same backbone network for feature extraction (i.e., feature extraction of the first image) and feature recovery (i.e., image construction based on the target feature parameters), which can avoid repeated calculation of features and migration and transmission of data between different subtask functional units, effectively improving the overall network operation efficiency, and thus enabling the disclosed embodiment to solve the technical problems of the current AI-ISP's huge neural network structure and high demand for computing resources.

In a possible implementation, the method further includes:

Step A10, acquiring a first image, performing multi-channel feature extraction on the first image, and obtaining multi-scale shared feature parameters;

Step A20, inputting the multi-scale shared feature parameters into each sub-task functional unit for multi-channel multi-task joint processing to obtain multi-scale target feature parameters;

Step A30, feature fusion is performed on the multi-scale target feature parameters to obtain fused feature parameters, and image construction is performed according to the fused feature parameters to construct a second image for output.

The disclosed embodiment acquires a first image, performs multi-channel feature extraction on the first image, obtains multi-scale shared feature parameters, and then performs multi-scale downsampling processing on the first image to extract multiple image features with different numbers of channels, i.e., multi-scale shared feature parameters, and then inputs the multi-scale shared feature parameters into each subtask functional unit for multi-channel multi-task joint processing, thereby obtaining multiple image features after multi-task joint processing with different numbers of channels, i.e., multi-scale target feature parameters. Then, the present embodiment performs feature fusion of the multi-scale target feature parameters to obtain fused feature parameters, and constructs an image based on the fused feature parameters to construct a second image for output, thereby performing upsampling processing on the fused feature parameters to obtain a second image that is consistent with the scale of the first image (i.e., restored to the size of the original image) and has been subjected to multi-task joint processing (e.g., image denoising and image enhancement).

Exemplarily, the types of the subtask functional units include an image noise reduction unit and an image enhancement unit, and the step of inputting the shared feature parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain the target feature parameters includes:

Step B10: input the shared feature parameters to the image denoising unit and the image enhancement unit connected in sequence to perform multi-task joint processing to obtain target feature parameters.

This embodiment extracts image features with multiple numbers of channels by performing downsampling processing on the first image at multiple scales, thereby facilitating improving the image processing effect of performing multi-task joint processing (such as image denoising processing and image enhancement processing) on the first image.

In one practicable manner, the method further comprises:

Step C10, acquiring first training data, wherein the first training data includes a third image with first noise and a fourth image with second noise obtained from the third image, and the second noise is greater than the first noise;

Step C20, training the image denoising unit using the first training data to perform supervised learning on a first error function that performs denoising task processing in the image denoising unit, until the first error function converges, thereby obtaining a trained image denoising unit;

Step C30, after the image denoising unit training is completed, fixing the first error function and acquiring second training data, wherein the second training data includes a fifth image of a first resolution and a sixth image of a second resolution obtained from the fifth image, and the first resolution is greater than the second resolution;

Step C40, using the second training data to train the image enhancement unit to perform supervised learning on the second error function that performs enhancement task processing in the image enhancement unit until the second error function converges, thereby obtaining a trained image enhancement unit.

In this embodiment, the image denoising unit in the first training link is trained by using the first training data to supervise the learning of the first error function for the denoising task in the image denoising unit until the first error function converges, thereby obtaining a trained image denoising unit. Then, after the image denoising unit is trained, the first error function is fixed, and the second training data is obtained, wherein the second training data includes a fifth image of a first resolution and a sixth image of a second resolution obtained from the fifth image, and the first resolution is greater than the second resolution. The image enhancement unit in the second training link is then trained by using the second training data to supervise the learning of the second error function for the enhancement task in the image enhancement unit until the second error function converges, thereby obtaining a trained image enhancement unit, so that the image denoising unit and the image enhancement unit can be decoupled at the feature level. During the training process, the parameters of the neural network of the front-end backbone network and the forward task module can be fixed to complete the separate training of the respective neural networks, that is, the image denoising unit can be trained by fixing the network parameters of the front-end backbone network module, wherein the data flow for training the image denoising unit does not pass through the network layer of the image enhancement unit, and then after the image denoising unit is trained, the front-end backbone network module can be fixed to complete the training of the image denoising unit. The network parameters of the network module and the image denoising unit are used to train the image enhancement unit until the network parameters of the image enhancement unit converge.

Compared with the fully end-to-end AI-ISP technology, the embodiment of the present disclosure decouples different functional modules at the feature level. Each module has an independent neural network and can be trained on the basis of a fixed forward network, thereby optimizing the overall ISP performance from the module level.

In a possible implementation manner, the type of the subtask functional unit further includes a tone mapping unit, and the step of inputting the shared characteristic parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain the target characteristic parameters includes:

Step D10, inputting the shared feature parameters into the image denoising unit, the image enhancement unit and the tone mapping unit connected in sequence to perform multi-task joint processing to obtain target feature parameters.

In this embodiment, the types of sub-task function units are further arranged to include a tone mapping unit, so that the image signal processing method of the disclosed embodiment can not only perform image noise reduction processing and image enhancement processing on the first image, but also perform tone mapping processing on the first image, thereby effectively expanding the image processing task function of the image signal processing device.

In an practicable manner, after the step of obtaining the trained image enhancement unit, the method further includes:

Fixing the second error function, and acquiring third training data, wherein the third training data includes a seventh image with a first contrast and an eighth image with a second contrast obtained from the seventh image, and the first contrast is greater than the second contrast;

The tone mapping unit is trained using the third training data to perform supervised learning on a third error function that performs mapping task processing in the tone mapping unit until the third error function converges, thereby obtaining a trained tone mapping unit.

In the embodiment of the present disclosure, after the training of the image enhancement unit is completed, the second error function is fixed and the third training data is obtained, wherein the third training data includes a seventh image of a first contrast and an eighth image of a second contrast obtained from the seventh image, and the first contrast is greater than the second contrast. Then, the third training data is used to train the tone mapping unit in the third training link, so as to supervise the learning of the third error function that performs mapping task processing in the tone mapping unit until the third error function converges, and a trained tone mapping unit is obtained, so that the image denoising unit, the image enhancement unit and the tone mapping unit can be decoupled at the feature level. During the training process, the parameters of the front-end backbone network and the forward task module neural network are fixed, and the training of each neural network can be completed. Separate training, that is, the image denoising unit can be trained by fixing the network parameters of the front-end backbone network module, wherein the data flow of the training image denoising unit does not pass through the network layer of the image enhancement unit, and then after the image denoising unit is trained, the image enhancement unit can be trained by fixing the network parameters of the front-end backbone network module and the image denoising unit until the network parameters of the image enhancement unit converge, wherein the data flow of the training image enhancement unit does not pass through the network layer of the tone mapping unit. Then the tone mapping unit can be trained by fixing the network parameters of the front-end backbone network module, the image denoising unit and the image enhancement unit until the network parameters of the tone mapping unit converge.

That is, the mainstream AI-ISP technology isolates tasks such as image denoising, image enhancement, and tone mapping from each other, and implements each function with independent neural network models. Each neural network model starts from the pixel level, performs feature extraction, task processing, and feature recovery on the image, and then completes the end-to-end functional implementation of each module. In the disclosed embodiment, each task module shares the same backbone network for feature extraction and feature recovery, which can avoid repeated calculation of features and data migration and transmission between different tasks, and effectively improve the overall operation efficiency of the network. Different subtask types can be decoupled at the feature level. By fixing the parameters of the front-end backbone network and the forward task module neural network during training, separate training of each neural network can be completed.

The image signal processing method provided in this embodiment and the image signal processing device provided in the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, reference can be made to the above-mentioned embodiments of the image signal processing device. This embodiment has the same beneficial effects as the various embodiments of the image signal processing device, which will not be repeated here.

In addition, the embodiment of the present disclosure also provides an image signal processing device, with reference to FIG6, which is a schematic diagram of the hardware structure of the image signal processing device involved in the embodiment of the present disclosure. As shown in FIG6, the image signal processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may optionally be a memory independent of the processor 1001. Storage equipment.

Those skilled in the art will appreciate that the structure shown in FIG6 does not limit the image signal processing device, and may include more or fewer components than shown, or combine certain components, or arrange components differently. As shown in FIG6 , the memory 1005 as a computer-readable storage medium may include an operating system, a data storage module, a network communication module, a user interface module, and an image signal processing program.

In the image signal processing device shown in FIG6 , the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in this embodiment can be set in the communication device, and the communication device calls the image signal processing program stored in the memory 1005 through the processor 1001, and executes the image signal processing method provided in any of the above embodiments.

The terminal proposed in this embodiment and the image signal processing method proposed in the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment can be referred to any of the above embodiments, and this embodiment has the same beneficial effects as executing the image signal processing method.

In addition, an embodiment of the present disclosure further proposes a computer storage medium, which may be a non-volatile computer-readable storage medium, on which an image signal processing program is stored, and when the image signal processing program is executed by a processor, the image signal processing method of the present disclosure as described above is implemented.

The various embodiments of the image signal processing device and the computer-readable storage medium disclosed in the present invention may refer to the various embodiments of the image signal processing method disclosed in the present invention, and will not be described in detail here.

It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

The serial numbers of the above-mentioned embodiments of the present disclosure are only for description and do not represent the advantages or disadvantages of the embodiments.

Through the above description of the implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. The essence of the technical solution or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM/RAM, disk, CD) as described above, and includes a number of instructions for enabling an image signal processing device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present disclosure.

The above are only preferred embodiments of the present disclosure, and are not intended to limit the patent scope of the present disclosure. Any equivalent structure or equivalent process transformation made using the contents of the present disclosure and the drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present disclosure.

Claims (13)

An image signal processing device, comprising: A front-end backbone network module is configured to obtain a first image, perform feature extraction on the first image, and obtain shared feature parameters; The core network module is configured to input the shared characteristic parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain target characteristic parameters, wherein the shared characteristic parameters are characteristic parameters having task correlation between the subtask functional units; The back-end backbone network module is configured to construct an image according to the target feature parameters, and construct a second image for output, wherein the quality of the second image is higher than that of the first image. The image signal processing device according to claim 1, wherein the front-end backbone network module comprises a preset number of feature extraction network layers of different scales, and the feature extraction network layer is configured to obtain a first image, perform multi-channel feature extraction on the first image, and obtain multi-scale shared feature parameters; The core network module includes a feature processing network layer connected to each feature extraction network layer in a one-to-one correspondence, each feature processing network layer includes a plurality of subtask functional units, and the feature processing network layer is configured to input multi-scale shared feature parameters into each subtask functional unit for multi-channel multi-task joint processing to obtain multi-scale target feature parameters; The back-end backbone network module includes an image restoration network layer connected to each feature processing network layer in a one-to-one correspondence. The image restoration network layer is configured to perform feature fusion on multi-scale target feature parameters to obtain fused feature parameters, and to construct an image based on the fused feature parameters to construct a second image for output. The image signal processing device as described in claim 1, wherein the types of the subtask functional units include an image denoising unit and an image enhancement unit, and wherein each feature processing network layer includes the image denoising unit and the image enhancement unit connected in sequence. The image signal processing device according to claim 3, wherein the image signal processing device further comprises a training module, and the training module is configured to: Acquire first training data, wherein the first training data includes a third image of first noise and a fourth image of second noise obtained from the third image, wherein the second noise is greater than the first noise; The image denoising unit in the first training link is trained using the first training data to Performing supervised learning on a first error function for performing denoising task processing in the image denoising unit until the first error function converges, thereby obtaining a trained image denoising unit, wherein the first training link is a front-end backbone network module, an image denoising unit, and a back-end backbone network module connected in sequence; After the image denoising unit training is completed, the first error function is fixed, and second training data is obtained, wherein the second training data includes a fifth image with a first resolution and a sixth image with a second resolution obtained from the fifth image, and the first resolution is greater than the second resolution; The image enhancement unit in the second training link is trained using the second training data to perform supervised learning on a second error function that performs enhancement task processing in the image enhancement unit until the second error function converges, thereby obtaining a trained image enhancement unit, wherein the second training link is a front-end backbone network module, an image denoising unit, an image enhancement unit, and a back-end backbone network module connected in sequence. The image signal processing device as described in claim 4, wherein the type of sub-task functional unit also includes a tone mapping unit, wherein each feature processing network layer includes the image denoising unit, the image enhancement unit and the tone mapping unit connected in sequence. A method for processing an image signal, comprising: Acquire a first image, perform feature extraction on the first image, and obtain shared feature parameters; Inputting the shared characteristic parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain target characteristic parameters, wherein the shared characteristic parameters are characteristic parameters with task correlation between the subtask functional units; An image is constructed according to the target feature parameters to obtain a second image for output, wherein the quality of the second image is higher than that of the first image. The image signal processing method according to claim 6, wherein the method further comprises: Acquire a first image, perform multi-channel feature extraction on the first image, and obtain multi-scale shared feature parameters; The multi-scale shared feature parameters are input into each sub-task functional unit for multi-channel multi-task joint processing to obtain multi-scale target feature parameters; The multi-scale target feature parameters are subjected to feature fusion to obtain fused feature parameters, and an image is constructed according to the fused feature parameters to obtain a second image for output. The image signal processing method according to claim 6, wherein the subtask function unit The types include image noise reduction units and image enhancement units, and the step of inputting the shared feature parameters into a plurality of preset subtask function units for multi-task joint processing to obtain target feature parameters includes: The shared feature parameters are input into the image denoising unit and the image enhancement unit connected in sequence to perform multi-task joint processing to obtain target feature parameters. The image signal processing method according to claim 8, wherein the method further comprises: Acquire first training data, wherein the first training data includes a third image of first noise and a fourth image of second noise obtained from the third image, and the second noise is greater than the first noise; Using the first training data to train an image denoising unit, so as to perform supervised learning on a first error function that performs denoising task processing in the image denoising unit, until the first error function converges, thereby obtaining a trained image denoising unit; After the image denoising unit training is completed, the first error function is fixed, and second training data is obtained, wherein the second training data includes a fifth image with a first resolution and a sixth image with a second resolution obtained from the fifth image, and the first resolution is greater than the second resolution; The image enhancement unit is trained using the second training data to perform supervised learning on a second error function that performs enhancement task processing in the image enhancement unit until the second error function converges, thereby obtaining a trained image enhancement unit. The image signal processing method according to claim 9, wherein the type of the subtask functional unit further includes a tone mapping unit, and the step of inputting the shared characteristic parameters into a plurality of preset subtask functional units for multi-task joint processing to obtain the target characteristic parameters comprises: The shared feature parameters are input into an image denoising unit, an image enhancement unit and a tone mapping unit connected in sequence to perform multi-task joint processing to obtain target feature parameters. The image signal processing method according to claim 10, wherein after the step of obtaining the trained image enhancement unit, the method further comprises: Fixing the second error function, and acquiring third training data, wherein the third training data includes a seventh image with a first contrast and an eighth image with a second contrast obtained from the seventh image, and the first contrast is greater than the second contrast; The tone mapping unit is trained using the third training data to perform supervised learning on a third error function that performs mapping task processing in the tone mapping unit until the third error function The training number converges and the trained tone mapping unit is obtained. An image signal processing device, comprising: a memory, a processor, and an image signal processing program stored in the memory and executable on the processor, wherein the image signal processing program, when executed by the processor, implements the image signal processing method according to any one of claims 6 to 11. A computer-readable storage medium, wherein an image signal processing program is stored on the computer-readable storage medium, and when the image signal processing program is executed by a processor, the image signal processing method according to any one of claims 6 to 11 is implemented.