CN116128999A - Image processing method, device, electronic device and storage medium - Google Patents

Abstract

The disclosure relates to an image processing method, device, electronic equipment, and storage medium, and relates to the technical fields of computer vision and deep learning. The method includes: obtaining an original image and a base image, wherein the original image includes a first object, and the base image includes a second object; performing semantic segmentation on the original image and the base image respectively, to obtain a first semantic layout image corresponding to the original image and The second semantic layout image corresponding to the base map; the first region in the first semantic layout image is fused with the second region in the second semantic layout image to obtain a fused semantic layout image, wherein the first region is different from the second Region: Based on the original image, base image and fused semantic layout image, the target image corresponding to the original image is obtained. By applying the present disclosure, the first area of the original image can be fused with the second area of the base image to obtain a natural converted image.

Description

Image processing method, device, electronic device and storage medium

technical field

The present application relates to the field of computer technology, specifically to the field of computer vision and deep learning technology, and in particular to an image processing method, device, electronic equipment and storage medium.

Background technique

With the development of science and technology, more and more application software has entered the life of users, gradually enriching the leisure life of users, such as short video APP and so on. Users can use video, photos, etc. to record their lives and upload them to the short video APP.

There are many special effects gameplay based on image algorithms and rendering technologies on the short video APP, among which the image synthesis algorithm is very popular with users because of its novelty and realism. For example, based on two pictures of a specific object, a relevant special effect of a target object having a head of one object and a body of another object is synthesized.

Contents of the invention

Embodiments of the present disclosure provide an image processing method, device, electronic equipment, and storage medium.

In a first aspect, an embodiment of the present disclosure provides an image processing method, including: acquiring an original image and a base image, wherein the original image includes a first object, and the base image includes a second object; Semantic segmentation, to obtain the first semantic layout image corresponding to the original image and the second semantic layout image corresponding to the base image; the first region in the first semantic layout image is fused with the second region in the second semantic layout image to obtain The semantic layout image is fused, wherein the first area is different from the second area; based on the original image, the base image, and the fused semantic layout image, a target image corresponding to the original image is obtained.

In a second aspect, an embodiment of the present disclosure provides an image processing device, including: an image acquisition unit configured to acquire an original image and a base image, wherein the original image includes a first object, and the base image includes a second object; the semantic The segmentation unit is configured to perform semantic segmentation on the original image and the base image respectively to obtain the first semantic layout image corresponding to the original image and the second semantic layout image corresponding to the base image; the semantic fusion unit is configured to convert the first semantic layout The first area in the image is fused with the second area in the second semantic layout image to obtain a fused semantic layout image, wherein the first area is different from the second area; the image processing unit is configured to be based on the original image and the base image And fuse the semantic layout image to get the target image corresponding to the original image.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including a memory, a processor, a bus, and a computer program stored in the memory and operable on the processor. When the processor executes the computer program, the The image processing method as described in the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the image processing method as described in the first aspect is implemented.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will be readily understood through the following description.

Description of drawings

The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present disclosure. in:

FIG. 1 is an exemplary system architecture diagram to which an embodiment of the image processing method of the present disclosure can be applied;

FIG. 2 is a schematic flow diagram of an embodiment of the image processing method of the present disclosure;

FIG. 3 is a schematic diagram of an application scenario of the image processing method of the present disclosure;

FIG. 4 is a schematic flowchart of another embodiment of the image processing method of the present disclosure;

5a and 5b are schematic diagrams of image processing of the image processing method of the present disclosure;

FIG. 6 is a schematic structural diagram of an embodiment of an image processing device of the present disclosure;

FIG. 7 is a schematic structural diagram of an embodiment of the electronic device of the present disclosure.

Detailed ways

It should be noted that the following detailed description is exemplary and intended to provide further explanation of the present disclosure. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

It should be noted that the terminology used herein is only for describing specific embodiments, and is not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly dictates otherwise, the singular is intended to include the plural, and it should also be understood that when the terms "comprising" and/or "comprising" are used in this specification, they mean There are features, steps, operations, means, components and/or combinations thereof.

In the case of no conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

In order to make the technical solutions and advantages of the present disclosure clearer, the present disclosure will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

It should be noted that the data related to the user obtained in this disclosure is obtained after obtaining the consent of the user or obtaining the full authorization of the user. The images participating in the model training obtained in the disclosure come from public data sets or have Obtain full authorization from the copyright owner of the image. In addition, the fused image mentioned in this disclosure is only a schematic representation of the fusion result of the two images, and does not represent any physical object. In addition, if the synthesized services involved in this disclosure may cause confusion or misidentification by the public, they have been marked prominently in the reasonable position and area of the generated or edited information content.

FIG. 1 shows an exemplary system architecture 100 to which embodiments of the image processing method or image processing apparatus of the present disclosure can be applied.

As shown in FIG. 1 , a system architecture 100 may include terminal devices 101 , 102 , 103 , a network 104 and a server 105 . The network 104 is used as a medium for providing communication links between the terminal devices 101 , 102 , 103 and the server 105 . Network 104 may include various connection types, such as wires, wireless communication links, or fiber optic cables, among others.

Users can use terminal devices 101 , 102 , 103 to interact with server 105 via network 104 to receive or send messages and the like. Various communication client applications, such as short video applications, image processing applications, etc., can be installed on the terminal devices 101, 102, and 103.

The terminal devices 101, 102, and 103 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they may be various electronic devices, including but not limited to smart phones, tablet computers, vehicle-mounted computers, laptop computers, desktop computers, and the like. When the terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. It can be implemented as a plurality of software or software modules (for example, to provide distributed services), or as a single software or software module. No specific limitation is made here.

The server 105 may be a server that provides various services, for example, a background server that provides support for short video applications installed on the terminal devices 101 , 102 , and 103 . The background server can obtain the original picture and the base picture input by the user through the terminal equipment 101, 102, 103, and replace the head of the original picture on the body of the base picture, and feed back the obtained head-changing image to the terminal equipment 101, 102, 103.

It should be noted that the server 105 may be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, for providing distributed services), or as a single software or software module. No specific limitation is made here.

It should be noted that, generally, the image processing method provided by the embodiment of the present disclosure may be executed by the terminal devices 101 , 102 , 103 , or may be executed by the server 105 . Correspondingly, the image processing apparatus can be set in the terminal devices 101 , 102 , 103 or in the server 105 .

It should be understood that the numbers of terminal devices, networks and servers in Fig. 1 are only illustrative. According to the implementation needs, there can be any number of terminal devices, networks and servers.

FIG. 2 shows a flow 200 of an embodiment of the image processing method of the present disclosure. As shown in Figure 2, the image processing method of this embodiment may include the following steps:

Step 201, obtaining the original image and the base image.

In this embodiment, the execution subject of the image processing method (such as the terminal devices 101, 102, 103 or the server 105 shown in FIG. 1) may acquire the original image and the base image from the user. Wherein, the original image includes the first object, and the base image includes the second object. The first object and the second object may belong to the same type, for example, both are portraits, or both are a certain animal. In some specific applications, both the first object and the second object are human bodies. The original image may be an image including a human head taken by the user through an image acquisition device installed in the terminal, and the base image may be an image including a human body crawled by the user through the network or stored locally.

In step 202, perform semantic segmentation on the original image and the base image respectively to obtain a first semantic layout image corresponding to the original image and a second semantic layout image corresponding to the base image.

After the execution subject obtains the original image and the base image, it can perform semantic segmentation on the original image and the base image respectively to obtain the first semantic layout image corresponding to the original image and the second semantic layout image corresponding to the base image. Specifically, the executive body can use a pre-trained semantic segmentation network to implement the above semantic segmentation, and the semantic segmentation network can include but not limited to: a full convolutional network, a deep convolutional encoding-decoding network for image segmentation.

The first semantic layout image and the second semantic layout image may include regions representing different semantics. For example, the first semantic layout image may include regions representing the head, regions representing the neck, regions representing the body, etc., and the second semantic layout image may include regions representing the neck, regions representing the body and limbs area etc. Regions with different semantics can be represented by different colors.

In step 203, the first region in the first semantic layout image is fused with the second region in the second semantic layout image to obtain a fused semantic layout image.

After obtaining the first semantic layout image and the second semantic layout image, the execution subject may extract the first area in the first semantic layout image and the second area in the second semantic layout image. Here, the first region may be at least one part of the first object, for example, may include the head and hair. The second area may be at least one part of the second object, for example, may include the body.

Then, the execution subject can fuse the above-mentioned first area with the second area, and the obtained image is called a fused semantic layout image. Specifically, the execution subject may input the first semantic layout image and the second semantic layout image into a pre-trained model to obtain a fused semantic layout image. Alternatively, the execution subject may directly join the second region in the second semantic layout image with the first region in the first semantic layout image. Or, the execution subject may remove the part outside the second area in the second semantic layout image, and move the first area of the first semantic layout image into the second semantic layout image.

Step 204, based on the original image, the base image and the fused semantic layout image, the target image corresponding to the original image is obtained.

After obtaining the fused semantic layout image, the execution subject can move the first area of the original image to the fused semantic layout image, and move the second area of the base image to the fused semantic layout image. The image processed in this way is used as the target image corresponding to the original image. In some specific practices, both the original image and the base image are portraits, the first area is the head area and the hair area, and the second area is the body area. After performing the above processing, the execution subject can also perform blurring or gradient processing on the connection between the head area of the original image and the body area of the base image, and the connection between the hair area of the original image and the body area of the base image. In order to make the processed image more natural.

It can be understood that the above-mentioned target image can be understood as a replacement part image of the base image, that is, the first area of the original image is replaced on the second area of the base image to achieve a conversion effect.

The image processing method provided by the above-mentioned embodiments of the present disclosure can combine the first area in the original image with the second area in the base image, so as to realize special conversion effects and improve user experience.

Continue to refer to FIG. 3 , which shows a schematic diagram of an application scenario of the image processing method according to the present disclosure. In the application scenario of FIG. 3 , after the user enters the original image 301 and the base image 302 in the short video application, a head-changing image 303 of the original image can be obtained. The obtained head-changing image 303 is real and natural, which improves entertainment.

Continue referring to FIG. 4 , which shows a flow 400 of another embodiment of the image processing method according to the present disclosure. As shown in Figure 4, the method in this embodiment may include the following steps:

Step 401, obtaining the original image and the base image.

In step 402, perform semantic segmentation on the original image and the base image respectively to obtain a first semantic layout image corresponding to the original image and a second semantic layout image corresponding to the base image.

Step 403: Determine the fused semantic layout image according to the first semantic layout image, the second semantic layout image and the pre-trained fusion model.

In this embodiment, after the execution subject obtains the first semantic layout image and the second semantic layout image, it can input them into a pre-trained fusion model, and the output of the model is the fusion semantic layout image. Here, the fusion model is used to fuse the first region in the first semantic layout image with the second region in the second semantic layout image. The fusion model can be various algorithms, for example, it can be a deep neural network, a convolutional neural network, and the like.

In this embodiment, the fusion model can be obtained through the following steps not shown in FIG. 4: acquiring a plurality of object images, wherein the object images include a third object of the same type as the first object or the second object; for each object image Perform random damage to obtain multiple sample object images; perform self-supervised learning according to each sample object image to obtain a fusion model.

First, the execution body needs to obtain multiple object images. Here, each object image includes a third object of the same type as the first object or the second object. That is, if the first object is a human body and the second object is an animal body, then the third object may be a human body or an animal body. It should be noted that the animal body of the third object is of the same type as the animal body of the second object, for example, both are dogs.

The execution subject can randomly destroy each object image to obtain multiple sample object images. Here, considering that some areas in the object image may be occluded, some areas need to be expanded when merging different object images, in order to simulate this scene during training, the acquired object images can be randomized during training. Damage, so that the model can learn to fill in the holes formed due to occlusion, or regenerate certain areas to achieve data enhancement. Taking the object image as a human body image as an example, sometimes hair covers the body, clothes cover the head, and the neck area needs to be re-expanded. In order to simulate this scene during training, the acquired human body image can be processed during training Random damage, so that the model can learn to fill in the holes formed by hair occlusion, or regenerate the neck area to achieve data enhancement.

The execution subject can perform self-supervised learning according to each sample object image, and train to obtain a fusion model. Self-supervised learning is a very effective feature learning method. Self-supervised learning assumes that the category labels of the training data are unknown, and under this premise, the characteristics of the training data are learned through the structural assumptions of the training data itself. After unsupervised pre-training of deep networks using self-supervised learning methods, deep networks can often achieve very good results in downstream computer vision tasks such as classification, monitoring, and segmentation. Specifically, the execution subject may use a self-supervised learning method to fuse the first region and the second region of the same object image. If the object image is a human body image, a self-supervised learning method can be used to fuse the head region, hair region, and body region of the human body image. Or, the execution subject can separate the semantic regions of the object image, and then use the self-supervised learning method to fuse the regions of the same image. According to the fused image and the original image, determine the loss function. The model is iteratively updated according to the value of the loss function to obtain the final fusion model.

It should be noted that the execution subject for training the fusion model and the execution subject of this embodiment may or may not be the same electronic device. If it is not the same electronic device, the executive body training the fusion model may send the trained fusion model to the execution body of this embodiment.

In some optional implementations of this embodiment, the executive body of training the fusion model can train the fusion model through the following steps: perform semantic segmentation on each sample object image, determine the sample semantic segmentation image; The first region is fused with the second region, and the initial fusion model is iteratively updated according to the fused image and the image of the sample object to obtain the fusion model.

In this implementation manner, each sample object image may first be semantically segmented to determine the sample semantically segmented image. Specifically, the execution subject can use the existing semantic segmentation algorithm to perform semantic segmentation on each sample object image to obtain each sample semantic segmentation image. Then, the execution subject may fuse the first region and the second region of the sample semantically segmented image to obtain a fused image. Then, according to the fused image and the sample object image, a loss function is determined. The parameters of the initial fusion model are then iteratively updated according to the loss function value. Until the training conditions are terminated, the final fusion model is obtained.

Step 404, adding noise to different semantic regions in the original image and the base image for multiple times.

The execution subject can add noise multiple times to different semantic regions of the original image and the base image. Specifically, the execution subject can add different noises to different semantic regions of the original image and the base image each time, such as adding random Gaussian noise. Or, add the same noise to different semantic regions of the original image and the base image, or add noise to only one semantic region at a time, etc.

In some optional implementations of this embodiment, the execution subject may add random Gaussian noise to the first area of the original image and the second area of the base image multiple times to obtain noise images respectively corresponding to the original image and the base image.

In this implementation manner, the execution subject may segment the image semantically, and determine the first area of the original image and the second area of the base image. Then, random Gaussian noise is added to the first region of the original image and the second region of the base image for multiple times to obtain noise images respectively corresponding to the original image and the base image. It can be understood that the noise image corresponding to the original image is a noise image set, and each noise image in the set corresponds to the added noise each time.

Step 405, according to the fused semantic layout image, perform multiple denoising processes on the noise image obtained after adding noise to generate a target image.

The execution subject can fuse different semantic regions in the original image after noise addition and the base image according to each semantic layout in the fused semantic layout image to obtain a fused image with noise added. Finally, the execution subject may denoise the above-mentioned fused noised image multiple times. The execution subject can use the pre-trained denoising network to achieve the above denoising. The execution subject may further directly use the denoised image as the target image, or perform further operations on the denoised image (such as smoothing processing, etc.) to finally obtain the target image.

Alternatively, the execution subject may splice the first region in the original image obtained after multiple noise additions and the second region in the base image obtained after multiple noise additions according to the semantic regions in the fused semantic layout image. Then denoise the spliced image multiple times, and use the denoised image directly as the processed image.

In some optional implementations of this embodiment, the executive body can implement denoising through the following steps: Before each denoising process, the first area in the noise image corresponding to the original image is compared with the noise image corresponding to the base image Synthesize the second region in the image to obtain a synthetic noise image; perform denoising processing on the synthetic noise image, and generate regions other than the first region and the second region in the fusion semantic layout image during the denoising process until the denoising process When the number of noise reaches the threshold, the target image is obtained.

In this implementation manner, before each denoising process, the first region in the noise image corresponding to the original image and the second region in the noise image corresponding to the base image may first be synthesized. Specifically, the execution subject may synthesize the two noise images obtained by the last noise addition to obtain a synthesized noise image. The synthetic noisy image can then be denoised. And in the process of denoising, generate regions other than the first region and the second region in the fused semantic layout image. Until the number of times of denoising reaches the threshold, the target image is obtained. It should be noted that, here, the generation of the regions other than the first region and the second region is directly obtained by the denoising process, that is, the above-mentioned regions except the first region and the second region will be directly generated during the denoising process. outside area.

In further processing, the execution subject may also consider a specific noise-added image before each denoising process. For example, the total number of noise additions is M, and in the N denoising process, the denoised image obtained in the last denoising process and the two noise images obtained in the M-N denoising processes need to be considered. Specifically, before the Nth denoising process, the execution subject can synthesize the image obtained by the N-1th denoising process and the noise image corresponding to the original image and the base image respectively obtained by the M-Nth denoising process, Get a synthetic noise image. Then, the Nth denoising process is performed on the above-mentioned synthesized noise image.

In some optional implementation manners of this embodiment, the second object also includes the first area. When performing the above synthesis, the first region in the noise image corresponding to the original image may be directly replaced with the first region in the noise image corresponding to the base image.

In this implementation, when the second object also includes the first region, the execution subject can directly replace the first region in the noise image corresponding to the original image with the first region in the noise image corresponding to the base image, so as to improve processing efficiency.

In a further implementation, before the Nth denoising process, the first area in the noise image corresponding to the original image obtained by the N-1th denoising process is replaced with the first region obtained by the N-1th denoising process The first area in the noise image corresponding to the base image is obtained to obtain a synthetic noise image, where N≥2.

In this implementation manner, before the N denoising process, the execution subject may first determine the noise image corresponding to the original image and the noise image corresponding to the base image obtained by the N-1 denoising process. Then, the first region in the former can be replaced with the first region in the latter to obtain a synthetic noise image. This can ensure that the effect of the first area in the target image is consistent with the effect of the second area.

When the object image is a human body image, the execution subject can add random Gaussian noise to the head region, hair region of the original image and body region of the base image multiple times to obtain noise images respectively corresponding to the original image and the base image. Then, before denoising the two noised images, you can first fuse the head area and hair area in the noise image corresponding to the original image with the body area in the noise image corresponding to the base image, which needs to be considered during fusion Information such as the contour of each semantic region in the semantic region image is fused. The fused image can be called a synthetic noise image. Then, the execution subject can perform denoising processing on the synthesized noise image until the number of times of denoising reaches a threshold to obtain the target image.

In this embodiment, the noise of the head region and hair region of the original image is preserved during the denoising process, while the noise of the body region of the base image is retained, so that the head region, hair region and Flawless reconstruction of body regions with detail preservation of heads, hair and clothing.

This embodiment is understood in conjunction with Fig. 5a and Fig. 5b, in Fig. 5a, the user enters the original image and the base image. After semantic segmentation, the semantic layout image corresponding to the original image and the semantic layout image corresponding to the base image can be obtained. By fusing the two semantically segmented images, a fused semantic layout image can be obtained. In Figure 5b, the target image can be obtained by inputting the original image, the base image, and the fused semantic layout image into the pre-trained model.

In the image processing method provided by the above-mentioned embodiments of the present disclosure, first, the fusion model is used to combine the semantic regions of two different people, which can be regarded as fusion at the semantic level. Then, using the pre-trained model and preserving the details of the original head, hair, and clothing, the final head-swapped result is generated, which can be viewed as a pixel-level fusion. Through these two stages, a natural head-changing image can be obtained.

Further referring to FIG. 6 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an image processing device. The device embodiment corresponds to the method embodiment shown in FIG. 2 , and the device can specifically Used in various electronic equipment.

As shown in FIG. 6 , the image processing device 600 of this embodiment includes: an image acquisition unit 601 , a semantic segmentation unit 602 , a semantic fusion unit 603 and an image processing unit 604 .

The image acquiring unit 601 is configured to acquire the original image and the base image. Wherein, the original image includes the first object, and the base image includes the second object.

The semantic segmentation unit 602 is configured to perform semantic segmentation on the original image and the base image respectively, to obtain a first semantic layout image corresponding to the original image and a second semantic layout image corresponding to the base image.

The semantic fusion unit 603 is configured to fuse the first region in the first semantic layout image with the second region in the second semantic layout image to obtain a fused semantic layout image. Wherein, the first area is different from the second area.

The image processing unit 604 is configured to obtain a target image corresponding to the original image based on the original image, the base image and the fused semantic layout image.

In addition, in the technical solution of the present application, an electronic device is also proposed.

Fig. 7 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

As shown in FIG. 7 , the electronic device may include a processor 701, a memory 702, a bus 703, and a computer program stored in the memory 702 and operable on the processor 701, wherein the processor 701 and the memory 702 complete the process through the bus 703. mutual communication. When the processor 701 executes the computer program, the steps of implementing the above method include, for example: acquiring an original image and a base image, wherein the original image includes a first object, and the base image includes a second object; Semantic segmentation is performed to obtain the first semantic layout image corresponding to the original image and the second semantic layout image corresponding to the base image; the first region in the first semantic layout image is fused with the second region in the second semantic layout image, A fused semantic layout image is obtained, wherein the first area is different from the second area; based on the original image, the base image, and the fused semantic layout image, a target image corresponding to the original image is obtained.

In addition, an embodiment of the present disclosure also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented, for example, including: acquiring the original image and A base image, wherein the original image includes a first object, and the base image includes a second object; the original image and the base image are respectively semantically segmented to obtain a first semantic layout image corresponding to the original image and a second semantic layout image corresponding to the base image ; The first region in the first semantic layout image is fused with the second region in the second semantic layout image to obtain a fusion semantic layout image, wherein the first region is different from the second region; based on the original image, the base map and The semantic layout image is fused to obtain the target image corresponding to the original image.

To sum up, in the technical solution of the present disclosure, the first area in the original image can be combined with the second area in the base image, so as to realize conversion effects and improve user experience.

The above descriptions are only preferred embodiments of the present disclosure, and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the present disclosure within the scope of protection.

Claims (12)

1. An image processing method, comprising:

obtaining an original image and a base image, wherein the original image comprises a first object, and the base image comprises a second object;

respectively carrying out semantic segmentation on the original image and the base image to obtain a first semantic layout image corresponding to the original image and a second semantic layout image corresponding to the base image;

fusing a first region in the first semantic layout image and a second region in the second semantic layout image to obtain a fused semantic layout image, wherein the first region is different from the second region;

and obtaining a target image corresponding to the original image based on the original image, the base image and the fusion semantic layout image.

2. The method of claim 1, wherein the fusing the first region in the first semantic layout image with the second region in the second semantic layout image to obtain a fused semantic layout image comprises:

determining a fusion semantic layout image according to the first semantic layout image, the second semantic layout image and a fusion model acquired in advance, wherein the fusion model is used for fusing a first area in the first semantic layout image with a second area in the second semantic layout image.

3. The method of claim 1, wherein the obtaining the target image corresponding to the artwork based on the artwork, the base image, and the fused semantic layout image comprises:

adding noise for different semantic areas in the original image and the base image for multiple times respectively;

and carrying out denoising treatment on the noise image obtained after adding noise for multiple times according to the fusion semantic layout image, and generating the target image.

4. A method according to claim 3, wherein said adding noise multiple times to different semantic regions in said artwork and said base map, respectively, comprises:

and adding random Gaussian noise to the first area of the original image and the second area of the base image for a plurality of times to obtain noise images respectively corresponding to the original image and the base image.

5. The method of claim 4, wherein the generating the target image by denoising the noise image obtained after adding noise multiple times according to the fused semantic layout image includes:

before each denoising process, synthesizing a first region in the noise image corresponding to the original image and a second region in the noise image corresponding to the base image to obtain a synthesized noise image;

denoising the synthesized noise image, and generating areas except the first area and the second area in the fused semantic layout image in the denoising process until the denoising times reach a threshold value to obtain the target image.

6. The method of claim 5, wherein the second object comprises the first region; and

the synthesizing the first region in the noise image corresponding to the original image and the second region in the noise image corresponding to the base image includes:

and replacing the first area in the noise image corresponding to the base map with the first area in the noise image corresponding to the original map.

7. The method of claim 6, wherein the synthesizing, before each denoising, the first region in the noise image corresponding to the original image and the second region in the noise image corresponding to the base image to obtain a synthesized noise image includes:

and before the Nth denoising treatment, replacing a first region in the noise image corresponding to the original image obtained by the last denoising with a first region in the noise image corresponding to the base image obtained by the last denoising, so as to obtain a synthesized noise image, wherein N is more than or equal to 2.

8. The method of claim 2, wherein the fusion model is obtained by:

acquiring a plurality of object images, wherein the object images comprise a third object with the same type as the first object or the second object;

randomly damaging each object image to obtain a plurality of sample object images;

and performing self-supervision learning according to each sample object image to obtain the fusion model.

9. The method of claim 8, wherein the third object comprises a first region and a second region; and

the self-supervision learning is performed according to each sample object image to obtain the fusion model, which comprises the following steps:

carrying out semantic segmentation on each sample object image to determine a sample semantic segmentation image;

and fusing the first region and the second region in the sample semantic segmentation image, and carrying out iterative updating on an initial fusion model according to the fused image and the sample object image to obtain the fusion model.

10. An image processing apparatus comprising:

an image acquisition unit configured to acquire an original image and a base image, wherein the original image includes a first object and the base image includes a second object;

the semantic segmentation unit is configured to perform semantic segmentation on the original image and the base image respectively to obtain a first semantic layout image corresponding to the original image and a second semantic layout image corresponding to the base image;

the semantic fusion unit is configured to fuse a first area in the first semantic layout image with a second area in the second semantic layout image to obtain a fused semantic layout image, wherein the first area is different from the second area;

and the image processing unit is configured to obtain a target image corresponding to the original image based on the original image, the base image and the fused semantic layout image.

11. An electronic device comprising a memory, a processor, a bus and a computer program stored on the memory and executable on the processor, wherein the processor implements the image processing method of any of claims 1 to 9 when the computer program is executed by the processor.

12. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when executed by a processor, implements the image processing method according to any one of claims 1 to 9.

Images