[go: up one dir, main page]

WO2024188171A1 - Procédé de traitement des images et son dispositif associé - Google Patents

Procédé de traitement des images et son dispositif associé Download PDF

Info

Publication number
WO2024188171A1
WO2024188171A1 PCT/CN2024/080717 CN2024080717W WO2024188171A1 WO 2024188171 A1 WO2024188171 A1 WO 2024188171A1 CN 2024080717 W CN2024080717 W CN 2024080717W WO 2024188171 A1 WO2024188171 A1 WO 2024188171A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
ldr
blocks
image blocks
processing
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/080717
Other languages
English (en)
Chinese (zh)
Inventor
汪海铃
李卫
席瑗苑
胡杰
陈汉亭
王云鹤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of WO2024188171A1 publication Critical patent/WO2024188171A1/fr
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20172—Image enhancement details
    • G06T2207/20208—High dynamic range [HDR] image processing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the embodiments of the present application relate to the field of artificial intelligence (AI) technology, and in particular, to an image processing method and related equipment.
  • AI artificial intelligence
  • HDR high dynamic range
  • a target object in a scene may be photographed at different exposure rates to obtain multiple low dynamic range (LDR) images of the target object. Then, the multiple LDR images may be input into a neural network model to fuse the multiple LDR images through the neural network model to obtain an HDR image of the target object.
  • LDR low dynamic range
  • the target object when shooting multiple LDR images, the target object may move in the scene, resulting in differences between the contents presented by the multiple LDR images. Therefore, the HDR image directly obtained by fusing the multiple LDR images is prone to artifacts.
  • the embodiment of the present application provides an image processing method and related equipment, which can achieve better fusion of multiple LDR images, so that the final HDR image does not have artifacts.
  • a first aspect of an embodiment of the present application provides an image processing method, which can be implemented by a target model.
  • the method includes:
  • the target object When it is necessary to obtain an HDR image of a target object, the target object may be photographed at different exposure rates, thereby acquiring a first LDR image of the target object and a second LDR image of the target object. It should be noted that the exposure rate used for photographing the second LDR image may be greater than the exposure rate used for photographing the first LDR image, or may be less than the exposure rate used for photographing the first LDR image.
  • the first LDR image of the target object and the second LDR image of the target object can be input into the target model.
  • the target model can perform image block matching on a plurality of first image blocks of the first LDR image of the target object and a plurality of second image blocks of the second LDR image of the target object, thereby establishing a one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks.
  • the target model can use the one-to-one correspondence between the multiple first image blocks and the multiple second image blocks to fuse the first LDR image and the second LDR image, thereby obtaining and outputting an HDR image of the target object.
  • the first LDR image of the target object and the second LDR image of the target object can be first collected, and the first LDR image and the second LDR image can be input into the target model.
  • the target model can match the image blocks of the first LDR image and the second LDR image, so as to obtain a one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image.
  • the target model can use the corresponding relationship to fuse the first LDR image and the second LDR image, so as to obtain and output the HDR image of the target object.
  • acquiring the correspondence between the plurality of first image blocks of the first LDR image and the plurality of second image blocks of the second LDR image comprises: based on the first LDR image and the second LDR image, acquiring the correspondence between the plurality of first image blocks of the first LDR image and the plurality of second image blocks of the second LDR image; The similarity between a plurality of first image blocks of an LDR image and a plurality of second image blocks of a second LDR image; based on the similarity, obtaining a one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks.
  • the target model may perform a series of processing on the first LDR image and the second LDR image, thereby obtaining the similarity between the plurality of first image blocks of the first LDR image and the plurality of second image blocks of the second LDR image.
  • the target model may use the similarity between the plurality of first image blocks and the plurality of second image blocks to accurately construct a one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks.
  • obtaining the similarity between the plurality of first image blocks of the first LDR image and the plurality of second image blocks of the second LDR image includes: extracting features from the first LDR image and the second LDR image to obtain first features of the plurality of first image blocks of the first LDR image and second features of the plurality of second image blocks of the second LDR image; and calculating the first features of the plurality of first image blocks and the second features of the plurality of second image blocks to obtain the similarity between the plurality of first image blocks and the plurality of second image blocks.
  • the target model after receiving the first LDR image of the target object and the second LDR image of the target object, extracts features from the first LDR image and the second LDR image respectively, thereby correspondingly obtaining first features of the plurality of first image blocks of the first LDR image and second features of the plurality of second image blocks of the second LDR image.
  • the target model may also calculate the first features of the plurality of first image blocks and the second features of the plurality of second image blocks to accurately obtain the similarity between the plurality of first image blocks and the plurality of second image blocks.
  • the plurality of first image blocks include a third image block, and based on the similarity, obtaining a one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks includes: based on the similarity between the third image block and the plurality of second image blocks, determining the second image block with the greatest similarity among the plurality of second image blocks as the fourth image block; and establishing a correspondence between the third image block and the fourth image block.
  • the target model may select the second image block with the greatest similarity among the plurality of second image blocks based on the similarity between the first image block and the plurality of second image blocks as the second image block most similar to the first image block (i.e., the aforementioned fourth image block). Then, the target model may establish a correspondence between the first image block and the second image block most similar to the first image block.
  • the target model may also perform the same operation as that performed on the first image block on the remaining first image blocks except the first image block among the plurality of first image blocks, so that a one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks may be finally obtained.
  • the first LDR image and the second LDR image are fused to obtain a high dynamic range HDR image of the target object, including: extracting features from the first LDR image and the second LDR image to obtain a third feature of multiple first image blocks of the first LDR image and a fourth feature of multiple second image blocks of the second LDR image; adjusting the order of the fourth features of the multiple second image blocks based on the correspondence to obtain the fourth features of the multiple second image blocks after the order is adjusted; processing the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the order is adjusted to obtain the HDR image of the target object.
  • the target model may also extract features from the first LDR image and the second LDR image to obtain the third features of the multiple first image blocks of the first LDR image and the fourth features of the multiple second image blocks of the second LDR image.
  • the target model can adjust the ordering of the fourth features of the multiple second image blocks based on the correspondence, thereby obtaining the fourth features of the multiple second image blocks after the ordering is adjusted.
  • the target model can perform a series of processing on the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the ordering is adjusted, thereby finally obtaining and externally outputting an HDR image of the target object.
  • processing the third feature of the plurality of first image blocks and the fourth feature of the plurality of second image blocks after adjustment and sorting to obtain an HDR image of the target object includes: processing the third feature of the plurality of first image blocks, the fourth feature of the plurality of second image blocks, and the fourth feature of the plurality of second image blocks after adjustment and sorting to obtain an HDR image of the target object.
  • the target model may perform a series of processing on the third feature of the plurality of first image blocks, the fourth feature of the plurality of second image blocks, and the fourth feature of the plurality of second image blocks after adjustment and sorting, so as to finally obtain and output the HDR image of the target object externally.
  • the aforementioned processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the target model can be based on the self-attention mechanism and the interactive attention mechanism to implement the first LDR image and the second LDR image.
  • Image fusion In the fusion process, the detail information of the first LDR image and the second LDR image can be effectively taken into account, so that the final HDR image maintains good details and has no artifacts.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • a second aspect of an embodiment of the present application provides a model training method, characterized in that the method includes: obtaining a first LDR image of a target object and a second LDR image of the target object, the first LDR image and the second LDR image being images obtained by photographing the target object based on different exposures; processing the first LDR image and the second LDR image by a model to be trained to obtain a high dynamic range HDR image of the target object, wherein the model to be trained is used to: obtain a one-to-one correspondence between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image based on the first LDR image and the second LDR image; based on the correspondence, fuse the first LDR image and the second LDR image to obtain an HDR image of the target object; and train the model to be trained based on the HDR image to obtain a target model.
  • the target model obtained by training the above method has an image processing function (for example, a function of fusing multiple LDR images into an HDR image, etc.). Specifically, when it is necessary to obtain an HDR image of the target object, the first LDR image of the target object and the second LDR image of the target object can be first collected, and the first LDR image and the second LDR image can be input into the target model. Then, the target model can match the image blocks of the first LDR image and the second LDR image, thereby obtaining a one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image.
  • an image processing function for example, a function of fusing multiple LDR images into an HDR image, etc.
  • the target model can use the corresponding relationship to fuse the first LDR image and the second LDR image, thereby obtaining and outputting the HDR image of the target object.
  • the process of obtaining the one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image it is equivalent to aligning the multiple first image blocks of the first LDR image with the multiple second image blocks of the second LDR image one by one in terms of content.
  • this correspondence as a guide to achieve the fusion between the first LDR image and the second LDR image, a better fusion can be achieved, so that the HDR image of the target object finally obtained does not have artifacts.
  • the model to be trained is used to: obtain the similarity between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image based on the first LDR image and the second LDR image; and obtain a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the model to be trained is used to: extract features from a first LDR image and a second LDR image to obtain first features of multiple first image blocks of the first LDR image and second features of multiple second image blocks of the second LDR image; and calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain similarities between the multiple first image blocks and the multiple second image blocks.
  • multiple first image blocks include a third image block
  • the model to be trained is used to: based on the similarity between the third image block and multiple second image blocks, determine the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establish a corresponding relationship between the third image block and the fourth image block.
  • the model to be trained is used to: extract features from the first LDR image and the second LDR image to obtain third features of multiple first image blocks of the first LDR image and fourth features of multiple second image blocks of the second LDR image; adjust the order of the fourth features of the multiple second image blocks based on the corresponding relationship to obtain fourth features of the multiple second image blocks after the order is adjusted; and process the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the order is adjusted to obtain an HDR image of the target object.
  • the model to be trained is used to process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting to obtain an HDR image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • a third aspect of an embodiment of the present application provides an image processing method, the method comprising: obtaining a first noisy image and a second noisy image of a target object, the first noisy image and the second noisy image being images obtained by photographing the target object based on different exposures; based on the first noisy image and the second noisy image, obtaining a one-to-one correspondence between multiple first image blocks of the first noisy image and multiple second image blocks of the second noisy image; based on the correspondence, fusing the first noisy image and the second noisy image to obtain a denoised image of the target object.
  • obtaining the correspondence between multiple first image blocks of the first noisy image and multiple second image blocks of the second noisy image includes: based on the first noisy image and the second noisy image, obtaining the similarity between the multiple first image blocks of the first noisy image and the multiple second image blocks of the second noisy image; based on the similarity, obtaining a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks.
  • obtaining the similarity between multiple first image blocks of the first noisy image and multiple second image blocks of the second noisy image includes: performing feature extraction on the first noisy image and the second noisy image to obtain first features of multiple first image blocks of the first noisy image and second features of multiple second image blocks of the second noisy image; and calculating the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain the similarity between the multiple first image blocks and the multiple second image blocks.
  • the multiple first image blocks include a third image block, and based on the similarity, obtaining a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks includes: based on the similarity between the third image block and the multiple second image blocks, determining the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establishing a correspondence between the third image block and the fourth image block.
  • the first noisy image and the second noisy image are fused to obtain a high dynamic range denoised image of the target object, including: extracting features from the first noisy image and the second noisy image to obtain a third feature of multiple first image blocks of the first noisy image and a fourth feature of multiple second image blocks of the second noisy image; based on the correspondence, adjusting the order of the fourth features of the multiple second image blocks to obtain the fourth features of the adjusted and sorted multiple second image blocks; processing the third features of the multiple first image blocks and the fourth features of the adjusted and sorted multiple second image blocks to obtain the denoised image of the target object.
  • processing the third feature of the multiple first image blocks and the fourth feature of the multiple second image blocks after adjustment and sorting to obtain a denoised image of the target object includes: processing the third feature of the multiple first image blocks, the fourth feature of the multiple second image blocks, and the fourth feature of the multiple second image blocks after adjustment and sorting to obtain a denoised image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • a fourth aspect of an embodiment of the present application provides an image processing method, the method comprising: acquiring a first low-resolution image of a target object and a second low-resolution image of the target object, the first low-resolution image and the second low-resolution image being images obtained by photographing the target object based on different exposures; based on the first low-resolution image and the second low-resolution image, acquiring a one-to-one correspondence between multiple first image blocks of the first low-resolution image and multiple second image blocks of the second low-resolution image; based on the correspondence, fusing the first low-resolution image and the second low-resolution image to obtain a high-resolution image of the target object.
  • obtaining the correspondence between multiple first image blocks of the first low-resolution image and multiple second image blocks of the second low-resolution image includes: based on the first low-resolution image and the second low-resolution image, obtaining the similarity between the multiple first image blocks of the first low-resolution image and the multiple second image blocks of the second low-resolution image; based on the similarity, obtaining a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks.
  • obtaining the similarity between multiple first image blocks of the first low-resolution image and multiple second image blocks of the second low-resolution image includes: performing feature extraction on the first low-resolution image and the second low-resolution image to obtain first features of the multiple first image blocks of the first low-resolution image and second features of the multiple second image blocks of the second low-resolution image; and calculating the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain the similarity between the multiple first image blocks and the multiple second image blocks.
  • the multiple first image blocks include a third image block, and based on the similarity, obtaining a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks includes: based on the similarity between the third image block and the multiple second image blocks, determining the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establishing a correspondence between the third image block and the fourth image block.
  • the first low-resolution image and the second low-resolution image are fused to obtain a high-dynamic range high-resolution image of the target object, including: extracting features from the first low-resolution image and the second low-resolution image to obtain a third feature of multiple first image blocks of the first low-resolution image and a fourth feature of multiple second image blocks of the second low-resolution image; based on the correspondence, adjusting the order of the fourth features of the multiple second image blocks to obtain the fourth features of the adjusted and sorted multiple second image blocks; processing the third features of the multiple first image blocks and the fourth features of the adjusted and sorted multiple second image blocks to obtain a high-resolution image of the target object.
  • processing the third feature of the multiple first image blocks and the fourth feature of the multiple second image blocks after adjustment and sorting to obtain a high-resolution image of the target object includes: processing the third feature of the multiple first image blocks, the fourth feature of the multiple second image blocks, and the fourth feature of the multiple second image blocks after adjustment and sorting to obtain a high-resolution image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • a fifth aspect of an embodiment of the present application provides an image processing device, which includes a target model, and the device includes: a first acquisition module, used to acquire a first LDR image of a target object and a second LDR image of the target object, the first LDR image and the second LDR image being images obtained by photographing the target object based on different exposures; a second acquisition module, used to acquire, based on the first LDR image and the second LDR image, a one-to-one correspondence between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image; and a fusion module, used to fuse the first LDR image and the second LDR image based on the correspondence to obtain an HDR image of the target object.
  • a first acquisition module used to acquire a first LDR image of a target object and a second LDR image of the target object, the first LDR image and the second LDR image being images obtained by photographing the target object based on different exposures
  • a second acquisition module used to acquire, based on the
  • the first LDR image of the target object and the second LDR image of the target object can be first collected, and the first LDR image and the second LDR image can be input into the target model.
  • the target model can match the image blocks of the first LDR image and the second LDR image, so as to obtain a one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image.
  • the target model can use the corresponding relationship to fuse the first LDR image and the second LDR image, so as to obtain and output the HDR image of the target object.
  • the second acquisition module is used to: acquire the similarity between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image based on the first LDR image and the second LDR image; and acquire a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the second acquisition module is used to: perform feature extraction on the first LDR image and the second LDR image to obtain first features of multiple first image blocks of the first LDR image and second features of multiple second image blocks of the second LDR image; and calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain similarities between the multiple first image blocks and the multiple second image blocks.
  • the multiple first image blocks include a third image block
  • the second acquisition module is used to: based on the similarity between the third image block and the multiple second image blocks, determine the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establish a corresponding relationship between the third image block and the fourth image block.
  • the fusion module is used to extract features from the first LDR image and the second LDR image to obtain third features of multiple first image blocks of the first LDR image and fourth features of multiple second image blocks of the second LDR image; based on the corresponding relationship, adjust the order of the fourth features of the multiple second image blocks to obtain the adjusted fourth features of the multiple second image blocks;
  • the third feature of the plurality of first image blocks and the fourth feature of the plurality of second image blocks after adjustment and sorting are processed to obtain an HDR image of the target object.
  • the fusion module is used to process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting to obtain an HDR image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • a sixth aspect of an embodiment of the present application provides a model training device, which includes: an acquisition module, used to acquire a first LDR image of a target object and a second LDR image of the target object, the first LDR image and the second LDR image being images obtained by photographing the target object based on different exposures; a processing module, used to process the first LDR image and the second LDR image through a model to be trained to obtain a high dynamic range HDR image of the target object, wherein the model to be trained is used to: based on the first LDR image and the second LDR image, obtain a one-to-one correspondence between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image; based on the correspondence, fuse the first LDR image and the second LDR image to obtain an HDR image of the target object; and a training module, used to train the model to be trained based on the HDR image to obtain a target model.
  • the target model obtained by training the above-mentioned device has an image processing function (for example, a function of fusing multiple LDR images into an HDR image, etc.). Specifically, when it is necessary to obtain an HDR image of the target object, the first LDR image of the target object and the second LDR image of the target object can be first collected, and the first LDR image and the second LDR image can be input into the target model. Then, the target model can match the image blocks of the first LDR image and the second LDR image, thereby obtaining a one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image.
  • an image processing function for example, a function of fusing multiple LDR images into an HDR image, etc.
  • the target model can use the corresponding relationship to fuse the first LDR image and the second LDR image, thereby obtaining and outputting the HDR image of the target object.
  • the process of obtaining the one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image it is equivalent to aligning the multiple first image blocks of the first LDR image with the multiple second image blocks of the second LDR image one by one in terms of content.
  • this correspondence as a guide to achieve the fusion between the first LDR image and the second LDR image, a better fusion can be achieved, so that the HDR image of the target object finally obtained does not have artifacts.
  • the model to be trained is used to: obtain the similarity between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image based on the first LDR image and the second LDR image; and obtain a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the model to be trained is used to: extract features from a first LDR image and a second LDR image to obtain first features of multiple first image blocks of the first LDR image and second features of multiple second image blocks of the second LDR image; and calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain similarities between the multiple first image blocks and the multiple second image blocks.
  • multiple first image blocks include a third image block
  • the model to be trained is used to: based on the similarity between the third image block and multiple second image blocks, determine the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establish a corresponding relationship between the third image block and the fourth image block.
  • the model to be trained is used to: extract features from the first LDR image and the second LDR image to obtain third features of multiple first image blocks of the first LDR image and fourth features of multiple second image blocks of the second LDR image; adjust the order of the fourth features of the multiple second image blocks based on the corresponding relationship to obtain fourth features of the multiple second image blocks after the order is adjusted; and process the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the order is adjusted to obtain an HDR image of the target object.
  • the model to be trained is used to process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting to obtain an HDR image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • a seventh aspect of an embodiment of the present application provides an image processing device, which includes a target model, and the device includes: a first acquisition module, used to acquire a first noisy image of a target object and a second noisy image of the target object, the first noisy image and the second noisy image being images obtained by photographing the target object based on different exposures; a second acquisition module, used to acquire, based on the first noisy image and the second noisy image, a one-to-one correspondence between multiple first image blocks of the first noisy image and multiple second image blocks of the second noisy image; and a fusion module, used to fuse the first noisy image and the second noisy image based on the correspondence to obtain a denoised image of the target object.
  • the second acquisition module is used to: acquire the similarity between multiple first image blocks of the first noisy image and multiple second image blocks of the second noisy image based on the first noisy image and the second noisy image; and acquire a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the second acquisition module is used to: perform feature extraction on the first noisy image and the second noisy image to obtain first features of multiple first image blocks of the first noisy image and second features of multiple second image blocks of the second noisy image; calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain the similarity between the multiple first image blocks and the multiple second image blocks.
  • the multiple first image blocks include a third image block
  • the second acquisition module is used to: based on the similarity between the third image block and the multiple second image blocks, determine the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establish a corresponding relationship between the third image block and the fourth image block.
  • a fusion module is used to extract features from a first noisy image and a second noisy image to obtain a third feature of multiple first image blocks of the first noisy image and a fourth feature of multiple second image blocks of the second noisy image; based on the corresponding relationship, the sorting of the fourth features of the multiple second image blocks is adjusted to obtain the fourth features of the multiple second image blocks after the adjusted sorting; and the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the adjusted sorting are processed to obtain a denoised image of the target object.
  • the fusion module is used to process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting to obtain a denoised image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • An eighth aspect of an embodiment of the present application provides an image processing device, which includes a target model, and the device includes: a first acquisition module, used to acquire a first low-resolution image of a target object and a second low-resolution image of the target object, the first low-resolution image and the second low-resolution image being images obtained by photographing the target object based on different exposures; a second acquisition module, used to acquire, based on the first low-resolution image and the second low-resolution image, a one-to-one correspondence between multiple first image blocks of the first low-resolution image and multiple second image blocks of the second low-resolution image; and a fusion module, used to fuse the first low-resolution image and the second low-resolution image based on the correspondence to obtain a high-resolution image of the target object.
  • a first acquisition module used to acquire a first low-resolution image of a target object and a second low-resolution image of the target object, the first low-resolution image and the second low-resolution
  • the second acquisition module is used to: acquire the similarity between multiple first image blocks of the first low-resolution image and multiple second image blocks of the second low-resolution image based on the first low-resolution image and the second low-resolution image; and acquire a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the second acquisition module is used to: perform feature extraction on the first low-resolution image and the second low-resolution image to obtain first features of multiple first image blocks of the first low-resolution image and second features of multiple second image blocks of the second low-resolution image; and calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain similarities between the multiple first image blocks and the multiple second image blocks.
  • the plurality of first image blocks include a third image block
  • the second acquisition module is used to: based on the third image block The similarity between the third image block and the plurality of second image blocks is determined, among the plurality of second image blocks, the second image block with the greatest similarity is determined as the fourth image block; and a corresponding relationship between the third image block and the fourth image block is established.
  • a fusion module is used to extract features from a first low-resolution image and a second low-resolution image to obtain a third feature of multiple first image blocks of the first low-resolution image and a fourth feature of multiple second image blocks of the second low-resolution image; based on the corresponding relationship, adjust the order of the fourth features of the multiple second image blocks to obtain the fourth features of the multiple second image blocks after the order is adjusted; and process the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the order is adjusted to obtain a high-resolution image of the target object.
  • the fusion module is used to process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting to obtain a high-resolution image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • a ninth aspect of an embodiment of the present application provides an image processing device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code.
  • the image processing device executes the method described in the first aspect, any possible implementation of the first aspect, the third aspect, any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.
  • the tenth aspect of an embodiment of the present application provides a model training device, which includes a memory and a processor; the memory stores code, and the processor is configured to execute the code.
  • the model training device executes the method described in the second aspect or any possible implementation method of the second aspect.
  • the eleventh aspect of an embodiment of the present application provides a circuit system, which includes a processing circuit, and the processing circuit is configured to execute the method described in the first aspect, any possible implementation of the first aspect, the second aspect, any possible implementation of the second aspect, the third aspect, any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.
  • the twelfth aspect of an embodiment of the present application provides a chip system, which includes a processor for calling a computer program or computer instructions stored in a memory so that the processor executes a method as described in the first aspect, any possible implementation of the first aspect, the second aspect, any possible implementation of the second aspect, the third aspect, any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.
  • the processor is coupled to the memory through an interface.
  • the chip system also includes a memory, in which a computer program or computer instructions are stored.
  • a thirteenth aspect of an embodiment of the present application provides a computer storage medium, which stores a computer program.
  • the program When the program is executed by a computer, the computer implements the method described in the first aspect, any possible implementation of the first aspect, the second aspect, any possible implementation of the second aspect, the third aspect, any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.
  • a fourteenth aspect of an embodiment of the present application provides a computer program product, which stores instructions, which, when executed by a computer, enable the computer to implement the method described in the first aspect, any possible implementation of the first aspect, the second aspect, any possible implementation of the second aspect, the third aspect, any possible implementation of the third aspect, the fourth aspect, or any possible implementation of the fourth aspect.
  • a first LDR image of the target object and a second LDR image of the target object may be first collected, and the first LDR image and the second LDR image may be input into the target model.
  • the target model may perform image block matching on the first LDR image and the second LDR image, thereby obtaining a one-to-one correspondence between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image.
  • the target model may use the correspondence to fuse the first LDR image and the second LDR image, thereby obtaining and outputting the HDR image of the target object.
  • FIG1 is a schematic diagram of a structure of an artificial intelligence main framework
  • FIG2a is a schematic diagram of a structure of an image processing system provided in an embodiment of the present application.
  • FIG2b is another schematic diagram of the structure of the image processing system provided in an embodiment of the present application.
  • FIG2c is a schematic diagram of an image processing related device provided in an embodiment of the present application.
  • FIG3 is a schematic diagram of the architecture of the system 100 provided in an embodiment of the present application.
  • FIG4 is a schematic diagram of a structure of a target model provided in an embodiment of the present application.
  • FIG5 is a schematic diagram of a flow chart of an image processing method provided in an embodiment of the present application.
  • FIG6 is a schematic diagram of a block search network provided in an embodiment of the present application.
  • FIG7 is another schematic diagram of the structure of a block search network provided in an embodiment of the present application.
  • FIG8 is a schematic diagram of a structure of a fusion transformer network provided in an embodiment of the present application.
  • FIG9 is another schematic diagram of the structure of the fusion transformer network provided in an embodiment of the present application.
  • FIG10 is a schematic diagram of a structure of a local reconstruction transformer network provided in an embodiment of the present application.
  • FIG11 is a schematic diagram of a structure of a self-attention mechanism module or an interactive attention mechanism module provided in an embodiment of the present application;
  • FIG12 is a schematic diagram of a structure of a transformer module provided in an embodiment of the present application.
  • FIG13a is a schematic diagram of a comparison result provided in an embodiment of the present application.
  • FIG13b is another schematic diagram of the comparison results provided in an embodiment of the present application.
  • FIG14 is another schematic diagram of comparison results provided in an embodiment of the present application.
  • FIG15 is another schematic diagram of comparison results provided in an embodiment of the present application.
  • FIG16 is a flow chart of a model training method provided in an embodiment of the present application.
  • FIG18 is a schematic diagram of a structure of a model training device provided in an embodiment of the present application.
  • FIG19 is a schematic diagram of a structure of an execution device provided in an embodiment of the present application.
  • FIG20 is a schematic diagram of a structure of a training device provided in an embodiment of the present application.
  • FIG. 21 is a schematic diagram of the structure of a chip provided in an embodiment of the present application.
  • the embodiment of the present application provides an image processing method and related equipment, which can achieve better fusion of multiple LDR images, so that the final HDR image does not have artifacts.
  • a target object in a scene may be photographed at different exposure rates to obtain multiple LDR images of the target object.
  • the multiple LDR images may be input into a neural network model to fuse the multiple LDR images through the neural network model to obtain an HDR image of the target object.
  • three LDR images of the same scene may be collected first.
  • the three LDR images are photographed at three exposure levels.
  • an HDR image with optimized image indicators such as color, brightness and contrast may be obtained.
  • the target object when taking multiple LDR images, the target object may move in the scene, resulting in differences between the contents presented by the multiple LDR images taken. Therefore, the neural network model is directly based on the HDR image obtained by fusing these multiple LDR images, which is prone to artifacts.
  • AI technology is a technical discipline that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence. AI technology obtains the best results by sensing the environment, acquiring knowledge and using knowledge.
  • artificial intelligence technology is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.
  • Using artificial intelligence for data processing is a common application of artificial intelligence.
  • Figure 1 is a structural diagram of the main framework of artificial intelligence.
  • the following is an explanation of the above artificial intelligence theme framework from the two dimensions of "intelligent information chain” (horizontal axis) and “IT value chain” (vertical axis).
  • the "intelligent information chain” reflects a series of processes from data acquisition to processing. For example, it can be a general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, intelligent execution and output. In this process, the data has undergone a condensation process of "data-information-knowledge-wisdom".
  • the "IT value chain” reflects the value that artificial intelligence brings to the information technology industry from the underlying infrastructure of human intelligence, information (providing and processing technology implementation) to the industrial ecology process of the system.
  • the infrastructure provides computing power support for the artificial intelligence system, enables communication with the outside world, and is supported by the basic platform. It communicates with the outside world through sensors; computing power is provided by smart chips (CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips); the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc. For example, sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.
  • smart chips CPU, NPU, GPU, ASIC, FPGA and other hardware acceleration chips
  • the basic platform includes distributed computing frameworks and networks and other related platform guarantees and support, which can include cloud storage and computing, interconnected networks, etc.
  • sensors communicate with the outside world to obtain data, and these data are provided to the smart chips in the distributed computing system provided by the basic platform for calculation.
  • the data on the upper layer of the infrastructure is used to represent the data sources in the field of artificial intelligence.
  • the data involves graphics, images, voice, text, and IoT data of traditional devices, including business data of existing systems and perception data such as force, displacement, liquid level, temperature, and humidity.
  • Data processing usually includes data training, machine learning, deep learning, search, reasoning, decision-making and other methods.
  • machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.
  • Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.
  • Decision-making refers to the process of making decisions after intelligent information is reasoned, usually providing functions such as classification, sorting, and prediction.
  • some general capabilities can be further formed based on the results of the data processing, such as an algorithm or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.
  • Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical applications. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.
  • FIG2a is a schematic diagram of a structure of an image processing system provided in an embodiment of the present application, wherein the image processing system includes a user device and a data processing device.
  • the user device includes an intelligent terminal such as a mobile phone, a personal computer or an information processing center.
  • the user device is the initiator of image processing, and as the initiator of the image processing request, the request is usually initiated by the user through the user device.
  • the above-mentioned data processing device can be a device or server with data processing function such as a cloud server, a network server, an application server and a management server.
  • the data processing device receives the image processing request from the intelligent terminal through the interactive interface, and then performs image processing in the form of machine learning, deep learning, search, reasoning, decision-making, etc. through the memory for storing data and the processor for data processing.
  • the memory in the data processing device can be a general term, including local storage and a database for storing historical data.
  • the database can be on the data processing device or on other network servers.
  • the user device can receive the user's instruction, for example, the user device can obtain multiple images input/selected by the user, and then initiate a request to the data processing device, so that the data processing device performs an image fusion application on the multiple images obtained by the user device, thereby obtaining corresponding fusion results for the multiple images.
  • the user device can obtain multiple LDR images input by the user, and then initiate an image fusion request to the data processing device, so that the data processing device performs a series of processing on the multiple LDR images based on the image fusion request, thereby obtaining the processing results of the multiple LDR images, that is, the HDR image obtained based on the fusion of the multiple LDR images.
  • the data processing device may execute the image processing method according to the embodiment of the present application.
  • Figure 2b is another structural schematic diagram of the image processing system provided in an embodiment of the present application.
  • the user device directly serves as a data processing device.
  • the user device can directly obtain input from the user and directly process it by the hardware of the user device itself.
  • the specific process is similar to that of Figure 2a. Please refer to the above description and will not be repeated here.
  • the user device can receive instructions from the user. For example, the user device can obtain multiple LDR images input by the user, and then perform a series of processing on the multiple LDR images to obtain processing results of the multiple LDR images, that is, an HDR image obtained based on the fusion of the multiple LDR images.
  • the user equipment itself can execute the image processing method of the embodiment of the present application.
  • FIG. 2c is a schematic diagram of an image processing related device provided in an embodiment of the present application.
  • the user device in the above Figures 2a and 2b can specifically be the local device 301 or the local device 302 in Figure 2c
  • the data processing device in Figure 2a can specifically be the execution device 210 in Figure 2c
  • the data storage system 250 can store the data to be processed of the execution device 210
  • the data storage system 250 can be integrated on the execution device 210, and can also be set on the cloud or other network servers.
  • the processors in Figures 2a and 2b can perform data training/machine learning/deep learning through a neural network model or other models (for example, a model based on a support vector machine), and use the model finally trained or learned from the data to execute image processing applications on the image, thereby obtaining corresponding processing results.
  • a neural network model or other models for example, a model based on a support vector machine
  • FIG 3 is a schematic diagram of the system 100 architecture provided in an embodiment of the present application.
  • the execution device 110 is configured with an input/output (I/O) interface 112 for data interaction with an external device.
  • the user can input data to the I/O interface 112 through the client device 140.
  • the input data may include: various tasks to be scheduled, callable resources and other parameters in the embodiment of the present application.
  • the execution device 110 When the execution device 110 preprocesses the input data, or when the computing module 111 of the execution device 110 performs calculation and other related processing (such as implementing the function of the neural network in the present application), the execution device 110 can call the data, code, etc. in the data storage system 150 for the corresponding processing, and can also store the data, instructions, etc. obtained by the corresponding processing in the data storage system 150.
  • the I/O interface 112 returns the processing result to the client device 140 so as to provide it to the user.
  • the training device 120 can generate corresponding target models/rules based on different training data for different goals or different tasks, and the corresponding target models/rules can be used to achieve the above goals or complete the above tasks, thereby providing the user with the desired results.
  • the training data can be stored in the database 130 and come from the training samples collected by the data collection device 160.
  • the user can manually give input data, and the manual giving can be operated through the interface provided by the I/O interface 112.
  • the client device 140 can automatically send input data to the I/O interface 112. If the client device 140 is required to automatically send input data and needs to obtain the user's authorization, the user can set the corresponding authority in the client device 140.
  • the user can view the results output by the execution device 110 on the client device 140, and the specific presentation form can be a specific method such as display, sound, action, etc.
  • the client device 140 can also be used as a data acquisition terminal to collect the input data of the input I/O interface 112 and the output results of the output I/O interface 112 as shown in the figure as new sample data, and store them in the database 130.
  • the I/O interface 112 directly stores the input data of the input I/O interface 112 and the output results of the output I/O interface 112 as new sample data in the database 130.
  • FIG3 is only a schematic diagram of a system architecture provided in an embodiment of the present application.
  • the positional relationship between the data storage system 150 and the execution device 110 does not constitute any limitation.
  • the data storage system 150 is an external memory relative to the execution device 110. In other cases, the data storage system 150 may also be placed in the execution device 110.
  • a neural network may be obtained by training according to the training device 120.
  • the embodiment of the present application also provides a chip, which includes a neural network processor NPU.
  • the chip can be set in the execution device 110 as shown in Figure 3 to complete the calculation work of the calculation module 111.
  • the chip can also be set in the training device 120 as shown in Figure 3 to complete the training work of the training device 120 and output the target model/rule.
  • Neural network processor NPU is mounted on the main central processing unit (CPU) (host CPU) as a coprocessor, and the main CPU assigns tasks.
  • the core part of NPU is the operation circuit, and the controller controls the operation circuit to extract data from the memory (weight memory or input memory) and perform operations.
  • the arithmetic circuit includes multiple processing units (process engines, PEs) internally.
  • the arithmetic circuit is a two-dimensional systolic array.
  • the arithmetic circuit can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition.
  • the arithmetic circuit is a general-purpose matrix processor.
  • the operation circuit takes the corresponding data of matrix B from the weight memory and caches it on each PE in the operation circuit.
  • the operation circuit takes the matrix A data from the input memory and performs matrix operations with matrix B.
  • the partial results or final results of the matrix are stored in the accumulator.
  • the vector calculation unit can further process the output of the operation circuit, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc.
  • the vector calculation unit can be used for network calculations of non-convolutional/non-FC layers in neural networks, such as pooling, batch normalization, local response normalization, etc.
  • the vector computation unit can store the processed output vector to a unified buffer.
  • the vector computation unit can apply a nonlinear function to the output of the computation circuit, such as a vector of accumulated values, to generate an activation value.
  • the vector computation unit generates a normalized value, a merged value, or both.
  • the processed output vector can be used as an activation input to the computation circuit, such as for use in a subsequent layer in a neural network.
  • the unified memory is used to store input data and output data.
  • the weight data is directly transferred from the external memory to the input memory and/or the unified memory through the direct memory access controller (DMAC), the weight data in the external memory is stored in the weight memory, and the data in the unified memory is stored in the external memory.
  • DMAC direct memory access controller
  • the bus interface unit (BIU) is used to enable interaction between the main CPU, DMAC and instruction fetch memory through the bus.
  • An instruction fetch buffer connected to the controller, used to store instructions used by the controller
  • the controller is used to call the instructions cached in the memory to control the working process of the computing accelerator.
  • the unified memory, input memory, weight memory and instruction fetch memory are all on-chip memories
  • the external memory is a memory outside the NPU, which can be a double data rate synchronous dynamic random access memory (DDR SDRAM), a high bandwidth memory (HBM) or other readable and writable memories.
  • DDR SDRAM double data rate synchronous dynamic random access memory
  • HBM high bandwidth memory
  • space is used here because the classified object is not a single thing, but a class of things, and space refers to the collection of all individuals of this class of things.
  • W is a weight vector, and each value in the vector represents the weight value of a neuron in the neural network of this layer.
  • the vector W determines the spatial transformation from the input space to the output space described above, that is, the weight W of each layer controls how to transform the space.
  • the purpose of training a neural network is to finally obtain the weight matrix of all layers of the trained neural network (the weight matrix formed by many layers of vectors W). Therefore, the training process of a neural network is essentially about learning how to control spatial transformations, or more specifically, learning the weight matrix.
  • Neural networks can use the error back propagation (BP) algorithm to correct the size of the parameters in the initial neural network model during the training process, so that the reconstruction error loss of the neural network model becomes smaller and smaller. Specifically, the forward transmission of the input signal to the output will generate error loss, and the parameters in the initial neural network model are updated by back propagating the error loss information, so that the error loss converges.
  • the back propagation algorithm is a back propagation movement dominated by error loss, which aims to obtain the optimal parameters of the neural network model, such as the weight matrix.
  • the method provided in the present application is described below from the training side of the neural network and the application side of the neural network.
  • the model training method provided in the embodiment of the present application involves the processing of data sequences, and can be specifically applied to methods such as data training, machine learning, and deep learning, and symbolizes and formalizes intelligent information modeling, extraction, preprocessing, and training of training data (for example, the first LDR image of the target object and the second LDR image of the target object in the model training method provided in the embodiment of the present application), and finally obtains a trained neural network (for example, the target model in the model training method provided in the embodiment of the present application); and the image processing method provided in the embodiment of the present application can use the above-mentioned trained neural network to input input data (for example, the first LDR image of the target object and the second LDR image of the target object in the image processing method provided in the embodiment of the present application) into the trained neural network to obtain output data (for example, the HDR image of the target object in the image processing method provided in the embodiment of the present application).
  • model training method and image processing method provided in the embodiment of the present application are inventions based on the same concept, and can also be understood as two parts in a system, or two stages of an overall process: such as the model training stage and the model application stage.
  • FIG4 is a schematic diagram of the structure of the target model provided in the embodiment of the present application.
  • the target model includes a block search network based on speech similarity, a fusion transformer network based on self-attention and interactive attention mechanisms, and a local Reconstruct the transformer network, wherein the input end of the block search network and the first input end of the fused transformer network serve as the input end of the entire target model, the output end of the block search network is connected to the second input end of the fused transformer network, the output end of the fused transformer network is connected to the input end of the local reconstruction transformer network, and the output end of the local reconstruction transformer network serves as the output end of the entire target model.
  • FIG5 is a flowchart of an image processing method provided in an embodiment of the present application. As shown in FIG5
  • the target object when it is necessary to obtain an HDR image of the target object, the target object can be photographed using different exposure rates, thereby acquiring a first LDR image of the target object (also referred to as a reference image of the target object) and a second LDR image of the target object (also referred to as a support image of the target object).
  • the target object may refer to an object in a scene, or an area in a scene containing an object, etc.
  • the target object may refer to a boy in the park, or may refer to the grass where the boy is in the park, etc.
  • the number of the second LDR images can be one or more. If multiple second LDR images are collected, then the multiple second LDR images are captured using multiple exposure rates (the multiple exposure rates are different from each other). For any second LDR image among the multiple second LDR images, the exposure rate used to capture the second LDR image can be greater than the exposure rate used to capture the first LDR image, or can be less than the exposure rate used to capture the first LDR image.
  • the first LDR image of the target object and the second LDR image of the target object can be input into the target model, so that the target model performs a series of processing on the first LDR image of the target object and the second LDR image of the target object, thereby obtaining an HDR image of the target object.
  • the target model After receiving a first LDR image of the target object and a second LDR image of the target object, the target model can perform block matching on multiple first image blocks of the first LDR image of the target object and multiple second image blocks of the second LDR image of the target object, thereby establishing a one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image.
  • the target model can obtain a one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks in the following manner:
  • a block search network of a target model may perform a series of processing on the first LDR image and the second LDR image, thereby obtaining similarities between a plurality of first image blocks of the first LDR image and a plurality of second image blocks of the second LDR image.
  • the block search network can use the similarities between the multiple first image blocks and the multiple second image blocks to construct a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks.
  • the block search network includes a feature extraction module and a block search module. Then, the block search network can obtain the similarity between the plurality of first image blocks and the plurality of second image blocks in the following manner:
  • the feature extraction module of the block search network may cooperate with the block search module to perform feature extraction on the first LDR image and the second LDR image respectively, thereby obtaining first features of multiple first image blocks of the first LDR image (also referred to as semantic features of multiple first image blocks, etc.) and second features of multiple second image blocks of the second LDR image (also referred to as semantic features of multiple second image blocks, etc.).
  • Figure 7 is another structural schematic diagram of the block search network provided in an embodiment of the present application
  • three LDR images are collected, namely support image 1, reference image and support image 2, and these three LDR images are shot using exposure rate 1, exposure rate 2 and exposure rate 3, respectively, wherein exposure rate 1>exposure rate 2>exposure rate 3.
  • the feature extraction module can first extract the overall semantic features of support image 1, the size of the overall semantic features of support image 1 is c (channel) ⁇ H (height) ⁇ W (width), and send the overall semantic features of support image 1 to the block search module.
  • the feature extraction module can also first extract the overall semantic features of the reference image, the size of the overall semantic features of the reference image is c ⁇ H ⁇ W, and send the overall semantic features of the reference image to the block search module.
  • the feature extraction module can also first extract the overall semantic features of support image 2, the size of the overall semantic features of support image 2 is c ⁇ H ⁇ W, and send the overall semantic features of support image 2 to the block search module. The features are sent to the block search module.
  • the supporting image 1 is composed of N2 supporting image blocks 1 (N can be 8 or 16, etc.), the reference image is composed of N2 reference image blocks, and the supporting image block 2 is composed of N2 supporting image blocks 2. Accordingly, the overall semantic features of the supporting image 1 are composed of the semantic features of the N2 supporting image blocks 1, the overall semantic features of the reference image are composed of the semantic features of the N2 reference image blocks, and the overall semantic features of the supporting image 2 are composed of the semantic features of the N2 supporting image blocks 2.
  • the block search module can divide the overall semantic features of the support image 1 into N2 semantic features of the support image blocks 1, and the size of the semantic features of each support image block 1 is c ⁇ (H/N) ⁇ (W/N).
  • the block search module can also divide the overall semantic features of the reference image into N2 semantic features of the reference image blocks, and the size of the semantic features of each reference image block is c ⁇ (H/N) ⁇ (W/N).
  • the block search module can also divide the overall semantic features of the support image 2 into N2 semantic features of the support image blocks 2, and the size of the semantic features of each support image block 2 is c ⁇ (H/N) ⁇ (W/N).
  • the block search module can also calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks, so as to obtain the similarity between the multiple first image blocks and the multiple second image blocks (which can also be called the cosine similarity between the multiple first image blocks and the multiple second image blocks, etc.).
  • the block search module can also calculate the cosine similarity of the semantic features of the N2 supporting image blocks 1 and the semantic features of the N2 reference image blocks, thereby obtaining a similarity matrix 1 with N2 rows and N2 columns, and the similarity matrix 1 includes the similarities between the N2 supporting image blocks 1 and the N2 reference image blocks.
  • the block search module can also calculate the cosine similarity of the semantic features of the N2 supporting image blocks 2 and the semantic features of the N2 reference image blocks, thereby obtaining a similarity matrix 2 with N2 rows and N2 columns, and the similarity matrix 2 includes the similarities between the N2 supporting image blocks 2 and the N2 reference image blocks.
  • the block search network can obtain the one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks in the following manner:
  • the block search module may select, based on the similarity between the first image block and the plurality of second image blocks, a second image block with the greatest similarity among the plurality of second image blocks as the second image block most similar to the first image block (i.e., the aforementioned fourth image block).
  • the block search module can construct a correspondence between the first image block and the second image block that is most similar to the first image block.
  • the block search module can also perform the same operation as that performed on the first image block on the remaining first image blocks except the first image block among the plurality of first image blocks, so that a one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks can be finally obtained, and the correspondence is sent to the fusion transformer network.
  • the first row represents the similarity between the first reference image block and the N2 supporting images 1
  • the block search module can extract the maximum similarity in the first row as the correspondence between the first reference image block and the supporting image 1 that is most similar to the first reference image block.
  • the block search module can also extract the correspondence between the second reference image block and the supporting image 1 that is most similar to the second reference image block in the second row, ..., until the correspondence between the N2th reference image block and the supporting image 1 that is most similar to the N2th reference image block is extracted in the N2th row.
  • the first row represents the similarity between the first reference image block and the N2 supporting images 2
  • the block search module can extract the maximum similarity in the first row as the correspondence between the first reference image block and the supporting image 2 most similar to the first reference image block.
  • the block search module can also extract the correspondence between the second reference image block and the supporting image 2 most similar to the second reference image block in the second row, ..., until the correspondence between the N2th reference image block and the supporting image 2 most similar to the N2th reference image block is extracted in the N2th row.
  • the block search module can obtain the one-to-one correspondence between the N2 reference image blocks and the N2 support images 1, and the one-to-one correspondence between the N2 reference image blocks and the N2 support images 2.
  • the block search module can also perform a reshape operation to present the one-to-one correspondence between the N2 reference image blocks and the N2 support images 1 as a similarity matrix 3 with N rows and N columns, and present the one-to-one correspondence between the N2 reference image blocks and the N2 support images 2 as a similarity matrix 4 with N rows and N columns, and send them to the fusion transformer network.
  • the first LDR image and the second LDR image are fused to obtain an HDR image of the target object.
  • the target model can use the one-to-one correspondence between the multiple first image blocks and the multiple second image blocks to fuse the first LDR image and the second LDR image, thereby obtaining and outputting an HDR image of the target object.
  • FIG. 8 is a schematic diagram of the structure of a fusion transformer network provided in an embodiment of the present application
  • the fusion transformer network of the target model includes feature extraction module, block alignment module, self-attention mechanism module, interactive attention mechanism module and splicing module. Then, the target model can obtain the HDR image of the target object in the following ways:
  • a feature extraction module e.g., a convolutional network, etc. fused with a transformer network may perform feature extraction on the first LDR image and the second LDR image, thereby obtaining third features of the plurality of first image blocks of the first LDR image (also referred to as depth features of the plurality of first image blocks, etc.), and fourth features of the plurality of second image blocks of the second LDR image (also referred to as depth features of the plurality of second image blocks, etc.).
  • a feature extraction module e.g., a convolutional network, etc. fused with a transformer network may perform feature extraction on the first LDR image and the second LDR image, thereby obtaining third features of the plurality of first image blocks of the first LDR image (also referred to as depth features of the plurality of first image blocks, etc.), and fourth features of the plurality of second image blocks of the second LDR image (also referred to as depth features of the plurality of second image blocks, etc.).
  • Figure 9 is another structural diagram of the fused transformer network provided in an embodiment of the present application
  • the feature extraction module can first extract the overall depth feature of the support image 1, the size of the overall depth feature of the support image 1 is c ⁇ H ⁇ W, and send the overall depth feature of the support image 1 to the block alignment module.
  • the feature extraction module can also first extract the overall depth feature of the reference image, the size of the overall depth feature of the reference image is c ⁇ H ⁇ W, and send the overall depth feature of the reference image to the block alignment module.
  • the feature extraction module can also first extract the overall depth feature of the support image 2, the size of the overall depth feature of the support image 2 is c ⁇ H ⁇ W, and send the overall depth feature of the support image 2 to the block alignment module.
  • the overall depth feature of the support image 1 is composed of the depth features of N2 support image blocks 1
  • the overall depth feature of the reference image is composed of the depth features of N2 reference image blocks
  • the overall depth feature of the support image 2 is composed of the depth features of N2 support image blocks 2.
  • the block alignment module After obtaining the fourth features of the plurality of second image blocks and the one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks, since the correspondence indicates a new ordering of the fourth features of the plurality of second image blocks, the block alignment module can adjust the ordering of the fourth features of the plurality of second image blocks based on the correspondence, thereby obtaining the fourth features of the plurality of second image blocks after the ordering is adjusted. Then, the block alignment module can send the third features of the plurality of first image blocks, the fourth features of the plurality of second image blocks, and the fourth features of the plurality of second image blocks after the ordering is adjusted to the self-attention mechanism module and the interactive attention mechanism module.
  • the block alignment module obtains the overall depth features of the support image 1, the overall depth features of the support image 2, the similarity matrix 3 and the similarity matrix 4, since in the overall depth features of the support image 1, the depth features of the N2 support image blocks 1 are set according to the original order (that is, in the support image 1, the original order of the N2 support image blocks 1), and the similarity matrix 3 indicates the new order of the depth features of the N2 support image blocks 1, the block alignment module can adjust the order of the depth features of the N2 support image blocks 1 according to the indication of the similarity matrix 3, thereby obtaining the overall depth features of the support image 1 after the adjusted order.
  • the block alignment module can divide the overall depth features of the support image 1 after the adjusted order into the semantic features of the N2 support image blocks 1 after the adjusted order, and the size of each semantic feature of the support image block 1 after the adjusted order is c ⁇ (H/N) ⁇ (W/N).
  • the block alignment module can adjust the order of the depth features of the N2 supporting image blocks 2 according to the indication of the similarity matrix 4, thereby obtaining the overall depth features of the supporting image 2 after the adjusted order. Then, the block alignment module can divide the overall depth features of the supporting image 2 after the adjusted order into the semantic features of the N2 supporting image blocks 2 after the adjusted order, and the size of each semantic feature of the supporting image block 2 after the adjusted order is c ⁇ (H/N) ⁇ (W/N).
  • the block alignment module and the block search module may also divide the overall depth features of the support image 1 into N2 depth features of the support image blocks 1, and the size of the depth features of each support image block 1 is c ⁇ (H/N) ⁇ (W/N).
  • the block search module may also divide the overall depth features of the reference image into N2 depth features of the reference image blocks, and the size of the depth features of each reference image block is c ⁇ (H/N) ⁇ (W/N).
  • the block search module may also divide the overall depth features of the support image 2 into N2 depth features of the support image blocks 2, and the size of the depth features of each support image block 2 is c ⁇ (H/N) ⁇ (W/N).
  • the block alignment module can send the depth features of the N2 supporting image blocks 1, the depth features of the N2 reference image blocks, the depth features of the N2 supporting image blocks 2, the adjusted and sorted semantic features of the N2 supporting image blocks 1, and the adjusted and sorted semantic features of the N2 supporting image blocks 2 to the self-attention mechanism module and the interactive attention mechanism module.
  • the self-attention mechanism module and the interactive attention mechanism module can cooperate with the local reconstruction transformer network to perform a series of processing on the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting, so as to obtain and output the HDR image of the target object.
  • the local reconstruction transformer network of the target model includes a convolution module, a transformer module, an addition module and an activation module. Then, the target model can obtain an HDR image of the target object in the following manner:
  • the self-attention mechanism module can perform a series of processing on the third features of multiple first image blocks and send the obtained processing results to the splicing module.
  • the interactive attention mechanism module can perform a series of processing on the third features of multiple first image blocks and the fourth features of multiple second image blocks, and send the obtained processing results to the splicing module.
  • the interactive attention mechanism module can also perform a series of processing on the third features of multiple first image blocks and the fourth features of multiple second image blocks after adjustment and sorting, and send the obtained processing results to the splicing module.
  • the splicing module can splice all the received processing results and send the obtained splicing results to the local reconstruction transformer network.
  • the self-attention module can process the depth features of the N2 reference image blocks to obtain processing result 1.
  • the interactive attention module 1 can process the depth features of the N2 reference image blocks and the depth features of the N2 supporting image blocks 1 to obtain processing result 2.
  • the interactive attention module 1 can process the depth features of the N2 reference image blocks and the depth features of the N2 supporting image blocks 2 to obtain processing result 2.
  • the interactive attention module 3 can process the depth features of the N2 reference image blocks and the depth features of the adjusted and sorted N2 supporting image blocks 1 to obtain processing result 4.
  • the interactive attention module 4 can process the depth features of the N2 reference image blocks and the depth features of the adjusted and sorted N2 supporting image blocks 2 to obtain processing result 5.
  • the stitching module can stitch processing results 1 to 5 to obtain the corresponding stitching result.
  • the splicing result is processed by the convolution module, transformer module, addition module and activation module in the local reconstruction transformer network respectively, and finally the HDR image of the target object can be obtained and output externally.
  • FIG11 is a structural diagram of a self-attention mechanism module or an interactive attention mechanism module provided in an embodiment of the present application
  • the structure of the self-attention mechanism module and the structure of the interactive attention module can be the same, and any one of these two modules may include: a normalization unit, a multi-head attention mechanism unit, an addition unit, and a multi-layer perceptron unit, etc. It can be seen that both modules can implement normalization processing, processing based on a multi-head attention mechanism, addition processing, processing based on a multi-layer perceptron, etc.
  • LDR image it is only necessary to replace the LDR image with a low-resolution image (for example, the first low-resolution image and the second low-resolution image) or a noisy image (for example, the first noisy image and the second noisy image), and replace the HDR image with a high-resolution image or a denoised image. No further details will be given here.
  • the target model provided in the embodiment of the present application i.e., IFT in Table 1
  • the model provided by the related art i.e., the remaining models except IFT in Table 1, for example, Sen, Hu, etc.
  • Table 1 the comparison results are shown in Table 1:
  • the target model provided in the embodiment of the present application i.e., IFT in Table 1
  • the model provided by the related art i.e., the remaining models except IFT in Table 1, for example, Sen, Hu, etc.
  • the comparison results are shown in Table 2:
  • a first LDR image of the target object and a second LDR image of the target object may be first collected, and the first LDR image and the second LDR image may be input into a target model.
  • the target model may perform image block matching on the first LDR image and the second LDR image, thereby obtaining a one-to-one correspondence between a plurality of first image blocks of the first LDR image and a plurality of second image blocks of the second LDR image.
  • the target model may use the correspondence to fuse the first LDR image and the second LDR image, thereby obtaining and outputting an HDR image of the target object.
  • the target model includes a fusion transformer network based on a self-attention mechanism and an interactive attention mechanism.
  • the network can effectively take into account the detail information of the first LDR image and the second LDR image themselves, so that the final HDR image maintains good details and has no artifacts.
  • FIG16 is a flow chart of the model training method provided by the embodiment of the present application. As shown in FIG16 , the method includes:
  • a batch of training data may be first obtained, the batch of training data including a first LDR image of the target object and a second LDR image of the target object, the first LDR image and the second LDR image being images obtained by photographing the target object at different exposures. It should be noted that, for the first LDR image and the second LDR image, the real HDR image of the target object is known.
  • the first LDR image and the second LDR image can be input into the model to be trained.
  • the model to be trained can perform a series of processing on the first LDR image and the second LDR image, thereby obtaining a one-to-one correspondence between a plurality of first image blocks of the first LDR image and a plurality of second image blocks of the second LDR image, and using the one-to-one correspondence between the plurality of first image blocks and the plurality of second image blocks, fuse the first LDR image and the second LDR image, thereby obtaining a (predicted) HDR image of the target object.
  • the model to be trained is used to: obtain the similarity between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image based on the first LDR image and the second LDR image; and obtain a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the model to be trained is used to: extract features from a first LDR image and a second LDR image to obtain first features of multiple first image blocks of the first LDR image and second features of multiple second image blocks of the second LDR image; and calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain similarities between the multiple first image blocks and the multiple second image blocks.
  • multiple first image blocks include a third image block
  • the model to be trained is used to: based on the similarity between the third image block and multiple second image blocks, determine the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establish a corresponding relationship between the third image block and the fourth image block.
  • the model to be trained is used to: perform feature extraction on the first LDR image and the second LDR image to obtain a third feature of multiple first image blocks of the first LDR image and a fourth feature of multiple second image blocks of the second LDR image; adjust the order of the fourth features of the multiple second image blocks based on the corresponding relationship to obtain the fourth features of the multiple second image blocks after the order is adjusted; and process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after the order is adjusted to obtain an HDR image of the target object.
  • the aforementioned processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, Intention mechanism processing, splicing processing, convolution processing, transformer network-based processing, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • step 1602 reference may be made to the relevant description of steps 502 to 503 in the embodiment shown in FIG. 5, which will not be repeated here.
  • the model to be trained is trained to obtain a target model.
  • the HDR image and the real HDR image can be calculated by a preset loss function to obtain a target loss, which is used to indicate the difference between the HDR image and the real HDR image.
  • the target loss can be used to update the parameters of the model to be trained, thereby obtaining the model to be trained after the updated parameters.
  • the next batch of training data can be used to continue training the model to be trained after the updated parameters until the model training conditions are met (for example, the target loss converges, etc.), thereby obtaining the target model in the embodiment shown in Figure 5.
  • the model to be trained can also fuse multiple low-resolution images into a high-resolution image (for example, fuse the first low-resolution image and the second low-resolution image into a high-resolution image), or fuse multiple noisy images into a denoised image, etc. (for example, fuse the first noisy image and the second noisy image into a denoised image).
  • These fusion processes and corresponding model training processes can refer to steps 1601 to 1603.
  • LDR image it is only necessary to replace the LDR image with a low-resolution image (for example, the first low-resolution image and the second low-resolution image) or a noisy image (for example, the first noisy image and the second noisy image), and replace the HDR image with a high-resolution image or a denoised image. No further details will be given here.
  • the target model obtained by training in the embodiment of the present application has an image processing function (for example, a function of fusing multiple LDR images into an HDR image, etc.). Specifically, when it is necessary to obtain an HDR image of a target object, a first LDR image of the target object and a second LDR image of the target object can be first collected, and the first LDR image and the second LDR image can be input into the target model. Then, the target model can match the image blocks of the first LDR image and the second LDR image, thereby obtaining a one-to-one correspondence between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image.
  • an image processing function for example, a function of fusing multiple LDR images into an HDR image, etc.
  • the target model can use the corresponding relationship to fuse the first LDR image and the second LDR image, thereby obtaining and outputting an HDR image of the target object.
  • the process of obtaining a one-to-one correspondence between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image it is equivalent to aligning multiple first image blocks of the first LDR image with multiple second image blocks of the second LDR image in terms of content.
  • this correspondence as a guide to achieve the fusion between the first LDR image and the second LDR image, a better fusion can be achieved, so that the HDR image of the target object finally obtained does not have artifacts.
  • FIG. 17 is a structural schematic diagram of the image processing device provided in the embodiment of the present application. As shown in FIG. 17 , the device includes:
  • a first acquisition module 1701 is used to acquire a first LDR image of a target object and a second LDR image of the target object, where the first LDR image and the second LDR image are images obtained by photographing the target object at different exposure levels;
  • a second acquisition module 1702 is used to acquire a one-to-one correspondence between a plurality of first image blocks of the first LDR image and a plurality of second image blocks of the second LDR image based on the first LDR image and the second LDR image;
  • the fusion module 1703 is used to fuse the first LDR image and the second LDR image based on the corresponding relationship to obtain an HDR image of the target object.
  • a first LDR image of the target object and a second LDR image of the target object may be first collected, and the first LDR image and the second LDR image may be input into the target model.
  • the target model may perform image block matching on the first LDR image and the second LDR image, thereby obtaining a one-to-one correspondence between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image.
  • the target model may use the correspondence to fuse the first LDR image and the second LDR image, thereby obtaining and outputting the HDR image of the target object.
  • the second acquisition module 1702 is used to: acquire the similarity between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image based on the first LDR image and the second LDR image; and acquire a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the second acquisition module 1702 is used to: perform feature extraction on the first LDR image and the second LDR image to obtain first features of multiple first image blocks of the first LDR image and second features of multiple second image blocks of the second LDR image; calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain similarities between the multiple first image blocks and the multiple second image blocks.
  • the plurality of first image blocks include a third image block
  • the second acquisition module 1702 is configured to: based on the similarity between the third image block and the plurality of second image blocks, determine the second image block with the greatest similarity among the plurality of second image blocks as the fourth image block; and establish a correspondence between the third image block and the fourth image block.
  • the fusion module 1703 is used to extract features from the first LDR image and the second LDR image to obtain third features of multiple first image blocks of the first LDR image and fourth features of multiple second image blocks of the second LDR image; based on the corresponding relationship, adjust the order of the fourth features of the multiple second image blocks to obtain the fourth features of the multiple second image blocks after the adjustment; and process the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the adjustment to obtain an HDR image of the target object.
  • the fusion module 1703 is used to process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting to obtain an HDR image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • the target model can also fuse multiple low-resolution images into a high-resolution image (for example, fusing the first low-resolution image and the second low-resolution image into a high-resolution image), or fuse multiple noisy images into a denoised image, etc. (for example, fusing the first noisy image and the second noisy image into a denoised image).
  • FIG18 is a schematic diagram of a structure of a model training device provided in an embodiment of the present application. As shown in FIG18 , the device includes:
  • An acquisition module 1801 is used to acquire a first LDR image of a target object and a second LDR image of the target object, where the first LDR image and the second LDR image are images obtained by photographing the target object at different exposure levels;
  • the processing module 1802 is used to process the first LDR image and the second LDR image through the model to be trained to obtain a high dynamic range HDR image of the target object, wherein the model to be trained is used to: obtain a one-to-one correspondence between a plurality of first image blocks of the first LDR image and a plurality of second image blocks of the second LDR image based on the first LDR image and the second LDR image; and fuse the first LDR image and the second LDR image based on the correspondence to obtain the HDR image of the target object;
  • the training module 1803 is used to train the model to be trained based on the HDR image to obtain a target model.
  • the target model obtained by training in the embodiment of the present application has an image processing function (for example, a function of fusing multiple LDR images into an HDR image, etc.). Specifically, when it is necessary to obtain an HDR image of the target object, the first LDR image of the target object and the second LDR image of the target object can be first collected, and the first LDR image and the second LDR image can be input into the target model. Then, the target model can perform image block matching on the first LDR image and the second LDR image to obtain a one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image.
  • an image processing function for example, a function of fusing multiple LDR images into an HDR image, etc.
  • the target model can use the corresponding relationship to fuse the first LDR image and the second LDR image to obtain and output the HDR image of the target object.
  • the process of obtaining the one-to-one correspondence between the multiple first image blocks of the first LDR image and the multiple second image blocks of the second LDR image it is equivalent to matching the multiple first image blocks of the first LDR image with the multiple second image blocks of the second LDR image.
  • the image block is aligned one by one with the second image blocks of the second LDR image in terms of content. Then, using this correspondence as a guide to achieve fusion between the first LDR image and the second LDR image, a better fusion can be achieved, so that the HDR image of the target object finally obtained does not have artifacts.
  • the model to be trained is used to: obtain the similarity between multiple first image blocks of the first LDR image and multiple second image blocks of the second LDR image based on the first LDR image and the second LDR image; and obtain a one-to-one correspondence between the multiple first image blocks and the multiple second image blocks based on the similarity.
  • the model to be trained is used to: extract features from a first LDR image and a second LDR image to obtain first features of multiple first image blocks of the first LDR image and second features of multiple second image blocks of the second LDR image; and calculate the first features of the multiple first image blocks and the second features of the multiple second image blocks to obtain similarities between the multiple first image blocks and the multiple second image blocks.
  • multiple first image blocks include a third image block
  • the model to be trained is used to: based on the similarity between the third image block and multiple second image blocks, determine the second image block with the greatest similarity among the multiple second image blocks as the fourth image block; and establish a corresponding relationship between the third image block and the fourth image block.
  • the model to be trained is used to: extract features from the first LDR image and the second LDR image to obtain third features of multiple first image blocks of the first LDR image and fourth features of multiple second image blocks of the second LDR image; adjust the order of the fourth features of the multiple second image blocks based on the corresponding relationship to obtain fourth features of the multiple second image blocks after the order is adjusted; and process the third features of the multiple first image blocks and the fourth features of the multiple second image blocks after the order is adjusted to obtain an HDR image of the target object.
  • the model to be trained is used to process the third features of the multiple first image blocks, the fourth features of the multiple second image blocks, and the fourth features of the multiple second image blocks after adjustment and sorting to obtain an HDR image of the target object.
  • the processing includes at least one of the following: processing based on a self-attention mechanism, processing based on an interactive attention mechanism, splicing processing, convolution processing, processing based on a transformer network, addition processing, and activation processing.
  • the processing based on the self-attention mechanism or the processing based on the interactive attention mechanism includes at least one of the following: normalization processing, processing based on a multi-head attention mechanism, addition processing, and processing based on a multi-layer perceptron.
  • the transformer network-based processing includes at least one of the following: processing based on a multi-head self-attention mechanism and processing based on a multi-layer perceptron.
  • the model to be trained can also fuse multiple low-resolution images into a high-resolution image (for example, fuse the first low-resolution image and the second low-resolution image into a high-resolution image), or fuse multiple noisy images into a denoised image, etc. (for example, fuse the first noisy image and the second noisy image into a denoised image).
  • LDR image it only needs to replace the LDR image with a low-resolution image (for example, the first low-resolution image and the second low-resolution image) or a noisy image (for example, the first noisy image and the second noisy image), and replace the HDR image with a high-resolution image or a denoised image. No further details will be given here.
  • FIG. 19 is a structural schematic diagram of the execution device provided by the embodiment of the present application.
  • the execution device 1900 can be specifically manifested as a mobile phone, a tablet, a laptop computer, an intelligent wearable device, a server, etc., which is not limited here.
  • the image processing device described in the corresponding embodiment of FIG. 17 can be deployed on the execution device 1900 to implement the image processing function in the corresponding embodiment of FIG. 5.
  • the execution device 1900 includes: a receiver 1901, a transmitter 1902, a processor 1903 and a memory 1904 (wherein the number of processors 1903 in the execution device 1900 can be one or more, and FIG.
  • the processor 19 takes one processor as an example), wherein the processor 1903 may include an application processor 19031 and a communication processor 19032.
  • the receiver 1901, the transmitter 1902, the processor 1903 and the memory 1904 may be connected via a bus or other means.
  • the memory 1904 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1903. A portion of the memory 1904 may also include a non-volatile random access memory (NVRAM).
  • NVRAM non-volatile random access memory
  • the memory 1904 stores processor and operation instructions, executable modules or data structures, or subsets thereof, or extended sets thereof, wherein the operation instructions may include various operation instructions for implementing various operations.
  • the processor 1903 controls the operation of the execution device.
  • the various components of the execution device are coupled together through a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc.
  • the bus system includes not only a data bus but also a power bus, a control bus, and a status signal bus, etc.
  • various buses are referred to as bus systems in the figure.
  • the method disclosed in the above embodiment of the present application can be applied to the processor 1903, or implemented by the processor 1903.
  • the processor 1903 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1903 or the instruction in the form of software.
  • the above processor 1903 can be a general processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and can further include an application specific integrated circuit (application specific integrated circuit, ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.
  • the processor 1903 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present application.
  • the general processor can be a microprocessor or the processor can also be any conventional processor, etc.
  • the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to be executed, or a combination of hardware and software modules in the decoding processor can be executed.
  • the software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc.
  • the storage medium is located in the memory 1904, and the processor 1903 reads the information in the memory 1904 and completes the steps of the above method in combination with its hardware.
  • the receiver 1901 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the execution device.
  • the transmitter 1902 can be used to output digital or character information through the first interface; the transmitter 1902 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group; the transmitter 1902 can also include a display device such as a display screen.
  • the processor 1903 is used to obtain an HDR image of the target object through the target model in the embodiment corresponding to FIG. 5 .
  • FIG. 20 is a schematic diagram of the structure of the training device provided by the embodiment of the present application.
  • the training device 2000 is implemented by one or more servers.
  • the training device 2000 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 2020 (for example, one or more processors) and a memory 2032, and one or more storage media 2030 (for example, one or more mass storage devices) storing application programs 2042 or data 2044.
  • the memory 2032 and the storage medium 2030 can be short-term storage or permanent storage.
  • the program stored in the storage medium 2030 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the training device. Furthermore, the central processing unit 2020 can be configured to communicate with the storage medium 2030 and execute a series of instruction operations in the storage medium 2030 on the training device 2000.
  • the training device 2000 may also include one or more power supplies 2026, one or more wired or wireless network interfaces 2050, one or more input and output interfaces 2058; or, one or more operating systems 2041, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.
  • the training device can execute the model training method in the embodiment corresponding to Figure 16.
  • An embodiment of the present application also relates to a computer storage medium, in which a program for signal processing is stored.
  • the program When the program is run on a computer, the computer executes the steps executed by the aforementioned execution device, or the computer executes the steps executed by the aforementioned training device.
  • An embodiment of the present application also relates to a computer program product, which stores instructions, which, when executed by a computer, enable the computer to execute the steps executed by the aforementioned execution device, or enable the computer to execute the steps executed by the aforementioned training device.
  • the execution device, training device or terminal device provided in the embodiments of the present application may specifically be a chip, and the chip includes: a processing unit and a communication unit, wherein the processing unit may be, for example, a processor, and the communication unit may be, for example, an input/output interface, a pin or a circuit, etc.
  • the processing unit may execute the computer execution instructions stored in the storage unit so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment.
  • the storage unit is a storage unit in the chip, such as a register, a cache, etc.
  • the storage unit may also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.
  • ROM read-only memory
  • RAM random access memory
  • the operation circuit 2103 includes multiple processing units (Process Engine, PE) inside.
  • the operation circuit 2103 is a two-dimensional systolic array.
  • the operation circuit 2103 can also be a one-dimensional systolic array or other electronic circuits capable of performing mathematical operations such as multiplication and addition.
  • the operation circuit 2103 is a general-purpose matrix processor.
  • the operation circuit takes the corresponding data of matrix B from the weight memory 2102 and caches it on each PE in the operation circuit.
  • the operation circuit takes the matrix A data from the input memory 2101 and performs matrix operations with matrix B, and the partial results or final results of the matrix are stored in the accumulator 2108.
  • the unified memory 2106 is used to store input data and output data.
  • the weight data is directly transferred to the weight memory 2102 through the direct memory access controller (DMAC) 2105.
  • the input data is also transferred to the unified memory 2106 through the DMAC.
  • DMAC direct memory access controller
  • BIU stands for Bus Interface Unit, which is used for the interaction between AXI bus, DMAC and instruction fetch buffer (IFB) 2109.
  • the bus interface unit 2113 (Bus Interface Unit, BIU for short) is used for the instruction fetch memory 2109 to obtain instructions from the external memory, and is also used for the storage unit access controller 2105 to obtain the original data of the input matrix A or the weight matrix B from the external memory.
  • DMAC is mainly used to transfer input data in the external memory DDR to the unified memory 2106 or to transfer weight data to the weight memory 2102 or to transfer input data to the input memory 2101.
  • the vector calculation unit 2107 includes multiple operation processing units, and when necessary, further processes the output of the operation circuit 2103, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional/fully connected layer network calculations in neural networks, such as Batch Normalization, pixel-level summation, upsampling of the predicted label plane, etc.
  • the vector calculation unit 2107 can store the processed output vector to the unified memory 2106.
  • the vector calculation unit 2107 can apply a linear function; or a nonlinear function to the output of the operation circuit 2103, such as linear interpolation of the predicted label plane extracted by the convolution layer, and then, for example, a vector of accumulated values to generate an activation value.
  • the vector calculation unit 2107 generates a normalized value, a pixel-level summed value, or both.
  • the processed output vector can be used as an activation input to the operation circuit 2103, for example, for use in a subsequent layer in a neural network.
  • An instruction fetch buffer 2109 connected to the controller 2104 is used to store instructions used by the controller 2104;
  • Unified memory 2106, input memory 2101, weight memory 2102 and instruction fetch memory 2109 are all on-chip memories. External memories are private to the NPU hardware architecture.
  • the processor mentioned in any of the above places may be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above programs.
  • the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.
  • the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.
  • the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a magnetic disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.
  • a computer device which can be a personal computer, a training device, or a network device, etc.
  • all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof.
  • all or part of the embodiments may be implemented in the form of a computer program product.
  • the computer program product includes one or more computer instructions.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
  • the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center.
  • the computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations.
  • the available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.
  • a magnetic medium e.g., a floppy disk, a hard disk, a tape
  • an optical medium e.g., a DVD
  • a semiconductor medium e.g., a solid-state drive (SSD)

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

La présente demande divulgue un procédé de traitement des images et son dispositif associé, pouvant mettre en œuvre une meilleure fusion d'une pluralité d'images à faible plage dynamique (LDR) de telle sorte qu'une image à plage dynamique élevée (HDR) finalement obtenue ne présente pas d'artéfacts. Le procédé de la présente demande comprend les étapes suivantes : lorsqu'une image HDR d'un objet cible doit être acquise, une première image LDR de l'objet cible et une seconde image LDR de l'objet cible peuvent être collectées en premier, et la première image LDR et la seconde image LDR sont entrées dans un modèle cible ; ainsi, le modèle cible peut effectuer une mise en correspondance de blocs d'image sur la première image LDR et la seconde image LDR, de façon à obtenir une correspondance biunivoque entre une pluralité de premiers blocs d'image de la première image LDR et une pluralité de seconds blocs d'image de la seconde image LDR ; puis, le modèle cible peut fusionner la première image LDR et la seconde image LDR à l'aide de la correspondance, de façon à obtenir et livrer en sortie l'image HDR de l'objet cible.
PCT/CN2024/080717 2023-03-15 2024-03-08 Procédé de traitement des images et son dispositif associé Pending WO2024188171A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310277464.6 2023-03-15
CN202310277464.6A CN116309226A (zh) 2023-03-15 2023-03-15 一种图像处理方法及其相关设备

Publications (1)

Publication Number Publication Date
WO2024188171A1 true WO2024188171A1 (fr) 2024-09-19

Family

ID=86786719

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/080717 Pending WO2024188171A1 (fr) 2023-03-15 2024-03-08 Procédé de traitement des images et son dispositif associé

Country Status (2)

Country Link
CN (1) CN116309226A (fr)
WO (1) WO2024188171A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116309226A (zh) * 2023-03-15 2023-06-23 华为技术有限公司 一种图像处理方法及其相关设备
CN117173626A (zh) * 2023-07-27 2023-12-05 华为技术有限公司 一种目标检测方法及其相关设备

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108259774A (zh) * 2018-01-31 2018-07-06 珠海市杰理科技股份有限公司 图像合成方法、系统和设备
CN112233032A (zh) * 2020-10-15 2021-01-15 浙江大学 一种高动态范围图像鬼影消除的方法
CN113592726A (zh) * 2021-06-29 2021-11-02 北京旷视科技有限公司 高动态范围成像方法、装置、电子设备和存储介质
CN114862734A (zh) * 2022-05-23 2022-08-05 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN115471435A (zh) * 2022-09-21 2022-12-13 Oppo广东移动通信有限公司 图像融合方法及装置、计算机可读介质和电子设备
US20220417414A1 (en) * 2020-04-28 2022-12-29 Honor Device Co., Ltd. High dynamic range image synthesis method and electronic device
CN116309226A (zh) * 2023-03-15 2023-06-23 华为技术有限公司 一种图像处理方法及其相关设备

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108259774A (zh) * 2018-01-31 2018-07-06 珠海市杰理科技股份有限公司 图像合成方法、系统和设备
US20220417414A1 (en) * 2020-04-28 2022-12-29 Honor Device Co., Ltd. High dynamic range image synthesis method and electronic device
CN112233032A (zh) * 2020-10-15 2021-01-15 浙江大学 一种高动态范围图像鬼影消除的方法
CN113592726A (zh) * 2021-06-29 2021-11-02 北京旷视科技有限公司 高动态范围成像方法、装置、电子设备和存储介质
CN114862734A (zh) * 2022-05-23 2022-08-05 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和计算机可读存储介质
CN115471435A (zh) * 2022-09-21 2022-12-13 Oppo广东移动通信有限公司 图像融合方法及装置、计算机可读介质和电子设备
CN116309226A (zh) * 2023-03-15 2023-06-23 华为技术有限公司 一种图像处理方法及其相关设备

Also Published As

Publication number Publication date
CN116309226A (zh) 2023-06-23

Similar Documents

Publication Publication Date Title
CN113284054B (zh) 图像增强方法以及图像增强装置
CN112580720B (zh) 一种模型训练方法及装置
CN113066001B (zh) 一种图像处理方法及相关设备
WO2022134971A1 (fr) Procédé de formation de modèle de réduction de bruit et appareil associé
WO2021018163A1 (fr) Procédé et appareil de recherche de réseau neuronal
CN114359289B (zh) 一种图像处理方法及相关装置
CN112862828B (zh) 一种语义分割方法、模型训练方法及装置
CN113284055B (zh) 一种图像处理的方法以及装置
WO2024188171A1 (fr) Procédé de traitement des images et son dispositif associé
WO2022111387A1 (fr) Procédé de traitement de données et appareil associé
WO2025021142A1 (fr) Procédé de détection d'objets et dispositif associé
WO2024104365A1 (fr) Procédé de mesure de température de dispositif et dispositif associé
WO2024217411A1 (fr) Procédé sensible au scénario et dispositif associé
WO2024179510A1 (fr) Procédé de traitement d'images et dispositif associé
WO2024160186A1 (fr) Procédé d'entraînement de modèle et dispositif associé
WO2025002088A1 (fr) Procédé de détection d'objet et dispositif associé
WO2025016352A1 (fr) Procédé d'évaluation de vidéo et ses dispositifs associés
WO2024179503A1 (fr) Procédé de traitement de la parole et dispositif associé
WO2024179485A1 (fr) Procédé de traitement d'image et son dispositif associé
CN112288638A (zh) 图像增强设备以及系统
CN115712828A (zh) 一种图像分类方法及其相关设备
CN116309074A (zh) 一种数据处理方法及其装置
WO2025031343A1 (fr) Procédé de traitement d'images et dispositif associé
WO2024245216A1 (fr) Procédé de traitement d'images et son appareil associé
WO2024175014A1 (fr) Procédé de traitement d'image et son dispositif associé

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24769864

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE