[go: up one dir, main page]

WO2025087392A1 - Procédé et appareil de traitement de données multimédia, et dispositif électronique et support de stockage - Google Patents

Procédé et appareil de traitement de données multimédia, et dispositif électronique et support de stockage Download PDF

Info

Publication number
WO2025087392A1
WO2025087392A1 PCT/CN2024/127406 CN2024127406W WO2025087392A1 WO 2025087392 A1 WO2025087392 A1 WO 2025087392A1 CN 2024127406 W CN2024127406 W CN 2024127406W WO 2025087392 A1 WO2025087392 A1 WO 2025087392A1
Authority
WO
WIPO (PCT)
Prior art keywords
multimedia data
reference image
image
original multimedia
color
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/127406
Other languages
English (en)
Chinese (zh)
Inventor
龚立雪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2025087392A1 publication Critical patent/WO2025087392A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/86—Camera processing pipelines; Components thereof for processing colour signals for controlling the colour saturation of colour signals, e.g. automatic chroma control circuits
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/85—Camera processing pipelines; Components thereof for processing colour signals for matrixing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
    • H04N23/88—Camera processing pipelines; Components thereof for processing colour signals for colour balance, e.g. white-balance circuits or colour temperature control

Definitions

  • the embodiments of the present disclosure relate to image technology, and more particularly to a method, device, electronic device and storage medium for processing multimedia data.
  • the current color adjustment method mainly adjusts parameters manually, which makes it difficult to quickly and accurately obtain a color adjustment result that meets the color adjustment requirements.
  • an embodiment of the present disclosure provides a method for processing multimedia data, comprising: obtaining original multimedia data; obtaining a reference image; adjusting image frames in the original multimedia data according to display properties of the reference image to obtain target multimedia data, and displaying the target multimedia data.
  • the target multimedia data includes a sequence of image frames in the original multimedia data that are adjusted based on the display properties of the reference image.
  • the embodiments of the present disclosure further provide a multimedia data processing device, the device comprising: a data acquisition module, used to acquire original multimedia data; an image acquisition module, used to acquire a reference image; an image adjustment module, used to adjust the image frames in the original multimedia data according to the display attributes of the reference image, obtain target multimedia data, and display the target multimedia data.
  • the target multimedia data includes a sequence of image frames in the original multimedia data that are adjusted based on the display attributes of the reference image.
  • an embodiment of the present disclosure further provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method for processing multimedia data as described in any embodiment of the present disclosure.
  • an embodiment of the present disclosure further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the method for processing multimedia data as described in any embodiment of the present disclosure.
  • FIG1 is a schematic flow chart of a method for processing multimedia data provided by an embodiment of the present disclosure
  • FIG2 is a schematic flow chart of another method for processing multimedia data provided by an embodiment of the present disclosure.
  • FIG3 is a schematic diagram of superimposing and displaying a reference image in an image frame of original multimedia data provided by an embodiment of the present disclosure
  • FIG4 is a schematic diagram of a reference image superimposed with line material provided by an embodiment of the present disclosure.
  • FIG5 is a schematic diagram of a color block special effect provided by an embodiment of the present disclosure.
  • FIG6 is a schematic diagram of a color block shift provided by an embodiment of the present disclosure.
  • FIG7 is a schematic diagram of the structure of a multimedia data processing device provided by an embodiment of the present disclosure.
  • FIG8 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.
  • the embodiments of the present disclosure provide a method, device, electronic device and storage medium for processing multimedia data, which can improve color adjustment efficiency and accuracy.
  • the disclosed embodiments provide a method, device, electronic device and storage medium for processing multimedia data, which adjusts the image frames in the original multimedia data by the display attributes of the reference image, obtains the target multimedia data, and displays the target multimedia data, so as to adjust the display style of the original multimedia data based on the color style of the reference image, so that the target multimedia data presents a display effect similar to the reference image, omits the step of manually adjusting the display attributes, and improves the color adjustment efficiency. Since the reference image can intuitively present the color adjustment result desired by the user, adjusting the original multimedia data based on the display attributes of the reference image can improve the adjustment accuracy and avoid repeated modification of parameters.
  • Figure 1 is a flow chart of a method for processing multimedia data provided by an embodiment of the present disclosure.
  • the embodiment of the present disclosure is applicable to the case of video color grading.
  • the method can be executed by a multimedia data processing device, which can be implemented in the form of software and/or hardware.
  • a multimedia data processing device which can be implemented in the form of software and/or hardware.
  • an electronic device which can be a mobile terminal, a PC or a server, etc.
  • the method comprises:
  • the original multimedia data may be a video or image selected by the user to be color-adjusted, etc.
  • a historical video in the client may be specified as the original multimedia data.
  • multimedia data captured by the user in real time may be obtained as the original multimedia data.
  • multimedia data selected by the user from a local album may be obtained as the original multimedia data.
  • the original multimedia data is obtained and played. The display properties of the original multimedia data are adjusted without interrupting the playback.
  • the reference image represents a reference image for adjusting the display properties of the image frame of the original multimedia data.
  • the reference image may be an image uploaded by a user that meets the color adjustment requirements.
  • the reference image may be an image taken by a user, or may be an image downloaded from the Internet, and the embodiment of the present disclosure does not limit the source of the reference image.
  • the reference image uploaded by the user is obtained. Since the display style of the reference image meets the color adjustment requirements of the user, the step of repeatedly modifying parameters in order to obtain a color adjustment result that meets the color adjustment requirements during the manual color adjustment process can be avoided, thereby improving the color adjustment efficiency.
  • the display attribute represents the display effect of the reference image in terms of color and brightness.
  • the display attribute may include brightness attribute and color attribute.
  • the target multimedia data may be Characterize the color adjustment result of the original multimedia data.
  • the target multimedia data includes a sequence of image frames in the original multimedia data adjusted based on the display attributes of the reference image, and the target multimedia data presents a display effect similar to the reference image. Exemplarily, for the image frames in the reference image and the original multimedia data, the target pixel interval corresponding to the image is determined according to the display attributes respectively.
  • attribute matching is performed on the image frame in the original multimedia data based on the display attributes of the pixels of the reference image.
  • the attribute matching results corresponding to each target pixel interval are fused to obtain the multimedia data processing results corresponding to the image frame.
  • Attribute matching may include matching attributes such as color and brightness.
  • the attribute matching result represents the processing result of migrating the attribute features of the reference image to the image frame of the original multimedia data. By migrating the attribute features of the reference image to the image frame of the original multimedia data, the image frame of the original multimedia data presents display attributes similar to the reference image.
  • the display attributes of the pixel are adjusted according to the display attributes of the reference image to obtain the skin color adjustment result corresponding to the image frame.
  • the skin color adjustment result and the multimedia data processing result are fused according to the skin color weight to obtain the target multimedia data.
  • the skin color weight represents the probability that each pixel in the image frame of the original multimedia data belongs to the skin color area.
  • the reference image and the image frame of the original multimedia data are respectively converted into color space. Based on the color space conversion result, the brightness channel data of the reference image and the image frame of the original multimedia data are respectively obtained. According to the brightness channel data, the histograms corresponding to the reference image and the image frame of the original multimedia data are respectively determined. According to the number of pixels corresponding to each brightness value in the histogram of the reference image, the brightness segmentation point corresponding to the reference image is determined. Then, according to the brightness segmentation point corresponding to the reference image, the reference image is segmented into a plurality of target pixel intervals.
  • the brightness segmentation point corresponding to the image frame in the original multimedia data is determined. Then, according to the brightness segmentation point corresponding to the image frame, the image frame is segmented into a plurality of target pixel intervals.
  • the target pixel intervals include shadow intervals, midtone intervals, and highlight intervals.
  • the pixels of the reference image are counted according to the brightness attribute, and the brightness segmentation points are determined according to the pixel statistics. Then, the reference image is segmented into a plurality of target pixel intervals according to the brightness segmentation points. The number of target pixel intervals is determined based on the number of brightness segmentation points, and adjacent target pixel intervals have overlapping areas.
  • the reference image is converted to a YUV color space.
  • the Y channel data is determined to be a brightness attribute, and a histogram of the Y channel of the reference image is generated through the Y channel data.
  • the pixels of the shadow, midtone, and highlight are determined according to each brightness value in the histogram of the Y channel.
  • the alternative segmentation point is determined so that the number of pixels corresponding to the shadow, midtone, and highlight is the same.
  • the alternative segmentation point is determined so that the areas of the shadow interval, the midtone interval, and the highlight interval are all the same.
  • adjacent intervals can be staggered by a certain range so that adjacent intervals have overlapping areas.
  • the area of the overlapping area can be set according to the actual application scenario. For example, the overlapping area can be set to n times the area of the color area corresponding to the reference image, and 0 ⁇ n ⁇ 1.
  • the brightness segmentation point is determined based on the brightness value corresponding to the starting point and the brightness value corresponding to the end point of each overlapping area. For example, if the target pixel interval includes the shadow interval, the midtone interval and the highlight interval, there are two overlapping areas in the target pixel interval, namely the overlapping area between the shadow interval and the midtone interval and the overlapping area between the midtone interval and the highlight interval. There are four brightness segmentation points in the target pixel interval, namely S1, S2, S3 and S4, and S1 ⁇ S2 ⁇ S3 ⁇ S4.
  • the range of the shadow interval is [0, S2]
  • the range of the midtone interval is [S1, S4]
  • the range of the highlight interval is [S3, 255].
  • the brightness segmentation points and the range of the target pixel interval of the image frame in the original multimedia data are determined in a manner similar to the above manner, which will not be described in detail here.
  • the first weight corresponding to the pixel and each target pixel interval is determined according to the brightness attribute of the pixel and the brightness segmentation point corresponding to the overlapping area.
  • the first weight corresponding to the pixel and each target pixel interval is determined according to the target pixel interval to which the pixel belongs.
  • a number of first weights can be determined according to its brightness.
  • the number of first weights is consistent with the number of target pixel intervals. For example, if the image frame in the original multimedia data is divided into a shadow interval, a midtone interval, and a highlight interval, three first weights of each pixel in the image frame in the original multimedia data can be calculated. That is, the shadow weight W shadow of the pixel corresponding to the shadow interval, the midtone weight W mid of the pixel corresponding to the midtone interval, and the highlight weight W highlight of the pixel corresponding to the highlight interval, and the sum of the three weights is 1.
  • the first weight of the pixel corresponding to the highlight range 0.
  • the first weights of pixels completely in the midtone range and the highlight range can be determined in the same manner, which will not be repeated here.
  • the pixel has both a shadow weight and a midtone weight.
  • the first weight W shadow of the pixel corresponding to the shadow interval is determined
  • the first weight W mid of the pixel corresponding to the midtone interval is determined
  • the first weight W highlight of the pixel corresponding to the highlight interval is determined to be 0.
  • the first weight corresponding to the reference image can be determined by using a method similar to the above, which will not be described in detail here.
  • attribute matching is performed on the image frames in the original multimedia data based on the display attributes of the pixels of the reference image.
  • the attribute matching results corresponding to each of the target pixel intervals are fused to obtain the multimedia data processing results corresponding to the image frames, including: performing brightness matching on the image frames in the original multimedia data based on the brightness attribute of the reference image to obtain brightness matching results of the image frames in the original multimedia data.
  • the image frames in the original multimedia data are converted to the YUV color space. Color matching is performed on the UV channel, and brightness matching is performed on the Y channel.
  • a histogram matching algorithm is used to perform histogram matching on the Y channel of the image frame in the original multimedia data and the Y channel of the reference image. Match and get the brightness matching result.
  • the color features and transformation matrix corresponding to the target pixel interval are determined according to the color attributes of each target pixel interval.
  • the color features represent the color distribution of pixels in the target pixel interval corresponding to the image frame and the reference image in the original multimedia data.
  • the color features may include UV channel mean and covariance matrix, etc.
  • the UV channel data of the pixels in the shadow interval, the midtone interval and the highlight interval are obtained respectively.
  • the UV mean and the covariance matrix are calculated according to the UV channel data.
  • the transformation matrix M is determined according to the UV mean and the covariance matrix.
  • the color features and transformation matrix corresponding to the target pixel interval are determined according to the color attributes of each target pixel interval, including: determining the first color mean and the first covariance matrix corresponding to the target pixel interval according to the color attributes of the pixels in each target pixel interval of the image frame in the original multimedia data. Determining the second color mean and the second covariance matrix corresponding to the target pixel interval according to the color attributes of the pixels in each target pixel interval of the reference image; determining the transformation matrix corresponding to the target pixel interval according to the first color mean, the second color mean, the first covariance matrix and the second covariance matrix, wherein the color features include the first color mean and the second color mean.
  • the UV channel mean of all shadow pixels in the image frame in the original multimedia data is calculated.
  • Covariance matrix ⁇ src The first color mean can be expressed as and The first covariance matrix can be expressed as ⁇ src .
  • Calculate the UV channel mean of all shadow pixels in the reference image The covariance matrix ⁇ ref .
  • the second color mean can be expressed as and The second covariance matrix can be expressed as ⁇ ref .
  • the transformation matrix M is determined based on the above mean and covariance matrix using the MKL algorithm.
  • the method further includes: determining an eigenvalue of the first covariance matrix.
  • the eigenvalue represents the shape of an ellipse presented by the pixels of the image frame in the original multimedia data according to the Gaussian distribution.
  • the unit matrix is used as the transformation matrix.
  • the preset condition represents the shape threshold of the ellipse presented by the Gaussian distribution of the pixels of the image frame in the original multimedia data. If the Gaussian distribution exceeding the shape threshold is subjected to color transformation, color abnormal points may be generated. In this case, the transformation matrix is calculated based on the covariance matrix, and the transformation matrix M is set to the unit matrix instead.
  • the eigenvalues e1 and e2 of the covariance matrix are determined according to the covariance matrix ⁇ src of the image frame in the original multimedia data, and e1>e2.
  • the eigenvalues of the covariance matrix represent the major axis e1 and minor axis e2 of the ellipse presented by the pixels of the image frame in the original multimedia data according to the Gaussian distribution. If e1/e2> the set shape threshold, it means that the ellipse presented by the Gaussian distribution is very flat. At this time, if the Gaussian distribution is subjected to color transformation, color outliers are likely to be generated. Therefore, the transformation matrix M is set to the unit matrix.
  • the first color mean values of the image frames in the original multimedia data in the shadow interval, the mid-tone interval and the highlight interval can be calculated respectively, which are: and and and
  • the covariance matrices of the image frames in the original multimedia data in the shadow interval, the midtone interval and the highlight interval are calculated respectively, namely ⁇ src0 , ⁇ src1 , ⁇ src2 .
  • the transformation matrices M 0 , M 1 , and M 2 of the image frames in the original multimedia data in the shadow interval, the midtone interval and the highlight interval are calculated respectively.
  • Color matching is performed on each of the target pixel intervals corresponding to the image frame in the original multimedia data according to the color features and the transformation matrix.
  • the second weight corresponding to the pixel is determined according to color information of the pixel in the target pixel interval and the first color mean value.
  • the second weight represents the degree of deviation of the color of the pixel from the mean in the Gaussian distribution.
  • the smaller the second weight the more the color value of the pixel deviates from the mean of the Gaussian distribution.
  • the larger the second weight the closer the color value of the pixel is to the mean of the Gaussian distribution.
  • the color information uv of the pixels in the shadow interval of the image frame in the original multimedia data in the UV channel can be expressed as a two-dimensional vector
  • the first color mean value of the shadow interval of the image frame in the original multimedia data It can be expressed as According to the pixel color information uv, the first color mean
  • the second weight is calculated by using the first covariance matrix ⁇ src0 .
  • the second weights corresponding to the pixels in the midtone interval and the highlight interval are calculated, which will not be repeated here.
  • the shadow interval Take the shadow interval as an example. Perform a color transformation on each pixel in the shadow interval of the image frame in the original multimedia data based on the transformation matrix to obtain new color information as the first color matching result. Adjust the color information of each pixel in the shadow interval of the image frame in the original multimedia data based on the color mean of the image frame in the original multimedia data and the reference image in the shadow interval to obtain new color information as the second color matching result. For example, the color information uv of the pixel of the image frame in the original multimedia data is combined with the transformation matrix M 0 corresponding to the shadow interval to obtain the first color matching result corresponding to the shadow interval, thereby realizing the color style transfer of the reference image to the image frame in the original multimedia data.
  • the color information uv of the pixel of the image frame in the original multimedia data minus the color information uv of the pixel of the image frame in the original multimedia data Back Fold With reference image A second color matching result corresponding to the shadow interval is obtained.
  • the second color mean of the shadow interval of the reference image can be expressed as
  • the first color matching result and the second color matching result are fused according to the second weight to obtain a color matching result of the target pixel interval.
  • the fusion coefficient is determined according to the second weight, and the first color matching result and the second color matching result corresponding to the shadow interval are fused according to the fusion coefficient to obtain the color matching result of the shadow interval.
  • the fusion coefficient includes the second weight, and the sum of the fusion coefficients is 1.
  • the color matching results of the midtone interval and the color matching results of the highlight interval are calculated in a similar manner, which will not be repeated here.
  • the midtone interval or the highlight interval there is only a color matching result for the shadow interval.
  • the midtone interval For pixels in the image frame in the original multimedia data whose brightness belongs to the overlapping area of the shadow interval and the midtone interval, there are a color matching result for the shadow interval and a color matching result for the midtone interval.
  • the midtone interval For pixels in the image frame in the original multimedia data whose brightness belongs to the overlapping area of the midtone interval and the highlight interval, there are a color matching result for the midtone interval and a color matching result for the highlight interval.
  • the color matching result of the target pixel interval is fused according to the first weight corresponding to the pixel and each target pixel interval to obtain the color matching result of the image frame in the original multimedia data; and the processing result image of the image frame in the original multimedia data is determined according to the color matching result and brightness matching result of the image frame in the original multimedia data as the multimedia data processing result corresponding to the image frame.
  • the first weights i.e., shadow weight, midtone weight, and highlight weight
  • the respective target pixel intervals i.e., shadow interval, midtone interval, and highlight interval
  • the color matching results of the pixel in the corresponding target pixel intervals i.e., shadow weight, midtone weight, and highlight weight
  • the color matching result of the pixel in the shadow interval For example, for each pixel of the image frame in the original multimedia data, according to the shadow weight, midtone weight and highlight weight of the pixel, the color matching result of the pixel in the shadow interval, the color matching result of the pixel in the midtone interval and the color matching result of the pixel in the highlight interval are fused. Thus, the color matching result of the image frame in the original multimedia data is obtained.
  • the display attributes of the pixels are adjusted according to the display attributes of the skin color area of the reference image to obtain a skin color adjustment result corresponding to the image frame. This includes: determining the probability that each pixel in the image frame of the original multimedia data belongs to a skin color area as a skin color weight, and determining the pixels included in the skin color area in the image frame of the original multimedia data according to the skin color weight.
  • the display attributes of the pixels are adjusted according to the mean values of the display attributes of the image frame and the reference image in the skin color area, respectively, to obtain a skin color adjustment result corresponding to the image frame.
  • the original multimedia data is input into the skin color segmentation model.
  • the skin color segmentation model is used to perform skin color segmentation on the image frames in the original multimedia data, and the probability that each pixel in the image frame belongs to the skin color area is obtained. The pixels contained in the skin color area of the image frame in the original multimedia data are determined according to the probability.
  • the reference image is input into the skin color segmentation model.
  • the skin color segmentation model is used to perform skin color segmentation on the reference image, and the probability that each pixel in the reference image belongs to the skin color area is obtained. The pixels contained in the skin color area of the reference image are determined according to the probability.
  • the brightness mean of the skin color area of the image frame in the original multimedia data is determined.
  • the brightness mean of the skin color area of the reference image is determined.
  • the chromaticity of each pixel contained in the skin color area of the image frame in the original multimedia data is determined.
  • the chromaticity mean of the skin color area of the reference image is determined.
  • the difference between the brightness of the pixel and the average brightness of the skin color area is calculated, and the difference is superimposed with the average brightness of the skin color area of the reference image to obtain a brightness adjustment result of the pixel.
  • the difference between the chromaticity of the pixel and the chromaticity mean of the skin color area is calculated, and the difference is superimposed with the chromaticity mean of the skin color area of the reference image to obtain the chromaticity adjustment result of the pixel.
  • a skin color adjustment result corresponding to the image frame is obtained.
  • the target multimedia data is obtained by fusing the multimedia data processing result and the skin color adjustment result of the image frame in a linear fusion manner based on the skin color weight. For example, the target multimedia data is determined based on the fusion result of the multimedia data processing result and the skin color adjustment result corresponding to the image frame sequence included in the target multimedia data.
  • the technical solution of the disclosed embodiment adjusts the image frame in the original multimedia data by the display attribute of the reference image, obtains the target multimedia data and displays it.
  • the display style of the original multimedia data is adjusted based on the color style of the reference image, so that the target multimedia data presents a display effect similar to that of the reference image, omitting the step of manually adjusting the display attribute, and improving the color adjustment efficiency. Since the reference image can intuitively present the color adjustment result desired by the user, adjusting the original multimedia data based on the display attribute of the reference image can improve the adjustment accuracy and avoid repeated modification of parameters.
  • FIG. 2 is a flow chart of another method for processing multimedia data provided by an embodiment of the present disclosure. Embodiment Based on the above embodiment, additional display steps of reference images and special effect elements are added. As shown in FIG2 , the method includes: S210, obtaining original multimedia data. S220, obtaining a reference image. S230, generating special effect elements according to the reference image, and superimposing and displaying the reference image and special effect elements on the original multimedia data.
  • the display effect is relatively simple and the user experience is poor. It may even cause the user to mistakenly believe that the reference image has not been successfully uploaded and repeat the operation, resulting in erroneous processing.
  • the single display effect and the erroneous operation of repeatedly uploading the reference image can be avoided.
  • the special effect element is superimposed on the set position of the reference image and is displayed and moves in a set direction.
  • the special effect element includes a color block special effect, etc.
  • the color block special effect may include color blocks of different colors.
  • the color of the color block represents the color category obtained by clustering the colors of the pixels of the reference image.
  • the color of the color block may also be a fixed color, and the embodiment of the present disclosure does not specifically limit the shape and color of the color block.
  • the color category included in the reference image is determined according to the color attribute of the pixels in the reference image, and a special effect element corresponding to the color represented by the color category is generated.
  • multiple cluster centers are pre-set, and pixels in the reference image are clustered according to the colors of the pixels and the colors of the pixels corresponding to the cluster centers to obtain multiple pixel groups.
  • the colors corresponding to the cluster centers of all groups of pixels can be used as the color categories included in the reference image.
  • FIG3 is a schematic diagram of superimposing and displaying a reference image in an image frame of original multimedia data provided by an embodiment of the present disclosure. As shown in FIG3, during the playback of the original multimedia data, the reference image is superimposed and displayed at a first position 310 of the original multimedia data at time t1 , and slides to a second position 320 at time t2 .
  • FIG4 is a schematic diagram of a reference image superimposed with line material provided by an embodiment of the present disclosure.
  • a mask image 410 with set transparency is generated according to the width and height of the reference image, and there are bright areas 411 and dark areas 412 in the mask image 410.
  • a dynamic image is synthesized based on multiple mask images 410 with different positions of bright areas 411. The dynamic image is superimposed on the reference image to obtain the effect of light scanning the reference image.
  • the special effect element is superimposed on the first area of the reference image according to the set superposition form.
  • the set superposition form represents the form of superimposing the color block special effect on the reference image.
  • the set superposition form can be to display the color blocks in different positions of the reference image in sequence according to the color of the color blocks.
  • the first area can be a preset area within the reference image.
  • FIG5 is a schematic diagram of a color block special effect provided by an embodiment of the present disclosure. As shown in FIG5, each color block 520 is displayed one by one in the reference image 510 until all the color blocks 520 are displayed in the first area of the reference image 510. During the display process, the width and height of the color block are changed from a first state to a second state.
  • the display position and brightness arrangement order corresponding to the special effect element are determined according to the set superposition form.
  • the display order of the special effect element is determined according to the brightness arrangement order.
  • the special effect element is superimposed and displayed in the first area of the reference image in the image frame in the original multimedia data.
  • the display position represents the position of the special effect element in the reference image.
  • the display position may include color blocks arranged horizontally (or vertically or obliquely) in the reference image, and the spacing between adjacent color blocks is the same (or gradually increases or gradually decreases).
  • the brightness arrangement order may include color or brightness arranged from dark to light, or from light to dark, etc.
  • each color block represents the color appearing in the reference image, and the color blocks have different color depths and different brightness.
  • the display order of the color blocks in the reference image can be determined according to the brightness sorting order specified in the set superposition form. Then, the color blocks are displayed one by one at the display position corresponding to the color blocks according to the display order.
  • the first state of the color block is first displayed in the reference image, and the process of the color block converting from the first state to the second state is presented. For example, a small color block is first displayed in the reference image, and then the small color block is enlarged to obtain a color block with a set width and height.
  • FIG. 6 is a schematic diagram of a color block shift provided by an embodiment of the present disclosure. As shown in Figure 6, after all color blocks are displayed in the first area of the reference image, all color blocks 610 are rotated from a horizontal arrangement to a vertical arrangement, and moved to the second area 620 corresponding to the reference image. After all color blocks 610 reach the second area 620, the spacing between adjacent color blocks is increased. S240. Adjust the image frame in the original multimedia data according to the display attributes of the reference image to obtain target multimedia data. S250. When the display state of the special effect element in the original multimedia data meets the set conditions, display the target multimedia data.
  • the set condition may represent a condition under which the special effect element disappears from the original multimedia data, and when the special effect element disappears from the original multimedia data, the target multimedia data is displayed. For example, when the spacing between adjacent color blocks is increased so that the spacing between adjacent color blocks is a set value, it is determined that the special effect color meets the set condition. Alternatively, when the spacing between adjacent color blocks is increased and the display time of the color blocks reaches a set time, it is determined that the special effect color meets the set condition.
  • the technical solution of the disclosed embodiment can intuitively display the color style of the reference image selected by the user by generating special effect elements corresponding to the reference image and superimposing the reference image and the special effect elements in the original multimedia data. It can also avoid erroneous operations such as a single display effect or repeated uploading of reference images in the process of adjusting the display properties of the original multimedia data based on the reference image by vividly displaying the special effect elements, thereby increasing interactivity and fun and improving user experience.
  • FIG. 7 is a schematic diagram of the structure of a multimedia data processing device provided in an embodiment of the present disclosure.
  • the device may be implemented in the form of software and/or hardware.
  • it may be implemented by an electronic device, which may be a mobile terminal, a PC, a server, etc.
  • the device includes: an image acquisition module 710 and an image adjustment module 720.
  • Block 710 is used to obtain original multimedia data;
  • image acquisition module 720 is used to obtain a reference image;
  • image adjustment module 730 is used to adjust the image frames in the original multimedia data according to the display properties of the reference image to obtain target multimedia data and display the target multimedia data.
  • the target multimedia data includes a sequence of image frames in the original multimedia data adjusted based on the display properties of the reference image.
  • the method further includes: a special effect element generation module, which is used to generate a special effect element according to the reference image after acquiring the reference image, and to superimpose and display the reference image and the special effect element in the original multimedia data.
  • the special effect element is superimposed on the reference image and displayed at a set position and moves in a set direction; when the display state of the special effect element in the original multimedia data meets the set condition, the step of displaying the target multimedia data is executed.
  • generating a special effect element according to the reference image includes: determining a color category included in the reference image according to color attributes of pixels in the reference image; and generating a special effect element corresponding to a color represented by the color category.
  • the superimposing and displaying the reference image and special effect element in the original multimedia data includes: superimposing and displaying the reference image in the image frame of the original multimedia data; superimposing and displaying line material in the reference image, wherein the line material moves along a first direction in the reference image; superimposing the special effect element to a first area in the reference image according to a set superimposition form; moving the special effect element from the first area of the reference image along a second direction to a second area corresponding to the reference image, and adjusting the display status of the special effect element in the second area.
  • superimposing the special effect element to the first area in the reference image according to the set superimposition form includes: determining a display position and a brightness arrangement order corresponding to the special effect element according to the set superimposition form; determining a display order of the special effect elements according to the brightness arrangement order; and superimposing and displaying the special effect element in the first area of the reference image in the image frame of the original multimedia data according to the display position and display order.
  • the image adjustment module 720 is specifically used to: for the reference image and the image frames in the original multimedia data, determine the target pixel intervals corresponding to the images according to the display attributes respectively; for each of the target pixel intervals, perform attribute matching on the image frames in the original multimedia data based on the display attributes of the pixels of the reference image, fuse the attribute matching results corresponding to each of the target pixel intervals, and obtain the multimedia data processing results corresponding to the image frames; for the pixels in the skin color area of the image frames of the original multimedia data, adjust the display attributes of the pixels according to the display attributes of the skin color area of the reference image, and obtain the skin color adjustment results corresponding to the image frames; for the image frames in the original multimedia data, fuse the skin color adjustment results and the multimedia data processing results according to the skin color weight, and obtain the target multimedia data.
  • the skin color weight represents the probability that each pixel in the image frame of the original multimedia data belongs to the skin color area.
  • adjusting the display attribute of the pixel according to the display attribute of the skin color area of the reference image to obtain a skin color adjustment result corresponding to the image frame includes: determining a probability that each pixel in the image frame of the original multimedia data belongs to the skin color area, as the skin color
  • the method comprises the steps of: determining pixels included in a skin color area in an image frame of the original multimedia data according to the skin color weight; determining a mean value of display attributes of the skin color area in the image frame of the original multimedia data according to display attributes of the pixels included in the skin color area in the image frame of the original multimedia data; and adjusting the display attributes of the pixels in the skin color area in the image frame of the original multimedia data according to the mean values of display attributes of the image frame and a reference image in the skin color area, so as to obtain a skin color adjustment result corresponding to the image frame.
  • the multimedia data processing device provided in the embodiments of the present disclosure can execute the multimedia data processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.
  • FIG8 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.
  • the terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG8 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.
  • the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803.
  • a processing device 801 e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • various programs and data required for the operation of the electronic device 800 are also stored.
  • the processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804.
  • An edit/output (I/O) interface 805 is also connected to the bus 804.
  • the following devices may be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809.
  • the communication device 809 may allow the electronic device 800 to communicate wirelessly or wired with other devices to exchange data.
  • FIG. 8 shows an electronic device 800 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802.
  • the processing device 801 the execution The above functions are defined in the method of the embodiment of the present disclosure.
  • the electronic device provided by the embodiment of the present disclosure and the method for processing multimedia data provided by the above embodiment belong to the same inventive concept.
  • the embodiments of the present disclosure provide a computer storage medium on which a computer program is stored.
  • the program is executed by a processor, the method for processing multimedia data provided by the above embodiments is implemented.
  • the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried.
  • This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above.
  • the computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device.
  • the program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network).
  • HTTP HyperText Transfer Protocol
  • Examples of communication networks include a local area network ("LAN”), a wide area network ("WAN”), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
  • the computer-readable medium carries one or more programs.
  • the electronic device When the one or more programs are executed by the electronic device, the electronic device: obtains original multimedia data; obtains a reference image; adjusts the image frame in the original multimedia data according to the display attribute of the reference image to obtain target multimedia data, and displays the target multimedia data.
  • the target multimedia data includes an image frame sequence in the original multimedia data that is adjusted based on the display attributes of the reference image.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., AT&T, MCI, Sprint, EarthLink, MSN, GTE, etc.
  • each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function.
  • the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved.
  • each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or hardware.
  • the name of a unit does not limit the unit itself in some cases.
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chips
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Processing (AREA)

Abstract

Les modes de réalisation de la présente divulgation concernent un procédé et un appareil de traitement de données multimédias, un dispositif électronique et un support de stockage. Le procédé consiste à : acquérir des données multimédias d'origine ; acquérir une image de référence ; et ajuster une trame d'image dans les données multimédias d'origine sur la base d'un attribut d'affichage de l'image de référence, de façon à obtenir des données multimédias cibles, et afficher les données multimédias cibles, les données multimédias cibles comprenant une séquence de trames d'image, qui est ajustée sur la base de l'attribut d'affichage de l'image de référence, dans les données multimédias d'origine. Selon les modes de réalisation de la présente divulgation, le style d'affichage de données multimédias d'origine est ajusté sur la base du style de couleur d'une image de référence, de telle sorte que les données multimédias cibles présentent un effet d'affichage similaire à l'image de référence. Par conséquent, l'étape de réglage manuel de l'attribut d'affichage est omise, ce qui permet d'améliorer l'efficacité de réglage de couleur. Étant donné que l'image de référence peut présenter visuellement un résultat de réglage de couleur que certains utilisateurs souhaitent, le fait d'ajuster les données multimédias d'origine sur la base de l'attribut d'affichage de l'image de référence peut améliorer la précision de réglage, ce qui permet d'éviter des modifications répétées de paramètres.
PCT/CN2024/127406 2023-10-27 2024-10-25 Procédé et appareil de traitement de données multimédia, et dispositif électronique et support de stockage Pending WO2025087392A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202311415449.X 2023-10-27
CN202311415449.XA CN117425086A (zh) 2023-10-27 2023-10-27 一种多媒体数据的处理方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2025087392A1 true WO2025087392A1 (fr) 2025-05-01

Family

ID=89529908

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/127406 Pending WO2025087392A1 (fr) 2023-10-27 2024-10-25 Procédé et appareil de traitement de données multimédia, et dispositif électronique et support de stockage

Country Status (2)

Country Link
CN (1) CN117425086A (fr)
WO (1) WO2025087392A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117425086A (zh) * 2023-10-27 2024-01-19 北京字跳网络技术有限公司 一种多媒体数据的处理方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116703707A (zh) * 2023-05-29 2023-09-05 深圳数联天下智能科技有限公司 训练肤色迁移模型的方法、生成美肤图像的方法及相关装置
CN116723357A (zh) * 2023-07-07 2023-09-08 北京字跳网络技术有限公司 视频特效添加方法、装置、设备、存储介质和程序产品
CN116758305A (zh) * 2023-05-23 2023-09-15 北京字跳网络技术有限公司 图像色彩调整方法、装置、设备、可读存储介质及产品
CN117425086A (zh) * 2023-10-27 2024-01-19 北京字跳网络技术有限公司 一种多媒体数据的处理方法、装置、电子设备及存储介质

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116758305A (zh) * 2023-05-23 2023-09-15 北京字跳网络技术有限公司 图像色彩调整方法、装置、设备、可读存储介质及产品
CN116703707A (zh) * 2023-05-29 2023-09-05 深圳数联天下智能科技有限公司 训练肤色迁移模型的方法、生成美肤图像的方法及相关装置
CN116723357A (zh) * 2023-07-07 2023-09-08 北京字跳网络技术有限公司 视频特效添加方法、装置、设备、存储介质和程序产品
CN117425086A (zh) * 2023-10-27 2024-01-19 北京字跳网络技术有限公司 一种多媒体数据的处理方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN117425086A (zh) 2024-01-19

Similar Documents

Publication Publication Date Title
CN112561840B (zh) 视频裁剪方法、装置、存储介质及电子设备
WO2022068451A1 (fr) Procédé et appareil de génération d'image de style, procédé et appareil de formation de modèle, dispositif et support
US20240112299A1 (en) Video cropping method and apparatus, storage medium and electronic device
WO2022142875A1 (fr) Procédé et appareil de traitement d'image, dispositif électronique et support de stockage
WO2022171036A1 (fr) Procédé de suivi de cible vidéo, appareil de suivi de cible vidéo, support de stockage et dispositif électronique
CN112348910B (zh) 获取图像的方法、装置、设备和计算机可读介质
CN110689478A (zh) 图像风格化处理方法、装置、电子设备及可读介质
WO2024037556A1 (fr) Appareil et procédé de traitement d'image, dispositif et support de stockage
CN112034984A (zh) 一种虚拟模型处理方法、装置、电子设备和存储介质
WO2025087392A1 (fr) Procédé et appareil de traitement de données multimédia, et dispositif électronique et support de stockage
WO2022233223A1 (fr) Procédé et appareil d'assemblage d'image, dispositif et support
CN112488947B (zh) 模型训练和图像处理方法、装置、设备和计算机可读介质
CN110399802B (zh) 处理面部图像眼睛亮度的方法、装置、介质和电子设备
CN114372974B (zh) 图像检测方法、装置、设备及存储介质
WO2025087280A1 (fr) Procédé et appareil de traitement d'image, dispositif électronique, et support de stockage
WO2023078284A1 (fr) Procédé et appareil de rendu d'image, dispositif, support de stockage et produit-programme
WO2025061019A1 (fr) Procédé et appareil de présentation d'image, support de stockage et dispositif électronique
WO2024131503A1 (fr) Procédé et appareil de génération d'images à effet spécial, dispositif et support de stockage
CN118840453A (zh) 文字颜色确定方法、装置、介质、电子设备及程序产品
US20230360286A1 (en) Image processing method and apparatus, electronic device and storage medium
CN112241941A (zh) 获取图像的方法、装置、设备和计算机可读介质
JP7752184B2 (ja) 画像輝度の調整方法、装置、電子機器及び媒体
CN114170341B (zh) 一种图像处理方法、装置、设备及介质
WO2024131652A1 (fr) Procédé et appareil de traitement d'effets spéciaux, dispositif électronique et support de stockage
US20230281983A1 (en) Image recognition method and apparatus, electronic device, and computer-readable medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24881752

Country of ref document: EP

Kind code of ref document: A1