WO2025087392A1 - Multimedia data processing method and apparatus, and electronic device and storage medium - Google Patents

Abstract

Provided in the embodiments of the present disclosure are a multimedia data processing method and apparatus, and an electronic device and a storage medium. The method comprises: acquiring original multimedia data; acquiring a reference image; and adjusting an image frame in the original multimedia data on the basis of a display attribute of the reference image, so as to obtain target multimedia data, and displaying the target multimedia data, wherein the target multimedia data comprises an image frame sequence, which is adjusted on the basis of the display attribute of the reference image, in the original multimedia data. According to the embodiments of the present disclosure, the display style of original multimedia data is adjusted on the basis of the color style of a reference image, such that the target multimedia data presents a display effect similar to the reference image. Therefore, the step of manually adjusting the display attribute is omitted, thereby improving the color adjustment efficiency. Since the reference image can visually present a color adjustment result that a user desires, adjusting the original multimedia data on the basis of the display attribute of the reference image can improve the adjustment accuracy, thereby avoiding repeated modifications of parameters.

Description

A method, device, electronic device and storage medium for processing multimedia data

This application claims priority to the Chinese invention patent application entitled “A method, device, electronic device and storage medium for processing multimedia data” filed on October 27, 2023, with application number 202311415449X. The entire contents of that application are incorporated by reference into this application.

Technical Field

The embodiments of the present disclosure relate to image technology, and more particularly to a method, device, electronic device and storage medium for processing multimedia data.

Background Art

With the popularization and application of mobile terminals and the Internet, more and more users can capture multimedia data and share the multimedia data through the Internet.

Before sharing, users may need to adjust the color style of multimedia data to obtain a color adjustment result that meets the requirements. The current color adjustment method mainly adjusts parameters manually, which makes it difficult to quickly and accurately obtain a color adjustment result that meets the color adjustment requirements.

Summary of the invention

In a first aspect, an embodiment of the present disclosure provides a method for processing multimedia data, comprising: obtaining original multimedia data; obtaining a reference image; adjusting image frames in the original multimedia data according to display properties of the reference image to obtain target multimedia data, and displaying the target multimedia data. The target multimedia data includes a sequence of image frames in the original multimedia data that are adjusted based on the display properties of the reference image.

In a second aspect, the embodiments of the present disclosure further provide a multimedia data processing device, the device comprising: a data acquisition module, used to acquire original multimedia data; an image acquisition module, used to acquire a reference image; an image adjustment module, used to adjust the image frames in the original multimedia data according to the display attributes of the reference image, obtain target multimedia data, and display the target multimedia data. The target multimedia data includes a sequence of image frames in the original multimedia data that are adjusted based on the display attributes of the reference image.

In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement a method for processing multimedia data as described in any embodiment of the present disclosure.

In a fourth aspect, an embodiment of the present disclosure further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the method for processing multimedia data as described in any embodiment of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

FIG1 is a schematic flow chart of a method for processing multimedia data provided by an embodiment of the present disclosure;

FIG2 is a schematic flow chart of another method for processing multimedia data provided by an embodiment of the present disclosure;

FIG3 is a schematic diagram of superimposing and displaying a reference image in an image frame of original multimedia data provided by an embodiment of the present disclosure;

FIG4 is a schematic diagram of a reference image superimposed with line material provided by an embodiment of the present disclosure;

FIG5 is a schematic diagram of a color block special effect provided by an embodiment of the present disclosure;

FIG6 is a schematic diagram of a color block shift provided by an embodiment of the present disclosure;

FIG7 is a schematic diagram of the structure of a multimedia data processing device provided by an embodiment of the present disclosure;

FIG8 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present disclosure is not limited in this respect.

The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. It should be understood by those skilled in the art that unless the context clearly indicates otherwise, it should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

The embodiments of the present disclosure provide a method, device, electronic device and storage medium for processing multimedia data, which can improve color adjustment efficiency and accuracy.

The disclosed embodiments provide a method, device, electronic device and storage medium for processing multimedia data, which adjusts the image frames in the original multimedia data by the display attributes of the reference image, obtains the target multimedia data, and displays the target multimedia data, so as to adjust the display style of the original multimedia data based on the color style of the reference image, so that the target multimedia data presents a display effect similar to the reference image, omits the step of manually adjusting the display attributes, and improves the color adjustment efficiency. Since the reference image can intuitively present the color adjustment result desired by the user, adjusting the original multimedia data based on the display attributes of the reference image can improve the adjustment accuracy and avoid repeated modification of parameters.

Figure 1 is a flow chart of a method for processing multimedia data provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the case of video color grading. The method can be executed by a multimedia data processing device, which can be implemented in the form of software and/or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a PC or a server, etc.

As shown in FIG1 , the method comprises:

S110, obtaining original multimedia data. The original multimedia data may be a video or image selected by the user to be color-adjusted, etc. In some embodiments, a historical video in the client may be specified as the original multimedia data. Alternatively, multimedia data captured by the user in real time may be obtained as the original multimedia data. Alternatively, multimedia data selected by the user from a local album may be obtained as the original multimedia data. Exemplarily, the original multimedia data is obtained and played. The display properties of the original multimedia data are adjusted without interrupting the playback.

S120, obtain a reference image. The reference image represents a reference image for adjusting the display properties of the image frame of the original multimedia data. In the embodiment of the present disclosure, the reference image may be an image uploaded by a user that meets the color adjustment requirements. The reference image may be an image taken by a user, or may be an image downloaded from the Internet, and the embodiment of the present disclosure does not limit the source of the reference image. Exemplarily, after detecting that the color adjustment function is turned on and obtaining the original multimedia data to be color adjusted selected by the user, the reference image uploaded by the user is obtained. Since the display style of the reference image meets the color adjustment requirements of the user, the step of repeatedly modifying parameters in order to obtain a color adjustment result that meets the color adjustment requirements during the manual color adjustment process can be avoided, thereby improving the color adjustment efficiency.

S130, adjusting the image frame in the original multimedia data according to the display attribute of the reference image to obtain target multimedia data, and displaying the target multimedia data. The display attribute represents the display effect of the reference image in terms of color and brightness. For example, the display attribute may include brightness attribute and color attribute. The target multimedia data may be Characterize the color adjustment result of the original multimedia data. The target multimedia data includes a sequence of image frames in the original multimedia data adjusted based on the display attributes of the reference image, and the target multimedia data presents a display effect similar to the reference image. Exemplarily, for the image frames in the reference image and the original multimedia data, the target pixel interval corresponding to the image is determined according to the display attributes respectively.

For each target pixel interval, attribute matching is performed on the image frame in the original multimedia data based on the display attributes of the pixels of the reference image. The attribute matching results corresponding to each target pixel interval are fused to obtain the multimedia data processing results corresponding to the image frame. Attribute matching may include matching attributes such as color and brightness. The attribute matching result represents the processing result of migrating the attribute features of the reference image to the image frame of the original multimedia data. By migrating the attribute features of the reference image to the image frame of the original multimedia data, the image frame of the original multimedia data presents display attributes similar to the reference image. For each pixel in the skin color area of the image frame of the original multimedia data, the display attributes of the pixel are adjusted according to the display attributes of the reference image to obtain the skin color adjustment result corresponding to the image frame. For the image frame in the original multimedia data, the skin color adjustment result and the multimedia data processing result are fused according to the skin color weight to obtain the target multimedia data. The skin color weight represents the probability that each pixel in the image frame of the original multimedia data belongs to the skin color area.

In the disclosed embodiment, the reference image and the image frame of the original multimedia data are respectively converted into color space. Based on the color space conversion result, the brightness channel data of the reference image and the image frame of the original multimedia data are respectively obtained. According to the brightness channel data, the histograms corresponding to the reference image and the image frame of the original multimedia data are respectively determined. According to the number of pixels corresponding to each brightness value in the histogram of the reference image, the brightness segmentation point corresponding to the reference image is determined. Then, according to the brightness segmentation point corresponding to the reference image, the reference image is segmented into a plurality of target pixel intervals. According to the number of pixels corresponding to each brightness value in the histogram of the image frame of the original multimedia data, the brightness segmentation point corresponding to the image frame in the original multimedia data is determined. Then, according to the brightness segmentation point corresponding to the image frame, the image frame is segmented into a plurality of target pixel intervals. For example, the target pixel intervals include shadow intervals, midtone intervals, and highlight intervals.

In some embodiments, the pixels of the reference image are counted according to the brightness attribute, and the brightness segmentation points are determined according to the pixel statistics. Then, the reference image is segmented into a plurality of target pixel intervals according to the brightness segmentation points. The number of target pixel intervals is determined based on the number of brightness segmentation points, and adjacent target pixel intervals have overlapping areas.

In some embodiments, the reference image is converted to a YUV color space. The Y channel data is determined to be a brightness attribute, and a histogram of the Y channel of the reference image is generated through the Y channel data. The pixels of the shadow, midtone, and highlight are determined according to each brightness value in the histogram of the Y channel. The alternative segmentation point is determined so that the number of pixels corresponding to the shadow, midtone, and highlight is the same. Alternatively, the alternative segmentation point is determined so that the areas of the shadow interval, the midtone interval, and the highlight interval are all the same.

In order to avoid color jumps in pixels corresponding to backup segmentation points, adjacent intervals can be staggered by a certain range so that adjacent intervals have overlapping areas. The area of the overlapping area can be set according to the actual application scenario. For example, the overlapping area can be set to n times the area of the color area corresponding to the reference image, and 0＜n＜1.

The brightness segmentation point is determined based on the brightness value corresponding to the starting point and the brightness value corresponding to the end point of each overlapping area. For example, if the target pixel interval includes the shadow interval, the midtone interval and the highlight interval, there are two overlapping areas in the target pixel interval, namely the overlapping area between the shadow interval and the midtone interval and the overlapping area between the midtone interval and the highlight interval. There are four brightness segmentation points in the target pixel interval, namely S1, S2, S3 and S4, and S1＜S2＜S3＜S4. The range of the shadow interval is [0, S2], the range of the midtone interval is [S1, S4], and the range of the highlight interval is [S3, 255].

The brightness segmentation points and the range of the target pixel interval of the image frame in the original multimedia data are determined in a manner similar to the above manner, which will not be described in detail here.

For each pixel of the image frame in the original multimedia data, if the pixel belongs to the overlapping area of the target pixel interval, then the first weight corresponding to the pixel and each target pixel interval is determined according to the brightness attribute of the pixel and the brightness segmentation point corresponding to the overlapping area. Alternatively, if the pixel belongs to the non-overlapping area of the target pixel interval, then the first weight corresponding to the pixel and each target pixel interval is determined according to the target pixel interval to which the pixel belongs.

After determining the brightness segmentation point, for each pixel of the image frame in the original multimedia data, a number of first weights can be determined according to its brightness. The number of first weights is consistent with the number of target pixel intervals. For example, if the image frame in the original multimedia data is divided into a shadow interval, a midtone interval, and a highlight interval, three first weights of each pixel in the image frame in the original multimedia data can be calculated. That is, the shadow weight W _shadow of the pixel corresponding to the shadow interval, the midtone weight W _mid of the pixel corresponding to the midtone interval, and the highlight weight W _highlight of the pixel corresponding to the highlight interval, and the sum of the three weights is 1.

For each pixel of the image frame in the original multimedia data, if the brightness of the pixel is completely in the shadow range, the first weight of the pixel corresponding to the shadow range is W _shadow = 1, the first weight of the pixel corresponding to the midtone range is W _mid = 0, and the first weight of the pixel corresponding to the highlight range is W _highlight = 0. The first weights of pixels completely in the midtone range and the highlight range can be determined in the same manner, which will not be repeated here.

If the brightness of the pixel belongs to the overlapping area of the shadow interval and the midtone interval, the pixel has both a shadow weight and a midtone weight. According to the brightness of the pixel and the brightness of the brightness segmentation points (S1 and S2) of the overlapping area, the first weight W _shadow of the pixel corresponding to the shadow interval is determined, the first weight W _mid of the pixel corresponding to the midtone interval is determined, and the first weight W _highlight of the pixel corresponding to the highlight interval is determined to be 0.

The first weight corresponding to the reference image can be determined by using a method similar to the above, which will not be described in detail here.

In some embodiments, for each of the target pixel intervals, attribute matching is performed on the image frames in the original multimedia data based on the display attributes of the pixels of the reference image. The attribute matching results corresponding to each of the target pixel intervals are fused to obtain the multimedia data processing results corresponding to the image frames, including: performing brightness matching on the image frames in the original multimedia data based on the brightness attribute of the reference image to obtain brightness matching results of the image frames in the original multimedia data. In order to match the image brightness and color separately, the image frames in the original multimedia data are converted to the YUV color space. Color matching is performed on the UV channel, and brightness matching is performed on the Y channel. For example, a histogram matching algorithm is used to perform histogram matching on the Y channel of the image frame in the original multimedia data and the Y channel of the reference image. Match and get the brightness matching result.

The color features and transformation matrix corresponding to the target pixel interval are determined according to the color attributes of each target pixel interval. The color features represent the color distribution of pixels in the target pixel interval corresponding to the image frame and the reference image in the original multimedia data. In the embodiment of the present disclosure, the color features may include UV channel mean and covariance matrix, etc.

For the image frame and the reference image in the original multimedia data, the UV channel data of the pixels in the shadow interval, the midtone interval and the highlight interval are obtained respectively. The UV mean and the covariance matrix are calculated according to the UV channel data. Then, the transformation matrix M is determined according to the UV mean and the covariance matrix.

In some embodiments, the color features and transformation matrix corresponding to the target pixel interval are determined according to the color attributes of each target pixel interval, including: determining the first color mean and the first covariance matrix corresponding to the target pixel interval according to the color attributes of the pixels in each target pixel interval of the image frame in the original multimedia data. Determining the second color mean and the second covariance matrix corresponding to the target pixel interval according to the color attributes of the pixels in each target pixel interval of the reference image; determining the transformation matrix corresponding to the target pixel interval according to the first color mean, the second color mean, the first covariance matrix and the second covariance matrix, wherein the color features include the first color mean and the second color mean.

Taking the shadow interval as an example, for all pixels whose brightness is in the shadow interval, the UV channel mean of all shadow pixels in the image frame in the original multimedia data is calculated. Covariance matrix Σ _src . The first color mean can be expressed as and The first covariance matrix can be expressed as Σ _src . Calculate the UV channel mean of all shadow pixels in the reference image The covariance matrix Σ _ref . The second color mean can be expressed as and The second covariance matrix can be expressed as Σ _ref . The transformation matrix M is determined based on the above mean and covariance matrix using the MKL algorithm.

In some embodiments, after determining the first color mean and the first covariance matrix corresponding to the target pixel interval, the method further includes: determining an eigenvalue of the first covariance matrix. The eigenvalue represents the shape of an ellipse presented by the pixels of the image frame in the original multimedia data according to the Gaussian distribution. When the eigenvalue satisfies the set conditions, the unit matrix is used as the transformation matrix.

The preset condition represents the shape threshold of the ellipse presented by the Gaussian distribution of the pixels of the image frame in the original multimedia data. If the Gaussian distribution exceeding the shape threshold is subjected to color transformation, color abnormal points may be generated. In this case, the transformation matrix is calculated based on the covariance matrix, and the transformation matrix M is set to the unit matrix instead.

For example, the eigenvalues e1 and e2 of the covariance matrix are determined according to the covariance matrix _Σsrc of the image frame in the original multimedia data, and e1>e2. The eigenvalues of the covariance matrix represent the major axis e1 and minor axis e2 of the ellipse presented by the pixels of the image frame in the original multimedia data according to the Gaussian distribution. If e1/e2> the set shape threshold, it means that the ellipse presented by the Gaussian distribution is very flat. At this time, if the Gaussian distribution is subjected to color transformation, color outliers are likely to be generated. Therefore, the transformation matrix M is set to the unit matrix.

By adopting the above method, the first color mean values of the image frames in the original multimedia data in the shadow interval, the mid-tone interval and the highlight interval can be calculated respectively, which are: and and and The covariance matrices of the image frames in the original multimedia data in the shadow interval, the midtone interval and the highlight interval are calculated respectively, namely Σ _src0 , Σ _src1 , Σ _src2 . The transformation matrices M ₀ , M ₁ , and M ₂ of the image frames in the original multimedia data in the shadow interval, the midtone interval and the highlight interval are calculated respectively.

Color matching is performed on each of the target pixel intervals corresponding to the image frame in the original multimedia data according to the color features and the transformation matrix.

Exemplarily, for each target pixel interval of the image frame in the original multimedia data, the second weight corresponding to the pixel is determined according to color information of the pixel in the target pixel interval and the first color mean value.

The second weight represents the degree of deviation of the color of the pixel from the mean in the Gaussian distribution. The smaller the second weight, the more the color value of the pixel deviates from the mean of the Gaussian distribution. The larger the second weight, the closer the color value of the pixel is to the mean of the Gaussian distribution.

Take the shadow interval as an example. The color information uv of the pixels in the shadow interval of the image frame in the original multimedia data in the UV channel can be expressed as a two-dimensional vector, The first color mean value of the shadow interval of the image frame in the original multimedia data It can be expressed as According to the pixel color information uv, the first color mean The second weight is calculated by using the first covariance matrix Σ _src0 . In a similar manner, the second weights corresponding to the pixels in the midtone interval and the highlight interval are calculated, which will not be repeated here.

Color matching is performed on the target pixel interval according to the color feature and the transformation matrix, respectively, to obtain a first color matching result and a second color matching result corresponding to the target pixel interval. For example, color matching can be performed on the target pixel interval of the image frame in the original multimedia data based on the transformation matrix to obtain a first color matching result. The first color matching result can be new color information obtained after color transformation is performed on the target pixel interval of the image frame in the original multimedia data based on the transformation matrix. The second color matching result is determined based on the color of the pixel in the image frame in the original multimedia data, the first color mean corresponding to the target pixel interval to which the pixel belongs, and the second color mean corresponding to the target pixel interval in the reference image. The second color matching result can be new color information obtained by adjusting the color information of the target pixel interval based on the color mean of the image frame in the original multimedia data and the reference image in the target pixel interval.

Take the shadow interval as an example. Perform a color transformation on each pixel in the shadow interval of the image frame in the original multimedia data based on the transformation matrix to obtain new color information as the first color matching result. Adjust the color information of each pixel in the shadow interval of the image frame in the original multimedia data based on the color mean of the image frame in the original multimedia data and the reference image in the shadow interval to obtain new color information as the second color matching result. For example, the color information uv of the pixel of the image frame in the original multimedia data is combined with the transformation matrix M ₀ corresponding to the shadow interval to obtain the first color matching result corresponding to the shadow interval, thereby realizing the color style transfer of the reference image to the image frame in the original multimedia data. The color information uv of the pixel of the image frame in the original multimedia data minus the color information uv of the pixel of the image frame in the original multimedia data Back Fold With reference image A second color matching result corresponding to the shadow interval is obtained. The second color mean of the shadow interval of the reference image can be expressed as

The first color matching result and the second color matching result are fused according to the second weight to obtain a color matching result of the target pixel interval.

Take the shadow interval as an example. The fusion coefficient is determined according to the second weight, and the first color matching result and the second color matching result corresponding to the shadow interval are fused according to the fusion coefficient to obtain the color matching result of the shadow interval. The fusion coefficient includes the second weight, and the sum of the fusion coefficients is 1. The color matching results of the midtone interval and the color matching results of the highlight interval are calculated in a similar manner, which will not be repeated here.

It should be noted that, for pixels in the image frame in the original multimedia data whose brightness completely belongs to the shadow interval, the midtone interval or the highlight interval, there is only a color matching result for the shadow interval. For pixels in the image frame in the original multimedia data whose brightness belongs to the overlapping area of the shadow interval and the midtone interval, there are a color matching result for the shadow interval and a color matching result for the midtone interval. For pixels in the image frame in the original multimedia data whose brightness belongs to the overlapping area of the midtone interval and the highlight interval, there are a color matching result for the midtone interval and a color matching result for the highlight interval.

For each pixel of the image frame in the original multimedia data, the color matching result of the target pixel interval is fused according to the first weight corresponding to the pixel and each target pixel interval to obtain the color matching result of the image frame in the original multimedia data; and the processing result image of the image frame in the original multimedia data is determined according to the color matching result and brightness matching result of the image frame in the original multimedia data as the multimedia data processing result corresponding to the image frame.

In some embodiments, for each pixel of the image frame in the original multimedia data, firstly, the first weights (i.e., shadow weight, midtone weight, and highlight weight) corresponding to the pixel and the respective target pixel intervals (i.e., shadow interval, midtone interval, and highlight interval) are determined. According to the first weights, the color matching results of the pixel in the corresponding target pixel intervals (i.e., shadow weight, midtone weight, and highlight weight) are fused to obtain the color matching result of the image frame in the original multimedia data.

For example, for each pixel of the image frame in the original multimedia data, according to the shadow weight, midtone weight and highlight weight of the pixel, the color matching result of the pixel in the shadow interval, the color matching result of the pixel in the midtone interval and the color matching result of the pixel in the highlight interval are fused. Thus, the color matching result of the image frame in the original multimedia data is obtained.

In some embodiments, for pixels in a skin color area in an image frame of the original multimedia data, the display attributes of the pixels are adjusted according to the display attributes of the skin color area of the reference image to obtain a skin color adjustment result corresponding to the image frame. This includes: determining the probability that each pixel in the image frame of the original multimedia data belongs to a skin color area as a skin color weight, and determining the pixels included in the skin color area in the image frame of the original multimedia data according to the skin color weight. Determining the original multimedia data according to the display attributes of the pixels included in the skin color area in the image frame of the original multimedia data For pixels in the skin color area in the image frame of the original multimedia data, the display attributes of the pixels are adjusted according to the mean values of the display attributes of the image frame and the reference image in the skin color area, respectively, to obtain a skin color adjustment result corresponding to the image frame.

The skin color adjustment result represents the processing result of migrating the color and brightness characteristics of the reference image to the skin color area in the image frame of the original multimedia data.

Exemplarily, the original multimedia data is input into the skin color segmentation model. The skin color segmentation model is used to perform skin color segmentation on the image frames in the original multimedia data, and the probability that each pixel in the image frame belongs to the skin color area is obtained. The pixels contained in the skin color area of the image frame in the original multimedia data are determined according to the probability. Similarly, the reference image is input into the skin color segmentation model. The skin color segmentation model is used to perform skin color segmentation on the reference image, and the probability that each pixel in the reference image belongs to the skin color area is obtained. The pixels contained in the skin color area of the reference image are determined according to the probability.

According to the brightness of each pixel contained in the skin color area of the image frame in the original multimedia data, the brightness mean of the skin color area of the image frame in the original multimedia data is determined. According to the brightness of each pixel contained in the skin color area of the reference image, the brightness mean of the skin color area of the reference image is determined. According to the chromaticity of each pixel contained in the skin color area of the image frame in the original multimedia data, the chromaticity mean of the skin color area of the image frame in the original multimedia data is determined. According to the chromaticity of each pixel contained in the skin color area of the reference image, the chromaticity mean of the skin color area of the reference image is determined.

For each pixel in the skin color area in the image frame of the original multimedia data, the difference between the brightness of the pixel and the average brightness of the skin color area is calculated, and the difference is superimposed with the average brightness of the skin color area of the reference image to obtain a brightness adjustment result of the pixel.

For each pixel in the skin color area of the image frame of the original multimedia data, the difference between the chromaticity of the pixel and the chromaticity mean of the skin color area is calculated, and the difference is superimposed with the chromaticity mean of the skin color area of the reference image to obtain the chromaticity adjustment result of the pixel.

According to the brightness adjustment result and the chromaticity adjustment result of each pixel in the skin color area in the image frame of the original multimedia data, a skin color adjustment result corresponding to the image frame is obtained.

The target multimedia data is obtained by fusing the multimedia data processing result and the skin color adjustment result of the image frame in a linear fusion manner based on the skin color weight. For example, the target multimedia data is determined based on the fusion result of the multimedia data processing result and the skin color adjustment result corresponding to the image frame sequence included in the target multimedia data.

The technical solution of the disclosed embodiment adjusts the image frame in the original multimedia data by the display attribute of the reference image, obtains the target multimedia data and displays it. Thus, the display style of the original multimedia data is adjusted based on the color style of the reference image, so that the target multimedia data presents a display effect similar to that of the reference image, omitting the step of manually adjusting the display attribute, and improving the color adjustment efficiency. Since the reference image can intuitively present the color adjustment result desired by the user, adjusting the original multimedia data based on the display attribute of the reference image can improve the adjustment accuracy and avoid repeated modification of parameters.

FIG. 2 is a flow chart of another method for processing multimedia data provided by an embodiment of the present disclosure. Embodiment Based on the above embodiment, additional display steps of reference images and special effect elements are added. As shown in FIG2 , the method includes: S210, obtaining original multimedia data. S220, obtaining a reference image. S230, generating special effect elements according to the reference image, and superimposing and displaying the reference image and special effect elements on the original multimedia data.

If only the original multimedia data is displayed during the process of adjusting the display properties of the image frame in the original multimedia data based on the reference image, the display effect is relatively simple and the user experience is poor. It may even cause the user to mistakenly believe that the reference image has not been successfully uploaded and repeat the operation, resulting in erroneous processing. By superimposing the reference image and special effect elements on the original multimedia data, the single display effect and the erroneous operation of repeatedly uploading the reference image can be avoided.

The special effect element is superimposed on the set position of the reference image and is displayed and moves in a set direction. For example, the special effect element includes a color block special effect, etc. The color block special effect may include color blocks of different colors. The color of the color block represents the color category obtained by clustering the colors of the pixels of the reference image. Alternatively, the color of the color block may also be a fixed color, and the embodiment of the present disclosure does not specifically limit the shape and color of the color block.

Exemplarily, the color category included in the reference image is determined according to the color attribute of the pixels in the reference image, and a special effect element corresponding to the color represented by the color category is generated.

In some embodiments, multiple cluster centers are pre-set, and pixels in the reference image are clustered according to the colors of the pixels and the colors of the pixels corresponding to the cluster centers to obtain multiple pixel groups. The colors corresponding to the cluster centers of all groups of pixels can be used as the color categories included in the reference image.

Exemplarily, the reference image is superimposed and displayed in the image frame of the original multimedia data, and an effect of sliding into the display screen of the original multimedia data along a set direction is presented. FIG3 is a schematic diagram of superimposing and displaying a reference image in an image frame of original multimedia data provided by an embodiment of the present disclosure. As shown in FIG3, during the playback of the original multimedia data, the reference image is superimposed and displayed at a first position 310 of the original multimedia data at time _t1 , and slides to a second position 320 at time _t2 .

Line material is superimposed and displayed in the reference image, and the line material moves along a first direction in the reference image. The movement of the line material in the reference image along the first direction can present an effect of light scanning the reference image. By superimposing a dynamic image of the line material moving along the first direction onto the reference image displayed in the original multimedia data, an effect of light scanning the reference image is presented. FIG4 is a schematic diagram of a reference image superimposed with line material provided by an embodiment of the present disclosure. As shown in FIG4 , a mask image 410 with set transparency is generated according to the width and height of the reference image, and there are bright areas 411 and dark areas 412 in the mask image 410. A dynamic image is synthesized based on multiple mask images 410 with different positions of bright areas 411. The dynamic image is superimposed on the reference image to obtain the effect of light scanning the reference image.

The special effect element is superimposed on the first area of the reference image according to the set superposition form. The set superposition form represents the form of superimposing the color block special effect on the reference image. For example, the set superposition form can be to display the color blocks in different positions of the reference image in sequence according to the color of the color blocks. The first area can be a preset area within the reference image. FIG5 is a schematic diagram of a color block special effect provided by an embodiment of the present disclosure. As shown in FIG5, each color block 520 is displayed one by one in the reference image 510 until all the color blocks 520 are displayed in the first area of the reference image 510. During the display process, the width and height of the color block are changed from a first state to a second state.

Exemplarily, the display position and brightness arrangement order corresponding to the special effect element are determined according to the set superposition form. The display order of the special effect element is determined according to the brightness arrangement order. According to the display position and display order, the special effect element is superimposed and displayed in the first area of the reference image in the image frame in the original multimedia data. The display position represents the position of the special effect element in the reference image. For example, the display position may include color blocks arranged horizontally (or vertically or obliquely) in the reference image, and the spacing between adjacent color blocks is the same (or gradually increases or gradually decreases). The brightness arrangement order may include color or brightness arranged from dark to light, or from light to dark, etc. Since each color block represents the color appearing in the reference image, and the color blocks have different color depths and different brightness. The display order of the color blocks in the reference image can be determined according to the brightness sorting order specified in the set superposition form. Then, the color blocks are displayed one by one at the display position corresponding to the color blocks according to the display order. When each color block is displayed, the first state of the color block is first displayed in the reference image, and the process of the color block converting from the first state to the second state is presented. For example, a small color block is first displayed in the reference image, and then the small color block is enlarged to obtain a color block with a set width and height.

Move the special effect element from the first area of the reference image along the second direction to the second area corresponding to the reference image, and adjust the display state of the special effect element in the second area. The second area represents the area range outside the reference image. Figure 6 is a schematic diagram of a color block shift provided by an embodiment of the present disclosure. As shown in Figure 6, after all color blocks are displayed in the first area of the reference image, all color blocks 610 are rotated from a horizontal arrangement to a vertical arrangement, and moved to the second area 620 corresponding to the reference image. After all color blocks 610 reach the second area 620, the spacing between adjacent color blocks is increased. S240. Adjust the image frame in the original multimedia data according to the display attributes of the reference image to obtain target multimedia data. S250. When the display state of the special effect element in the original multimedia data meets the set conditions, display the target multimedia data.

The set condition may represent a condition under which the special effect element disappears from the original multimedia data, and when the special effect element disappears from the original multimedia data, the target multimedia data is displayed. For example, when the spacing between adjacent color blocks is increased so that the spacing between adjacent color blocks is a set value, it is determined that the special effect color meets the set condition. Alternatively, when the spacing between adjacent color blocks is increased and the display time of the color blocks reaches a set time, it is determined that the special effect color meets the set condition.

The technical solution of the disclosed embodiment can intuitively display the color style of the reference image selected by the user by generating special effect elements corresponding to the reference image and superimposing the reference image and the special effect elements in the original multimedia data. It can also avoid erroneous operations such as a single display effect or repeated uploading of reference images in the process of adjusting the display properties of the original multimedia data based on the reference image by vividly displaying the special effect elements, thereby increasing interactivity and fun and improving user experience.

7 is a schematic diagram of the structure of a multimedia data processing device provided in an embodiment of the present disclosure. The device may be implemented in the form of software and/or hardware. Optionally, it may be implemented by an electronic device, which may be a mobile terminal, a PC, a server, etc.

As shown in FIG7 , the device includes: an image acquisition module 710 and an image adjustment module 720. Block 710 is used to obtain original multimedia data; image acquisition module 720 is used to obtain a reference image; image adjustment module 730 is used to adjust the image frames in the original multimedia data according to the display properties of the reference image to obtain target multimedia data and display the target multimedia data. The target multimedia data includes a sequence of image frames in the original multimedia data adjusted based on the display properties of the reference image.

Optionally, the method further includes: a special effect element generation module, which is used to generate a special effect element according to the reference image after acquiring the reference image, and to superimpose and display the reference image and the special effect element in the original multimedia data. The special effect element is superimposed on the reference image and displayed at a set position and moves in a set direction; when the display state of the special effect element in the original multimedia data meets the set condition, the step of displaying the target multimedia data is executed.

Furthermore, generating a special effect element according to the reference image includes: determining a color category included in the reference image according to color attributes of pixels in the reference image; and generating a special effect element corresponding to a color represented by the color category.

Furthermore, the superimposing and displaying the reference image and special effect element in the original multimedia data includes: superimposing and displaying the reference image in the image frame of the original multimedia data; superimposing and displaying line material in the reference image, wherein the line material moves along a first direction in the reference image; superimposing the special effect element to a first area in the reference image according to a set superimposition form; moving the special effect element from the first area of the reference image along a second direction to a second area corresponding to the reference image, and adjusting the display status of the special effect element in the second area.

Furthermore, superimposing the special effect element to the first area in the reference image according to the set superimposition form includes: determining a display position and a brightness arrangement order corresponding to the special effect element according to the set superimposition form; determining a display order of the special effect elements according to the brightness arrangement order; and superimposing and displaying the special effect element in the first area of the reference image in the image frame of the original multimedia data according to the display position and display order.

Optionally, the image adjustment module 720 is specifically used to: for the reference image and the image frames in the original multimedia data, determine the target pixel intervals corresponding to the images according to the display attributes respectively; for each of the target pixel intervals, perform attribute matching on the image frames in the original multimedia data based on the display attributes of the pixels of the reference image, fuse the attribute matching results corresponding to each of the target pixel intervals, and obtain the multimedia data processing results corresponding to the image frames; for the pixels in the skin color area of the image frames of the original multimedia data, adjust the display attributes of the pixels according to the display attributes of the skin color area of the reference image, and obtain the skin color adjustment results corresponding to the image frames; for the image frames in the original multimedia data, fuse the skin color adjustment results and the multimedia data processing results according to the skin color weight, and obtain the target multimedia data. The skin color weight represents the probability that each pixel in the image frame of the original multimedia data belongs to the skin color area.

Further, for a pixel in a skin color area in an image frame of the original multimedia data, adjusting the display attribute of the pixel according to the display attribute of the skin color area of the reference image to obtain a skin color adjustment result corresponding to the image frame includes: determining a probability that each pixel in the image frame of the original multimedia data belongs to the skin color area, as the skin color The method comprises the steps of: determining pixels included in a skin color area in an image frame of the original multimedia data according to the skin color weight; determining a mean value of display attributes of the skin color area in the image frame of the original multimedia data according to display attributes of the pixels included in the skin color area in the image frame of the original multimedia data; and adjusting the display attributes of the pixels in the skin color area in the image frame of the original multimedia data according to the mean values of display attributes of the image frame and a reference image in the skin color area, so as to obtain a skin color adjustment result corresponding to the image frame.

The multimedia data processing device provided in the embodiments of the present disclosure can execute the multimedia data processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

FIG8 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Referring to FIG8 below, it shows a schematic diagram of the structure of an electronic device (e.g., a terminal device or server in FIG8 ) 800 suitable for implementing an embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG8 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG8 , the electronic device 800 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage device 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 are also stored. The processing device 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An edit/output (I/O) interface 805 is also connected to the bus 804.

Typically, the following devices may be connected to the I/O interface 805: input devices 806 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 807 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 808 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 809. The communication device 809 may allow the electronic device 800 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 8 shows an electronic device 800 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 809, or installed from a storage device 808, or installed from a ROM 802. When the computer program is executed by the processing device 801, the execution The above functions are defined in the method of the embodiment of the present disclosure.

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

The electronic device provided by the embodiment of the present disclosure and the method for processing multimedia data provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, reference can be made to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

The embodiments of the present disclosure provide a computer storage medium on which a computer program is stored. When the program is executed by a processor, the method for processing multimedia data provided by the above embodiments is implemented.

It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains original multimedia data; obtains a reference image; adjusts the image frame in the original multimedia data according to the display attribute of the reference image to obtain target multimedia data, and displays the target multimedia data. The target multimedia data includes an image frame sequence in the original multimedia data that is adjusted based on the display attributes of the reference image.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases.

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

Although the subject matter has been described in language specific to structural features and/or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims (11)

A method for processing multimedia data, comprising: Obtaining original multimedia data; Get a reference image; The image frames in the original multimedia data are adjusted according to the display properties of the reference image to obtain target multimedia data, and the target multimedia data is displayed, wherein the target multimedia data includes a sequence of image frames in the original multimedia data adjusted based on the display properties of the reference image. The method according to claim 1, wherein after acquiring the reference image, it further comprises: Generating a special effect element according to the reference image, and superimposing and displaying the reference image and the special effect element in the original multimedia data, wherein the special effect element is superimposed and displayed at a set position of the reference image and moves in a set direction; When the display state of the special effect element in the original multimedia data meets the set condition, the step of displaying the target multimedia data is executed. The method according to claim 2, wherein generating a special effect element according to the reference image comprises: Determining the color category included in the reference image according to the color attributes of the pixels in the reference image; A special effect element corresponding to the color represented by the color category is generated. The method according to claim 2, wherein the superimposing and displaying the reference image and the special effect element in the original multimedia data comprises: Overlaying and displaying the reference image in an image frame of the original multimedia data; Overlaying and displaying line material in the reference image, wherein the line material moves along a first direction in the reference image; superimposing the special effect element onto the first area in the reference image according to a set superimposition form; The special effect element is moved from the first area of the reference image to the second area corresponding to the reference image along the second direction, and the display state of the special effect element in the second area is adjusted. The method according to claim 4, wherein superimposing the special effect element onto the first area in the reference image according to a set superposition form comprises: Determine the display position and brightness arrangement order corresponding to the special effect element according to the set superposition form; Determining a display order of the special effect elements according to the brightness arrangement order; The special effect element is displayed in a superimposed manner in a first area of a reference image in an image frame of the original multimedia data according to the display position and display order. The method according to claim 1, wherein adjusting the image frame in the original multimedia data according to the display attribute of the reference image to obtain the target multimedia data comprises: For the reference image and the image frame in the original multimedia data, respectively determine the image pair according to the display attribute The corresponding target pixel interval; For each of the target pixel intervals, attribute matching is performed on the image frame in the original multimedia data based on the display attributes of the pixels of the reference image, and the attribute matching results corresponding to each of the target pixel intervals are merged to obtain the multimedia data processing result corresponding to the image frame; For pixels in a skin color area in an image frame of the original multimedia data, adjusting display properties of the pixels according to display properties of the skin color area of the reference image to obtain a skin color adjustment result corresponding to the image frame; For the image frame in the original multimedia data, the skin color adjustment result and the multimedia data processing result are fused according to the skin color weight to obtain the target multimedia data, wherein the skin color weight represents the probability that each pixel in the image frame of the original multimedia data belongs to the skin color area. The method according to claim 6, wherein for pixels in a skin color area in an image frame of the original multimedia data, adjusting display attributes of the pixels according to display attributes of the skin color area of the reference image to obtain a skin color adjustment result corresponding to the image frame comprises: Determine the probability that each pixel in the image frame of the original multimedia data belongs to the skin color area as the skin color weight, and determine the pixels included in the skin color area in the image frame of the original multimedia data according to the skin color weight; Determining a mean value of the display attributes of the skin color area in the image frame of the original multimedia data according to the display attributes of the pixels included in the skin color area in the image frame of the original multimedia data; For pixels in a skin color area in an image frame of the original multimedia data, display properties of the pixels are adjusted according to mean values of display properties of the image frame and a reference image in the skin color area, to obtain a skin color adjustment result corresponding to the image frame. A multimedia data processing device, comprising: A data acquisition module, used for acquiring original multimedia data; An image acquisition module, used for acquiring a reference image; An image adjustment module is used to adjust the image frames in the original multimedia data according to the display properties of the reference image, obtain target multimedia data, and display the target multimedia data, wherein the target multimedia data includes a sequence of image frames in the original multimedia data that are adjusted based on the display properties of the reference image. An electronic device, comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the multimedia data processing method as described in any one of claims 1 to 7. A storage medium containing computer executable instructions, wherein the computer executable instructions are used to execute the multimedia data processing method as claimed in any one of claims 1 to 7 when executed by a computer processor. A computer program product tangibly stored in a computer storage medium and The device also comprises computer executable instructions, which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 7.