WO2022199102A1 - Image processing method and device - Google Patents

Abstract

The present disclosure relates to the technical field of image processing, and relates to an image processing method and device. The method comprises: in response to a special effect display instruction, acquiring a movement trajectory input by a user in a display page comprising a face image; according to first position information of at least two face key points in the face image displayed at the current moment, and trajectory position information of at least one trajectory point in the movement trajectory, determining the relative position of each trajectory point relative to the face image; and repeatedly executing the image special effect processing, the image special effect processing comprising: converting the relative position according to second position information of the face key points in the face image displayed in the display page after the current moment to obtain a first absolute position of each trajectory point on a display screen; and connecting the trajectory points located at the first absolute positions, and generating and displaying a first special effect line.

Description

Image processing method and device

cross reference

This application is based on the Chinese patent application with the application number of 202110328694.1 and the filing date of March 26, 2021, and claims the priority of the Chinese patent application. The entire content of the Chinese patent application is incorporated herein by reference.

technical field

The present disclosure relates to the technical field of image processing, and in particular, to an image processing method, an apparatus, an electronic device, and a storage medium.

Background technique

With the development of intelligent terminals, in image generation scenarios such as photography, video shooting, and webcasting, image special effects processing on face images has become a mainstream image processing technology.

The current image special effect processing process may include: the user clicks on a set special effect template (eg, a set animal image template, a set decoration template, etc.). After receiving the click input for setting the special effect template, the terminal may perform special effect fusion on the selected setting special effect template and the user's face image, and display the fused special effect image. Afterwards, the terminal may receive the movement track input by the user on the special effect image, and draw a line pattern indicating the movement track on a fixed position of the movement track.

SUMMARY OF THE INVENTION

The present disclosure provides an image processing method, apparatus, electronic device and storage medium.

According to a first aspect of the embodiments of the present disclosure, there is provided an image processing method, the method comprising:

In response to the special effect display instruction, acquiring the movement track input by the user in the display page including the face image;

According to the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the movement trajectory, determine the relative position of each trajectory point relative to all the trajectory points. relative position of the face image;

The image special effect processing is repeatedly performed, and the image special effect processing includes:

Convert the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current time, to obtain the first position of each track point on the display screen. absolute position;

Connect the trajectory points located at each of the first absolute positions to generate a first special effect line;

In the display page, the first special effect line is displayed.

In a possible implementation manner, the at least two face key points include: a first face key point, a second face key point and a target face key point, the first face key point and the first face key point The two face key points are symmetrical with respect to the target face key point, and the target face key point is any face key point on the symmetry axis of the face image.

In a possible implementation manner, in the face image displayed at the current moment according to the display page, the first position information of at least two face key points, and the track position of at least one track point in the movement track information to determine the relative position of each trajectory point relative to the face image, including:

For each of the track points, according to the first position information of the target face key point and the track position information of the track point, determine the translation vector of the track point pointing to the target face key point;

According to the first position information of the first face key point and the second face key point, determine the first rotation matrix and the first scaling matrix of the current face posture in the face image;

According to the first rotation matrix, the first scaling matrix and the translation vector, the relative position of the trajectory point relative to the face image is obtained.

In a possible implementation manner, both the first position information and the track position information include absolute coordinates on the display screen,

Determining the first rotation matrix and the first scaling matrix of the current face in the face image according to the first position information of the first face key point and the second face key point, including:

According to the first position information of the first face key point and the second face key point, and the first length, the first rotation matrix is obtained, and the first length is the first face key point to the length of the first vector of the second face key point;

According to the reference length of the connecting line between the first face key point and the second face key point, and the first length, the first scaling matrix is obtained, and the reference length is for the face image The first length set by the face in the facing posture.

In a possible implementation manner, obtaining the relative position of the trajectory point relative to the face image according to the first rotation matrix, the first scaling matrix and the translation vector includes:

The relative position is obtained according to the first scaling matrix, the first rotation matrix, the translation vector and a first formula, where the first formula includes:

表示所述平移向量。 Wherein, Q represents the relative position, Ms ₁ represents the first scaling matrix, Mr ₁ represents the first rotation matrix,

represents the translation vector.

In a possible implementation manner, the relative position is converted according to the second position information of the face key points in the face image displayed on the display page after the current time, to obtain each The first absolute position of the trajectory point on the display screen, including:

For each of the trajectory points, according to the second position information of the first face key point and the second face key point, determine the second rotation matrix and the second rotation matrix of the current face pose in the face image scaling matrix;

The first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, and the relative position.

In a possible implementation manner, the second position information includes absolute coordinates on the display screen, and the determination is determined according to the second position information of the first face key point and the second face key point The second rotation matrix and the second scaling matrix of the current face posture in the face image, including:

According to the second position information of the first face key point and the second face key point, and the second length, the second rotation matrix is obtained, and the second length is the first face key point point to the length of the second vector of the second face key point;

According to the reference length of the line connecting the first face key point and the second face key point, and the second length, the second scaling matrix is obtained, and the reference length is for the face image The second length set by the face in the facing posture.

In a possible implementation manner, obtaining the first position of the trajectory point according to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, and the relative position an absolute position, including:

According to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, the relative position and the second formula, the first absolute position of the trajectory point is obtained, and the The second formula includes:

R=Mr ₂ ·Ms ₂ ·(x _q , y _q ) ^T +(x _c , y _c ) ^T ;

Wherein, R represents the first absolute position, Mr ₂ represents the second rotation matrix, Ms ₂ represents the second scaling matrix, (x _q , y _q ) represents the relative position, (x _c , y _{c )} ) represents the second position information of the target face key point, and T represents the transposition process.

In a possible implementation manner, the image special effect processing further includes:

generating a second special effect line symmetrical to the first special effect line according to the first special effect line, where the second special effect line and the first special effect line are left-right symmetrical with respect to the face in the face image;

The second special effect line is displayed in the display page.

In a possible implementation, the method further includes:

According to the relative position of each track point relative to the face image, determine the relative position of the symmetry point of each track point relative to the face image, and the symmetry point and the track point take the face as the The benchmark is left and right symmetrical;

The generating a second special effect line symmetrical with the first special effect line according to the first special effect line, comprising:

Determine the second absolute position of each of the symmetrical points on the display screen according to the second position information of the face key points and the relative positions of each of the symmetrical points;

Connecting the symmetrical points located at each of the second absolute positions to generate the second special effect line.

In a possible implementation manner, the relative position includes relative coordinates relative to the face image, and the symmetry point of each trajectory point is determined according to the relative position of each trajectory point relative to the face image The relative position relative to the face image, including:

Perform positive and negative conversion processing on the coordinate value of the first direction in the relative coordinates of the trajectory point to obtain the processed coordinate value, and the first direction is perpendicular to the symmetry axis of the face image;

updating the relative position of the trajectory point, so that the coordinate value of the first direction in the updated relative position is the processed coordinate value;

The updated relative position is determined as the relative position of the symmetry point.

According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, the apparatus comprising:

an acquisition module, used for acquiring the movement track input by the user in the display page including the face image in response to the special effect display instruction;

The determining module is configured to determine the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the moving trajectory. the relative position of the trajectory point relative to the face image;

An image special effect processing module, configured to repeatedly perform image special effect processing, wherein the image special effect processing includes:

Convert the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current time, to obtain the first position of each track point on the display screen. absolute position;

Connect the trajectory points located at each of the first absolute positions to generate a first special effect line;

In the display page, the first special effect line is displayed.

In a possible implementation manner, the at least two face key points include: a first face key point, a second face key point and a target face key point, the first face key point and the first face key point The two face key points are symmetrical with respect to the target face key point, and the target face key point is any face key point on the symmetry axis of the face image.

In a possible implementation manner, the determining module is further configured to:

For each of the track points, according to the first position information of the target face key point and the track position information of the track point, determine the translation vector of the track point pointing to the target face key point;

According to the first position information of the first face key point and the second face key point, determine the first rotation matrix and the first scaling matrix of the current face posture in the face image;

According to the first rotation matrix, the first scaling matrix and the translation vector, the relative position of the trajectory point relative to the face image is obtained.

In a possible implementation manner, both the first position information and the track position information include absolute coordinates on the display screen, and the determining module is further configured to:

According to the first position information of the first face key point and the second face key point, and the first length, the first rotation matrix is obtained, and the first length is the first face key point to the length of the first vector of the second face key point;

According to the reference length of the connecting line between the first face key point and the second face key point, and the first length, the first scaling matrix is obtained, and the reference length is for the face image The first length set by the face in the facing posture.

In a possible implementation manner, the determining module is further configured to:

The relative position is obtained according to the first scaling matrix, the first rotation matrix, the translation vector and a first formula, where the first formula includes:

表示所述平移向量。 Wherein, Q represents the relative position, Ms ₁ represents the first scaling matrix, Mr ₁ represents the first rotation matrix,

represents the translation vector.

In a possible implementation manner, the image special effect processing module is further configured to:

For each of the trajectory points, according to the second position information of the first face key point and the second face key point, determine the second rotation matrix and the second rotation matrix of the current face pose in the face image scaling matrix;

The first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, and the relative position.

In a possible implementation manner, the second position information includes absolute coordinates on the display screen, and the image special effect processing module is further configured to:

According to the second position information of the first face key point and the second face key point, and the second length, the second rotation matrix is obtained, and the second length is the first face key point point to the length of the second vector of the second face key point;

According to the reference length of the line connecting the first face key point and the second face key point, and the second length, the second scaling matrix is obtained, and the reference length is for the face image The second length set by the face in the facing posture.

In a possible implementation manner, the image special effect processing module is further configured to:

According to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, the relative position and the second formula, the first absolute position of the trajectory point is obtained, and the The second formula includes:

R=Mr ₂ ·Ms ₂ ·(x _q , y _q ) ^T +(x _c , y _c ) ^T ;

Wherein, R represents the first absolute position, Mr ₂ represents the second rotation matrix, Ms ₂ represents the second scaling matrix, (x _q , y _q ) represents the relative position, (x _c , y _{c )} ) represents the second position information of the target face key point, and T represents the transposition process.

In a possible implementation manner, the image special effect processing further includes:

generating a second special effect line symmetrical to the first special effect line according to the first special effect line, where the second special effect line and the first special effect line are left-right symmetrical with respect to the face in the face image;

The second special effect line is displayed in the display page.

In a possible implementation manner, the determining module is further configured to:

According to the relative position of each track point relative to the face image, determine the relative position of the symmetry point of each track point relative to the face image, and the symmetry point and the track point take the face as the The benchmark is left and right symmetrical;

The image special effect processing module is also used for:

Determine the second absolute position of each of the symmetrical points on the display screen according to the second position information of the face key points and the relative positions of each of the symmetrical points;

Connecting the symmetrical points located at each of the second absolute positions to generate the second special effect line.

In a possible implementation manner, the relative position includes relative coordinates relative to the face image, and the determining module is further configured to:

Perform positive and negative conversion processing on the coordinate value of the first direction in the relative coordinates of the trajectory point to obtain the processed coordinate value, and the first direction is perpendicular to the symmetry axis of the face image;

updating the relative position of the trajectory point, so that the coordinate value of the first direction in the updated relative position is the processed coordinate value;

The updated relative position is determined as the relative position of the symmetry point.

According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, including:

one or more processors;

one or more memories for storing the one or more processor-executable instructions;

Wherein, the one or more processors are configured to execute the image processing method described in the first aspect or any possible implementation manner of the first aspect.

According to a fourth aspect of the embodiments of the present disclosure, there is provided a non-volatile computer-readable storage medium, which, when the instructions in the non-volatile computer-readable storage medium are executed by a processor of an electronic device, causes all the The electronic device can execute the image processing method described in the first aspect or any possible implementation manner of the first aspect.

According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, realizes the image described in the first aspect or any possible implementation manner of the first aspect Approach.

In the embodiment of the present disclosure, the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory of at least one trajectory point in the movement trajectory input by the user on the display page can be obtained. Position information, to determine the relative position of each track point relative to the face image. In order to repeat the conversion of the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current moment, the first absolute position of each track point on the display screen is obtained, and the first absolute position of each track point on the display screen is obtained. , the process of displaying the first special effect line formed by connecting the track points located at the first absolute positions. In the above technical solution, the first special effect line is drawn according to the movement trajectory input by the user, which realizes the function of the user to draw the special effect independently. Moreover, after obtaining the relative positions of each track point in the moving track and the current face image, the position information of the key points of the face and the relative position of each track point in the face image displayed in real time on the display page can be used to determine each track. point at the first absolute position on the display screen, so that the first special effect line is generated and displayed after connecting the track points located at the first absolute positions. In this way, the display position of the generated first special effect line will change with the change of the display position of the face image displayed in real time on the display page, realizing the special effect that the first special effect line moves with the face, and enriching the special effect display effect.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

Fig. 1 is a flowchart of an image processing method according to an exemplary embodiment.

FIG. 2 is a schematic diagram of a human face image according to an exemplary embodiment.

Fig. 3 is a flow chart of a method for determining relative positions of track points according to an exemplary embodiment.

Fig. 4 is a schematic diagram of a display page of a face image according to an exemplary embodiment.

Fig. 5 is a schematic diagram of a display page of a face image according to an exemplary embodiment.

Fig. 6 is a flow chart of a method for determining the first absolute position of a track point according to an exemplary embodiment.

Fig. 7 is a flowchart of a method for generating a second special effect line according to an exemplary embodiment.

Fig. 8 is a flowchart of another method for generating a second special effect line according to an exemplary embodiment.

Fig. 9 is a block diagram of an image processing apparatus according to an exemplary embodiment.

Fig. 10 is a block diagram of an electronic device according to an exemplary embodiment.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

Fig. 1 is a flowchart of an image processing method according to an exemplary embodiment. The image processing method can be applied to electronic equipment. The electronic device may be a terminal with a display screen, and the terminal may be installed with an application program for performing image special effect processing on the face image. The embodiments of the present application are described by taking an electronic device as a terminal as an example. As shown in FIG. 1, the image processing method may include the following steps 101-103:

In step 101, in response to the special effect display instruction, the movement track input by the user in the display page including the face image is acquired.

In the embodiment of the present disclosure, the user can perform image special effects processing on the face image when he wants to use the terminal to take photos, video captures, or live webcasts and other processes of capturing faces. The face image may include not only the face but also the background. For example, the background may be a building or a landscape or the like. In some embodiments, the user may operate the terminal to open an application program with an image special effect processing function, and display a display page including a face image in the application program on the terminal. After receiving the special effect display instruction, the terminal may acquire the movement track input by the user in the display page including the face image in response to the special effect display instruction.

The special effect display instruction may be triggered after the terminal receives and executes the setting operation on the display page. For example, the special effect display instruction may be triggered after the user performs a setting operation on the self-drawn control. The setting operation may include input in the form of click, long press, swipe, or voice for the self-drawn control. The display page including the face image may be a shooting interface, a live broadcast interface, or a short (long) video shooting interface, and the like.

In some embodiments, the trajectory of movement of the user input may be the trajectory of the user moving the input. The input member may be a user's finger or a stylus or the like. The movement track may include at least one track point arranged in a movement order. The at least one trajectory point refers to one or more trajectory points. In some embodiments, acquiring the movement trajectory input by the user by the terminal may refer to: acquiring the trajectory position information of at least one trajectory point input by the user by the terminal. The track position information of the track point refers to the absolute position of the track point on the display screen of the terminal. For example, the position information of the track point may be the absolute coordinates of the track point, where the absolute coordinates refer to the position coordinates relative to the specific point on the display screen with a specific point (eg, center point) of the display screen as the origin.

For example, the user wants to add a rabbit ear special effect to his face during the webcast. The user can operate the terminal to open the webcast application, and display a display page including the user's face image on the terminal. The user performs a click operation on the self-drawn icon on the display page, and can use a finger to slide and draw a line in the shape of a left rabbit ear at the upper left position of the face of the face image displayed on the display page. After receiving the click operation on the self-drawn icon, the terminal can generate a special effect display instruction, and respond to the special effect display instruction. Obtain the finger movement trajectory corresponding to the drawn left rabbit ear shape line input by the user. Through the subsequent steps, the human face in the human face image included in the display page can have a rabbit ear special effect.

In step 102, according to the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the moving trajectory, determine the relative face of each trajectory point The relative position of the image.

In the embodiment of the present disclosure, the terminal may acquire the first position information of at least two face key points in the face image displayed at the current moment, and the track position information of each track point in the movement track. The relative position of each track point relative to the face image is determined according to the first position information of the at least two face key points and the track position information of at least one track point. In some embodiments, the relative position of the track point relative to the face image may be represented by a vector that points to the track point from the target face key point. Alternatively, the relative coordinate representation of the track point relative to the face image can also be used. The embodiments of the present disclosure use the relative coordinates of the track points to the face image to represent the relative positions of the track points to the face image.

In some embodiments, the terminal may obtain at least two face key points in the face image by performing face key point detection processing on the face image displayed on the display page. For example, the terminal may use an artificial intelligence (Artificial Intelligence, AI) technology to implement face key point detection processing on a face image.

In some embodiments, the at least two face key points may include: a first face key point, a second face key point, and a target face key point. The target face key point can be any face key point on the symmetry axis of the face image. The first face key point and the second face key point may be symmetrical according to the target face key point. For example, the target face key point is the anchor point of the line connecting the first face key point and the second face key point. The connection line between the first face key point and the second face key point can follow the movement of the target face key point. In this way, since the first face key point and the second face key point are two face key points symmetrical about the target key point, and the target key point is the key point on the symmetry axis of the face image, the first face key point is The inclination angle of the line connecting the point and the second face key point can better reflect the rotation angle of the face in the face image displayed on the display page. And the first position information of the midpoint of the connection line located on the symmetry axis of the face image can reflect the position information of the face image, so that the position and posture information of the current face in the face image are considered, and each The relative position of the track point or the first absolute position has higher accuracy.

In the embodiment of the present disclosure, the first position information of the face key point may be absolute position information of the face key point on the display screen. For example, the first position information of the face key points may be absolute coordinates of the face key points. The absolute coordinates refer to the position coordinates relative to the specific point on the display screen with a specific point (eg, a center point) of the display screen as the origin. Referring to FIG. 2, it shows a schematic diagram of a human face image according to an exemplary embodiment. As shown in FIG. 2 , the key point C of the target face can be a point at the tip of the nose of the face, and is located on the symmetry axis of the face image. The first face key point A and the second face key point B may be two symmetrical points located on both sides of the edge of the face. The inclination angle of the line between the first face key point A and the second face key point B can be used to reflect the rotation angle of the face.

In some embodiments, the terminal may determine the relative position of each trajectory point relative to the face image by processing the absolute position of each trajectory point on the display screen through spatial change processing. Exemplarily, as shown in FIG. 3 , the terminal determines each face image according to the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the moving trajectory. The process of the relative position of the track point relative to the face image may include the following steps 1021 to 1023 .

In step 1021, for each track point, according to the first position information of the target face key point and the track position information of the track point, a translation vector of the track point pointing to the target face key point is determined.

In this embodiment of the present disclosure, for each track point in at least one track point, the terminal may calculate the translation vector of the track point pointing to the target face key point according to the first position information of the target face key point and the track position information of the track point. to perform a translation operation on the track point. The translation vector represents the translation gesture information from the track point to the target face key point, that is, the relative position of the track point and the target face key point.

为(x _p-x _c1,y _p-y _c1) _。 In some embodiments, it is assumed that in the face image displayed on the display page at the current moment, the absolute coordinates (x _c1 , y _c1 ) of the target face key point C. The absolute coordinates (x _a1 , y _a1 ) of the key point A of the first face. The absolute coordinates (x _b1 , y _b1 ) of the second face key point B. And it is assumed that the absolute coordinates (x _p , y _p ) of a track point P in the finger movement track. Then for the trajectory point P, the trajectory point P determined by the terminal points to the translation vector of the target face key point C

is (x _p -x _c1 ,y _p -y _c1 ) _.

In step 1022, a first rotation matrix and a first scaling matrix of the current face pose in the face image are determined according to the first position information of the first face key point and the second face key point.

In this embodiment of the present disclosure, the first rotation matrix may represent the rotation attitude information of the current face attitude in the face image. The first scaling matrix may represent scaling pose information of the current face pose in the face image.

In some embodiments, in the case that the first position information of the face key point and the trajectory position information of the track point are both absolute coordinates on the display screen of the terminal, the terminal according to the first face key point and the second person The process of determining the first position information of the face key point, the first rotation matrix and the first scaling matrix of the current face pose in the face image may include the following steps:

According to the first position information of the first face key point and the second face key point, and the first length, the first rotation matrix is obtained, and the first length is the first face key point pointing to the second face key point. the length of a vector;

According to the reference length of the line connecting the first face key point and the second face key point, and the first length, a first scaling matrix is obtained, and the reference length is the first scale set for the face in the face-up posture in the face image. length.

In this embodiment of the present disclosure, the terminal may obtain a first vector pointing from the first face key point to the second face key point according to the first position information of the first face key point and the second face key point. A first length of the first vector is determined. After the first length is determined, the first rotation matrix may be obtained according to the first position information of the first face key point, the first position information of the second face key point, and the first length.

为(x _a1-x _b1,y _a1-y _b1)。进而计算得到第一向量

的第一长度为|AB|，

Illustratively, continue to take the assumed face key point in step 1021 as an example to illustrate the assumed trajectory point P schematically. The terminal calculates and determines the first vector according to the first position information of the first face key point A is (x _a1 , y _a1 ), and the first position information of the second face key point B is (x _b1 , y _b1 )

is (x _a1 -x _b1 ,y _a1 -y _b1 ). Then calculate the first vector

The first length of is |AB|,

Then, according to the first position information of the first face key point A, the second position information of the second face key point B, and the first length |AB|, the first rotation matrix M _r1 is obtained.

进行旋转处理，即根据人脸图像中当前人脸姿态的旋转姿态信息，对平移向量

进行设定比例的旋转处理。 Among them, the first rotation matrix M _r1 can be used for the translation vector

Perform rotation processing, that is, according to the rotation attitude information of the current face attitude in the face image, the translation vector

Performs rotation processing at the set scale.

的第一长度为|AB|，

In some embodiments, continue to take the assumed face key point in step 1021 as an example to illustrate the assumed trajectory point P in a schematic manner. first vector

The first length of is |AB|,

Then, according to the reference length D of the line connecting the first face key point and the second face key point, and the first length |AB|, the first scaling matrix M _s1 is obtained.

进行缩放处理，即根据人脸图像中当前人脸姿态的缩放姿态信息，对平移向量

进行设定比例的缩放处理。该设定比例可以为D：1。在一些实施例中，D可以为100。 Among them, the first scaling matrix M _s1 can be used for the translation vector

Perform zoom processing, that is, according to the zoom pose information of the current face pose in the face image, the translation vector

Perform scaling processing at the set ratio. The set ratio may be D:1. In some embodiments, D may be 100.

In the embodiment of the present disclosure, due to the connection between the first face key point and the second face key point, the inclination angle can better reflect the rotation angle of the face in the face image displayed on the display page. Therefore, according to the first length of the first vector of the first face key point pointing to the second face key point, the accuracy of the determined first rotation matrix used to indicate the rotation attitude information of the current face in the face image is more accurate. high. And at the same time, the length of the face in the face-up posture in the face image by the connection between the first face key point and the second face key point, and the real first length of the connection in the current face image, can be obtained. On the basis of repeatedly using the relevant information of the connection between the first face key point and the second face key point, a first scaling matrix for indicating the scaling posture information of the current face in the face image is determined, thereby reducing the calculation of the terminal. quantity.

In step 1023, the relative position of the trajectory point relative to the face image is obtained according to the first rotation matrix, the first scaling matrix and the translation vector.

In some embodiments, the process that the terminal obtains the relative position of the trajectory point with respect to the face image according to the first rotation matrix, the first scaling matrix and the translation vector may include: The vector and the first formula to get the relative position. The first formula includes:

表示平移向量。这样，由于轨迹点指向目标人脸关键点的平移向量可以反映轨迹点相对人脸图像的相对距离，第一旋转矩阵可以反映人脸图像中当前人脸的旋转姿态信息。第一缩放矩阵可以反映人脸图像中当前人脸的缩放姿态信息。而第一公式的公式因子包括第一缩放矩阵、第一旋转矩阵和平移向量，因此采用第一公式计算轨迹点相对人脸图像的相对位置，可以考虑到人脸图像中当前人脸的各类姿态信息，从而使得计算得到的轨迹点相对人脸图像的相对位置的准确性较高。 Among them, Q represents the relative position of the trajectory point relative to the face image, M _s1 represents the first scaling matrix, M _r1 represents the first rotation matrix,

represents the translation vector. In this way, since the translation vector of the track point pointing to the target face key point can reflect the relative distance of the track point relative to the face image, the first rotation matrix can reflect the rotation attitude information of the current face in the face image. The first scaling matrix may reflect the scaling posture information of the current face in the face image. The formula factors of the first formula include the first scaling matrix, the first rotation matrix and the translation vector. Therefore, using the first formula to calculate the relative position of the track point relative to the face image can take into account the various types of the current face in the face image. attitude information, so that the accuracy of the relative position of the calculated trajectory point relative to the face image is high.

In the embodiment of the present disclosure, the scheme of calculating the relative position of the trajectory point relative to the face image according to the first rotation matrix and the first scaling matrix of the current face posture in the face image and the translation vector of the trajectory point pointing to the target face key point , since the translation vector of the track point pointing to the target face key point can reflect the relative distance of the track point relative to the face image, the first rotation matrix can reflect the rotation attitude information of the current face in the face image. The first scaling matrix may reflect the scaling posture information of the current face in the face image. Therefore, in the case of considering various pose information of the current face in the face image, the relative position of the trajectory point and the face image can be obtained more realistically. Therefore, the accuracy of the relative position calculated according to the first rotation matrix and the first scaling matrix of the current face posture in the face image is high.

In step 103, image special effect processing is repeatedly performed. The image special effect processing includes: converting the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current moment, so as to obtain the first absolute position of each track point on the display screen; The trajectory points of each first absolute position generate a first special effect line. In the display page, the first special effect line is displayed.

In the embodiment of the present disclosure, after the terminal acquires the movement trajectory input by the user and before the terminal draws the first special effect line, the user's face may also undergo posture changes such as tilt and side head. Therefore, when the terminal calculates the relative position of the trajectory point relative to the face image, and when the terminal performs image special effect processing, the face pose in the face image displayed on the display interface may be different.

Based on this, the terminal needs to calculate the absolute position of the trajectory point on the display screen according to the face image displayed in real time after acquiring the movement trajectory input by the user, so as to generate and display the first special effect line. Similarly, the special effect line (the first special effect line and the subsequent second special effect line collectively) displayed on the terminal has a refresh rate. In the process of each refresh, the image special effect processing needs to be performed repeatedly, so that the terminal can display the latest (latest) special effect line drawn, and the special effect line has the same position relative to the face in each face image. Therefore, it can be regarded as the same line that moves with the face visually. Among them, the user's face may also change in posture during each refresh interval. Therefore, the image special effect processing is performed by using the face key points of the face image displayed by the terminal in real time.

For example, please refer to FIG. 4 , which shows a schematic diagram of a face image of a display page provided by an embodiment of the present disclosure. As shown in FIG. 4 , the shown face image is the face image when the user inputs the movement track. In FIG. 4 , located at the upper left of the head in the face image, the broken line drawn by the dotted line in the shape of an ear is the movement trajectory L0 input by the user. P is a point on the moving trajectory L0. Please refer to FIG. 5 , which shows a schematic diagram of a face image of a display page provided by an embodiment of the present disclosure. As shown in FIG. 5 , the shown face image is the face image displayed by the terminal after the current moment of the display page in the process of executing the image special effect processing. With respect to the face image shown in FIG. 4 , the face image produces a posture change in which the head of the face is tilted. The broken line L1 shown by the dotted line at the upper left of the head in the face image in FIG. 5 is the special effect line corresponding to the movement track input by the user shown in FIG. 4 . The P1 point on the special effect line corresponds to the P point on the movement track. 4 and 5 are face images of the same user at different times. In the face images shown in FIG. 4 and FIG. 5 , the face key points are the same target face key point C, the same first face key point A, and the same second face key B.

In some embodiments, the terminal may repeatedly perform image special effect processing until receiving the special effect closing instruction. Thereby, the display position of the first special effect line (that is, the absolute position on the display screen) will change with the change of the display position of the face image displayed in real time, so as to realize the special effect that the first special effect line moves with the face, and enrich the special effects. display effect. The special effect closing instruction may be triggered after performing a setting operation on the display page. For example, the special effect closing instruction may be triggered by the user performing a setting input for the special effect triggering control. The special effect triggering control may also be a special effect button in the display page. The setting input may include input in the form of click, long press, swipe, or voice for the special effect trigger control.

Wherein, the image special effect processing process includes the following steps:

Convert the relative position according to the second position information of the key point of the face in the face image displayed on the display page after the current moment, so as to obtain the first absolute position of each track point on the display screen;

Connect the track points located at the first absolute positions to generate the first special effect line;

In the display page, the first special effect line is displayed.

In some embodiments, the relative position is converted according to the second position information of the face key points in the face image displayed on the display page after the current moment, so as to obtain the first absolute position of each track point on the display screen.

In the embodiment of the present disclosure, the terminal may acquire the second position information of the key points of the face in the face image displayed on the display page after the current moment. After acquiring the second position information, the terminal can convert the relative position of each track point relative to the face image according to the second position information of the face key point to obtain the first absolute position corresponding to each track point on the display screen. Wherein, the acquired face key points in the face image are the same as at least two face key points in step 102 . In some embodiments, the first absolute position of each track point on the display screen may be represented by coordinates of the track point in the pixel coordinate system of the display screen.

In some embodiments, as shown in FIG. 6 , the terminal converts the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current moment, and obtains each track point on the display screen. The process of the first absolute position may include the following steps 1031 to 1032.

In step 1031, for each track point, a second rotation matrix and a second scaling matrix of the current face pose in the face image are determined according to the second position information of the first face key point and the second face key point.

In the embodiment of the present disclosure, the second rotation matrix may represent the rotation posture information of the current face posture in the face image. The second scaling matrix may represent the scaling pose information of the current face pose in the face image.

In some embodiments, when the second position information of the face key point and the trajectory position information of the trajectory point are both absolute coordinates on the display screen of the terminal, the terminal for each trajectory point in the at least one trajectory point, according to The second position information of the first face key point and the second face key point, the process of determining the second rotation matrix and the second scaling matrix of the current face pose in the face image may include the following steps:

According to the second position information of the first face key point and the second face key point, and the second length, a second rotation matrix is obtained, and the second length is the first face key point pointing to the second face key point. the length of the two vectors;

The first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information and the relative position of the target face key point.

In some embodiments, a second rotation matrix is obtained according to the second position information of the first face key point and the second face key point, and the second length, and the second length is that the first face key point points to the second The length of the second vector of face keypoints.

In this embodiment of the present disclosure, the terminal may obtain a second vector pointing from the first face key point to the second face key point according to the second position information of the first face key point and the second face key point. A second length of the second vector is determined. After the second length is determined, a second rotation matrix may be obtained according to the second position information of the first face key point, the second position information of the second face key point, and the second length.

For example, it is assumed that after the terminal acquires the movement trajectory input by the user, the terminal performs image special effect processing, and before the first special effect line is generated and displayed, the posture of the user's head changes. At this time, the face position of the face image displayed by the terminal on the display page is changed. For example, after the terminal acquires the movement track input by the user, and before the terminal performs image special effect processing and generates and displays the first special effect line, the user's head changes from the posture shown in FIG. 4 to the posture shown in FIG. 5 .

为(x _a2-x _b2,y _a2-y _b2)。确定第二向量

的第二长度为|AB|，

根据第一人脸关键点A的第二位置信息、第二人脸关键点B的第二位置信息以及第二长度|AB|，得到第二旋转矩阵M _r2，即

Continuing to take the assumed face key point in step 1021 as an example, a schematic illustration of the assumed trajectory point P is given. The terminal obtains the second vector according to the second position information (x _a2 , y _a2 ) of the first face key point A and the second position information (x _b2 , y _b2 ) of the second face key point B

is (x _a2 -x _b2 ,y _a2 -y _b2 ). determine the second vector

The second length of is |AB|,

According to the second position information of the first face key point A, the second position information of the second face key point B, and the second length |AB|, the second rotation matrix M _r2 is obtained, that is,

Wherein, the second rotation matrix M _r1 can be used to perform rotation processing changes on the relative positions of the trajectory points, that is, according to the rotation attitude information of the current face posture in the face image, the relative positions of the trajectory points are subjected to a set ratio of rotation processing changes.

In some embodiments, the second scaling matrix is obtained according to the reference length of the line connecting the first face key point and the second face key point, and the second length. The reference length is the second length set for the face in the face-up posture in the face image.

的第二长度为|AB|，

根据第一人脸关键点和第二人脸关键点连线的参考长度D，以及第二长度|AB|，得到第二缩放矩阵M _s2，即

In the embodiment of the present disclosure, the first length and the second length set for the face in the face-up posture in the face image are equal. Illustratively, continue to take the assumed face key point in step 1021 as an example to illustrate the assumed trajectory point P. second vector

The second length of is |AB|,

According to the reference length D of the line connecting the first face key point and the second face key point, and the second length |AB|, the second scaling matrix M _s2 is obtained, that is,

The second scaling matrix M _s2 can be used to perform scaling processing and conversion on the relative position of the trajectory points, that is, performing scaling processing and conversion on the relative position of the trajectory points according to the scaling posture information of the current face posture in the face image. The set ratio may be D:1. In some embodiments, D may be 100.

In the embodiment of the present disclosure, due to the connection between the first face key point and the second face key point, the inclination angle can better reflect the rotation angle of the face in the face image displayed on the display page. Therefore, according to the second length of the second vector of the first face key point pointing to the second face key point, the accuracy of the determined second rotation matrix used to indicate the rotation attitude information of the current face in the face image is more accurate. high. And at the same time, the length of the face in the face-up posture in the face image by the connection between the first face key point and the second face key point, and the real length of the connection in the current face image, can be repeated in Based on the related information of the connection between the first face key point and the second face key point, a second scaling matrix for indicating the scaling posture information of the current face in the face image is determined, thereby reducing the amount of calculation of the terminal.

In step 1032, the first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information and the relative position of the target face key point.

In some embodiments, the process that the terminal obtains the first absolute position of the trajectory point according to the second rotation matrix, the second scaling matrix, the second position information and the relative position of the target face key point may include: the terminal rotates according to the second The matrix, the second scaling matrix, the second position information of the target face key point, the relative position, and the second formula are used to obtain the first absolute position of the track point. The second formula includes:

R=Mr ₂ ·Ms ₂ ·(x _q , y _q ) ^T +(x _c , y _c ) ^T ;

Wherein, R represents the first absolute position of the trajectory point on the display screen, Mr ₂ represents the second rotation matrix, Ms ₂ represents the second scaling matrix, (x _q , y _q ) represents the relative position of the trajectory point relative to the face image, ( x _c , y _c ) represents the second position information of the key points of the target face, and T represents the transposition processing.

In this way, since the second rotation matrix can reflect the rotation posture information of the current face in the face image. The second scaling matrix can reflect the scaling posture information of the current face in the face image. Therefore, using the second formula to determine the first absolute position of the track point can not only make the first absolute position of the track point change with the change of the display position of the face image, but also change due to the rotation posture of the current face in the face image. information as well as the zoom pose information. In this way, the generated first special effect lines can not only follow the movement of the human face in the face image, but also can rotate and zoom following the human face in the face image, thereby enriching the special effect display effect by connecting the trajectory points of the first absolute positions.

It should be noted that, if the facial posture in the face image displayed by the terminal when performing the aforementioned step 102 changes from the facial posture in the facial image displayed by the terminal when performing step 103, then for the same trajectory point, The relative position of the trajectory point obtained in step 102 relative to the face image is different from the absolute position of the trajectory point obtained in step 103 relative to the display screen. If the facial posture in the face image displayed by the terminal when performing the aforementioned step 102 does not change from the facial posture in the facial image displayed by the terminal when performing the step 103, then for the same trajectory point, the result obtained in the step 102 The relative position of the track point relative to the face image is the same as the absolute position of the track point obtained in step 103 relative to the display screen. That is, the two positions coincide.

In the embodiment of the present disclosure, the first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information and the relative position of the target face key points, because the second rotation matrix can reflect the human The rotational pose information of the current face in the face image. The second scaling matrix may reflect the scaling posture information of the current face in the face image. Therefore, using the second rotation matrix, the second scaling matrix and the second position information of the target face key point to convert the relative position of the trajectory point into the first absolute position of the trajectory point can not only make the first absolute position of the trajectory point It changes with the change of the display position of the face image, and also changes due to the change of the rotation attitude information and the zoom attitude information of the current face in the face image. In this way, the generated first special effect lines can not only follow the movement of the human face in the face image, but also can rotate and zoom following the human face in the face image, thereby enriching the special effect display effect by connecting the trajectory points of the first absolute positions.

In some embodiments, the trajectory points located at the first absolute positions are connected to generate a first special effect line.

In some embodiments, the terminal may generate the first special effect line by connecting the trajectory points of the first absolute positions corresponding to the trajectory points according to the arrangement order of the trajectory points in the movement trajectory input by the user.

Exemplarily, the movement track includes track points X1 , track points X2 and track points X3 arranged in sequence. The first absolute position of the trajectory point X1 is Y1. The first absolute position of the trajectory point X2 is Y2. The first absolute position of the trajectory point X3 is Y3. The terminal sequentially connects the trajectory point located at the first absolute position Y1, the trajectory point located at the first absolute position Y2 and the trajectory located at the first absolute position Y3 according to the sorting order of the trajectory point X1, the trajectory point X2 and the trajectory point X3. Click to generate the first special effect line.

In some embodiments, in the display page, the first special effect line is displayed.

In this embodiment of the present disclosure, the terminal may display the generated first special effect line on the display page currently displayed by the terminal.

In the embodiment of the present disclosure, the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory of at least one trajectory point in the movement trajectory input by the user on the display page can be obtained. Position information, to determine the relative position of each track point relative to the face image. In order to repeat the conversion of the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current moment, the first absolute position of each track point on the display screen is obtained, and the first absolute position of each track point on the display screen is obtained. , the process of displaying the first special effect line formed by connecting the track points located at the first absolute positions.

In the above technical solution, the first special effect line is drawn according to the movement trajectory input by the user, so that the user can draw the special effect independently. Moreover, after obtaining the relative positions of each track point in the moving track and the current face image, the position information of the key points of the face and the relative position of each track point in the face image displayed in real time on the display page can be used to determine each track. point at the first absolute position on the display screen, so that the first special effect line is generated and displayed after connecting the track points located at the first absolute positions. In this way, the display position of the generated first special effect line will change with the change of the display position of the face image displayed in real time on the display page, realizing the special effect that the first special effect line moves with the face, and enriching the special effect display effect.

In some embodiments, the terminal may not only draw the first special effect line corresponding to the movement trajectory independently drawn according to the movement trajectory input by the user. A second special effect line that is symmetrical to the first special effect line may also be drawn according to the first special effect line. The second special effect line and the first special effect line are left-right symmetrical on the basis of the face in the face image. For example, as shown in FIG. 5 , the terminal can not only draw the special effect line L1 in the shape of an ear as shown by the dotted line at the upper left of the head in the face image in FIG. 5 . It is also possible to draw a special effect line L2 in the shape of an ear as shown by the dotted line at the upper right of the head in the face image in FIG. 5 . The special effect line L1 and the special effect line L2 are left-right symmetrical on the basis of the face in the face image. The point P1 on the special effect line L1 and the point P2 on the special effect line L2 are left-right symmetrical on the basis of the face in the face image. In this embodiment of the present disclosure, the image special effect processing may further include the following:

According to the first special effect line, a second special effect line symmetrical with the first special effect line is generated, and the second special effect line and the first special effect line are left-right symmetrical based on the face in the face image;

Display the second effect line in the display page.

In some embodiments, a second special effect line symmetrical to the first special effect line is generated according to the first special effect line. The second special effect line and the first special effect line are left and right symmetrical on the basis of the face in the face image.

In this embodiment of the present disclosure, the terminal may generate, according to the first special effect line, a left-right symmetrical second special effect line based on the face in the face image currently displayed by the terminal. There are various implementations for the terminal to generate a second special effect line symmetrical to the first special effect line according to the first special effect line. The embodiments of the present disclosure are described by taking the following two examples as examples.

In a first implementation manner, as shown in FIG. 7 , the process for the terminal to generate a second special effect line symmetrical to the first special effect line according to the first special effect line may include the following steps 701 to 702 .

In step 701, the second absolute position of each symmetrical point on the display screen is determined according to the second position information of the face key point and the relative position of each symmetrical point.

In the embodiment of the present disclosure, the terminal performs the above step 102 according to the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the movement trajectory, After determining the relative position of each track point relative to the face image. The terminal can also determine the relative position of the symmetrical point of each track point with respect to the face image according to the relative position of each track point with respect to the face image. Among them, the symmetrical point and the trajectory point are left and right symmetrical with the face as the reference.

In some embodiments, when the relative position of the track point relative to the face image is the relative coordinate of the track point relative to the face image, if the relative coordinates belong to a two-dimensional coordinate system, one of its axes may be the symmetry axis of the face image. . Then, according to the relative position of each trajectory point relative to the face image, the process of determining the relative position of the symmetrical point of each trajectory point relative to the face image by the terminal may include: the terminal performs positive and negative on the coordinate value of the first direction in the relative coordinates of the trajectory point. Digital conversion processing is performed to obtain processed coordinate values, and the first direction is perpendicular to the symmetry axis of the face image. The relative position of the track point is updated, so that the coordinate value of the first direction in the updated relative position is the processed coordinate value. The updated relative position is determined as the relative position of the symmetry point. Wherein, when the symmetry point and the trajectory point are left-right symmetrical with the face as a reference, the first direction may be a direction perpendicular to the symmetry axis of the left-right symmetry of the face image.

For example, it is assumed that the relative coordinates of the trajectory point P1 determined by the terminal relative to the face image are (x _q , y _q ). And the first direction is a direction perpendicular to the symmetry axis of the left-right symmetry of the face image, that is, the x-axis direction. The terminal performs positive and negative conversion processing on the coordinate value in the first direction in the relative coordinates of the trajectory point, and obtains the processed coordinate value -x _q . Then the relative position of the symmetrical point is (-x _q , y _q ).

In the embodiment of the present disclosure, the terminal determines the second absolute position of each symmetrical point on the display screen according to the second position information of the face key point and the relative position of each symmetrical point. That is, the terminal converts the relative position of each symmetrical point according to the second position information of the key point of the face in the face image displayed on the display page after the current moment, and obtains the second absolute position of each symmetrical point on the display screen. . The process of determining the second absolute position of each symmetrical point on the display screen by the terminal according to the second position information of the key point of the face and the relative position of each symmetrical point may refer to step A in the aforementioned image special effect processing process, according to the display page In the face image displayed after the current moment, the second position information of the key points of the face converts the relative positions to obtain the first absolute position of each track point on the display screen, which is not performed in this embodiment of the present disclosure. Repeat.

In step 702, the symmetrical points located at the second absolute positions are connected to generate a second special effect line.

In some embodiments, the terminal may, according to the arrangement order of the trajectory points in the movement trajectory input by the user, connect the trajectory points of the second absolute positions among the symmetrical points corresponding to the trajectory points, and generate the second special effect line .

Exemplarily, the movement track includes track points X1 , track points X2 and track points X3 arranged in sequence. The second absolute position of the symmetrical point X4 corresponding to the trajectory point X1 is Y4. The second absolute position of the symmetrical point X5 corresponding to the trajectory point X2 is Y5. The second absolute position of the symmetrical point X6 corresponding to the trajectory point X3 is Y6. The terminal sequentially connects the trajectory point located at the second absolute position Y4, the trajectory point located at the second absolute position Y5 and the trajectory located at the second absolute position Y6 according to the sorting order of the trajectory point X1, the trajectory point X2 and the trajectory point X3 Click to generate the second special effect line.

For the second implementation manner, as shown in FIG. 8 , the process of generating the second special effect line symmetrical to the first special effect line by the terminal according to the first special effect line may include the following steps 801 to 802 .

In step 801, the second absolute position of the symmetrical point of each trajectory point on the display screen is determined according to the second position information of the face key point and the first absolute position of each trajectory point. The trajectory point and the symmetry point are left and right symmetrical based on the face.

In some embodiments, when the second position information of the face key points is the absolute coordinates of the face key points on the display screen. The process that the terminal determines the second absolute position of the symmetrical point of each trajectory point on the display screen according to the second position information of the face key point and the first absolute position of each trajectory point may include the following steps:

According to the second position information of the first face key point and the second face key point, a second vector pointing from the first face key point to the second face key point, and a third vector perpendicular to the second vector are obtained ;

According to the second position information of the target face key point and the first absolute position of the track point, the fourth vector pointing to the track point from the target face key point is obtained;

According to the second vector, the third vector, the fourth vector and the second position information of the target face key point, the second absolute position of the symmetrical point is obtained.

In some embodiments, according to the second position information of the first face key point and the second face key point, a second vector pointing from the first face key point to the second face key point is obtained, and The third vector of two vectors.

为(x _a2-x _b2,y _a2-y _b2)。以及垂直与第二向量

的第三向量

为(y _b2-y _a2,x _a2-x _b2)。 Illustratively, continue to take the assumed face key point in step 1021 as an example to illustrate the assumed trajectory point P schematically. The terminal obtains the second vector according to the second position information (x _a2 , y _a2 ) of the first face key point A and the second position information (x _b2 , y _b2 ) of the second face key point B

is (x _a2 -x _b2 ,y _a2 -y _b2 ). and the vertical with the second vector

the third vector of

is (y _b2 -y _a2 ,x _a2 -x _b2 ).

In some embodiments, according to the second position information of the target face key point and the first absolute position of the track point, a fourth vector pointing from the target face key point to the track point is obtained.

为(x _r-x _c,y _r-y _c) _。 Illustratively, continue to take the assumed face key point in step 1021 as an example to illustrate the assumed trajectory point P schematically. Assume that the first absolute position of the trajectory point P is (x _r , y _r ). According to the second position information (x _c , y _c ) of the target face key point C and the first absolute position (x _r , y _r ) of the trajectory point P, the terminal obtains the first position from the target face key point to the trajectory point. four vector

is (x _r -x _c ,y _r -y _c ) _.

In some embodiments, the second absolute position of the symmetrical point is obtained according to the second vector, the third vector, the fourth vector and the second position information of the target face key point.

In some embodiments, the terminal obtains the second absolute position of the symmetry point according to the second vector, the third vector, the fourth vector, the second position information of the target face key point, and the third formula. The third formula includes:

表示第二向量、

表示第三向量、

表示第四向量以及(x _c,y _c)表示目标人脸关键点的第二当前位置信息。 where M represents the second absolute position of the symmetry point,

represents the second vector,

represents the third vector,

represents the fourth vector and (x _c , y _c ) represents the second current position information of the target face key point.

In the embodiment of the present disclosure, in the first implementation manner, the symmetrical point and the trajectory point can be directly left and right symmetrical with the face as the reference, and the relative coordinates of the trajectory point can be directly converted to the first one that is perpendicular to the symmetry axis of the face image. After performing positive and negative conversion processing on the coordinate value of the direction, the obtained relative coordinate is determined as the relative coordinate of the symmetrical point of the trajectory point. Then, according to the steps similar to those in the aforementioned step 103, the relative positions of the symmetrical points are converted according to the second position information of the face key points in the face image displayed on the display page after the current moment, to obtain the symmetrical points on the display screen. Second absolute position. Compared with the second implementation manner, the process of determining the second absolute position of the symmetrical point on the display screen is simplified, and the calculation efficiency of the second absolute position of the symmetrical point is improved.

In step 802, the symmetrical points located at the second absolute positions are connected to generate a second special effect line.

In some embodiments, the terminal may, according to the arrangement order of the trajectory points in the movement trajectory input by the user, connect the trajectory points of the second absolute positions among the symmetrical points corresponding to the trajectory points, and generate the second special effect line .

Exemplarily, the movement track includes track points X1 , track points X2 and track points X3 arranged in sequence. The second absolute position of the symmetrical point X4 corresponding to the trajectory point X1 is Y4. The second absolute position of the symmetrical point X5 corresponding to the trajectory point X2 is Y5. The second absolute position of the symmetrical point X6 corresponding to the trajectory point X3 is Y6. The terminal sequentially connects the trajectory point located at the second absolute position Y4, the trajectory point located at the second absolute position Y5 and the trajectory located at the second absolute position Y6 according to the sorting order of the trajectory point X1, the trajectory point X2 and the trajectory point X3 Click to generate the second special effect line.

In some embodiments, the second effect line is displayed in the display page.

The terminal may display the generated second special effect line on the display page currently displayed by the terminal.

In this embodiment of the present disclosure, the terminal may generate, according to the first special effect line, a second special effect line that is left-right symmetrical with the first special effect line based on the human face in the face image. And display the second special effect line in the display page. It realizes the function that the user can independently draw the left and right symmetrical special effects lines based on the face in the face image.

In addition, after acquiring the relative positions of the symmetrical points of each trajectory point in the moving trajectory input by the user, the terminal adopts the second position information of the key points of the face and the relative positions of each symmetrical point in the face image displayed on the real-time page. , the determined second absolute position of the symmetrical point of each track point on the display screen. Thereby, the second effect lines generated by connecting the symmetrical points of the second absolute positions are connected. Therefore, the display position of the generated second special effect line will change with the change of the display position of the face image displayed on the display page in real time. The special effect of the second special effect line following the movement of the face is realized. In this way, the left and right symmetrical special effects lines drawn on the basis of the human face can move with the human face, enriching the special effect display effect.

In the embodiment of the present disclosure, the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory of at least one trajectory point in the movement trajectory input by the user on the display page can be obtained. Position information, to determine the relative position of each track point relative to the face image. In order to repeat the conversion of the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current moment, the first absolute position of each track point on the display screen is obtained, and the first absolute position of each track point on the display screen is obtained. , the process of displaying the first special effect line formed by connecting the track points located at the first absolute positions. In the above technical solution, the first special effect line is drawn according to the movement trajectory input by the user, which realizes the function of the user to draw the special effect independently. Moreover, after obtaining the relative positions of each track point in the moving track and the current face image, the position information of the key points of the face and the relative position of each track point in the face image displayed in real time on the display page can be used to determine each track. point at the first absolute position on the display screen, so that the first special effect line is generated and displayed after connecting the track points located at the first absolute positions. In this way, the display position of the generated first special effect line will change with the change of the display position of the face image displayed in real time on the display page, realizing the special effect that the first special effect line moves with the face, and enriching the special effect display effect.

Fig. 9 is a block diagram of an image processing apparatus according to an exemplary embodiment. Referring to FIG. 9 , the image processing apparatus 900 includes: an acquisition module 901 , a determination module 902 and an image special effect processing module 903 .

an acquisition module 901, configured to acquire the movement track input by the user in the display page including the face image in response to the special effect display instruction;

The determining module 902 is configured to determine the relative position of each track point relative to the human face according to the first position information of at least two face key points and the track position information of at least one track point in the moving track in the face image displayed on the display page at the current moment. the relative position of the face image;

The image special effect processing module 903 is used to repeatedly perform image special effect processing, and the image special effect processing includes:

Convert the relative position according to the second position information of the key point of the face in the face image displayed on the display page after the current moment, so as to obtain the first absolute position of each track point on the display screen;

Connect the track points located at the first absolute positions to generate the first special effect line;

In the display page, the first special effect line is displayed.

In a possible implementation manner, the at least two face key points include: a first face key point, a second face key point, and a target face key point, and the first face key point and the second face key point are related to The target face key point is symmetrical, and the target face key point is any face key point on the symmetry axis of the face image.

In a possible implementation manner, the determining module 902 is further configured to:

For each track point, according to the first position information of the target face key point and the track position information of the track point, determine the translation vector of the track point pointing to the target face key point;

Determine the first rotation matrix and the first scaling matrix of the current face pose in the face image according to the first position information of the first face key point and the second face key point;

According to the first rotation matrix, the first scaling matrix and the translation vector, the relative position of the trajectory point relative to the face image is obtained.

In a possible implementation manner, both the first position information and the track position information include absolute coordinates on the display screen, and the determining module 902 is further configured to:

According to the first position information of the first face key point and the second face key point, and the first length, the first rotation matrix is obtained, and the first length is the first face key point pointing to the second face key point. the length of a vector;

According to the reference length of the line connecting the first face key point and the second face key point, and the first length, a first scaling matrix is obtained, and the reference length is the first scale set for the face in the face-up posture in the face image. length.

In a possible implementation manner, the determining module 902 is further configured to:

The relative position is obtained according to the first scaling matrix, the first rotation matrix, the translation vector and the first formula, and the first formula includes:

表示平移向量。 Wherein, Q represents the relative position, Ms ₁ represents the first scaling matrix, Mr ₁ represents the first rotation matrix,

represents the translation vector.

In a possible implementation manner, the image special effect processing module 903 is further configured to:

For each track point, according to the second position information of the first face key point and the second face key point, determine the second rotation matrix and the second scaling matrix of the current face posture in the face image;

The first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information and the relative position of the target face key point.

In a possible implementation manner, the second position information includes absolute coordinates on the display screen, and the image special effect processing module 903 is further configured to:

According to the second position information of the first face key point and the second face key point, and the second length, a second rotation matrix is obtained, and the second length is the first face key point pointing to the second face key point. the length of the two vectors;

According to the reference length of the line connecting the first face key point and the second face key point, as well as the second length, a second scaling matrix is obtained, and the reference length is the second set for the face in the face-up posture in the face image. length.

In a possible implementation manner, the image special effect processing module 903 is further configured to:

According to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, the relative position and the second formula, the first absolute position of the trajectory point is obtained, and the second formula includes:

R=Mr ₂ ·Ms ₂ ·(x _q , y _q ) ^T +(x _c , y _c ) ^T ;

Wherein, R represents the first absolute position, Mr ₂ represents the second rotation matrix, Ms ₂ represents the second scaling matrix, (x _q , y _q ) represents the relative position, (x _c , y _c ) represents the target face key point The second position information, and T represents the transposition process.

In a possible implementation manner, the image special effect processing further includes:

According to the first special effect line, a second special effect line symmetrical with the first special effect line is generated, and the second special effect line and the first special effect line are left-right symmetrical based on the face in the face image;

Display the second effect line in the display page.

In a possible implementation manner, the determining module 902 is further configured to:

According to the relative position of each track point relative to the face image, determine the relative position of the symmetry point of each track point relative to the face image, and the symmetry point and the track point are left and right symmetrical with the face as the benchmark;

The image special effect processing module 903 is also used for:

Determine the second absolute position of each symmetrical point on the display screen according to the second position information of the face key point and the relative position of each symmetrical point;

Connect the symmetrical points at each second absolute position to generate a second effect line.

In a possible implementation manner, the relative position includes relative coordinates relative to the face image, and the determining module 902 is further configured to:

Perform positive and negative conversion processing on the coordinate value of the first direction in the relative coordinates of the trajectory point to obtain the processed coordinate value, and the first direction is perpendicular to the symmetry axis of the face image; update the relative position of the trajectory point, so that the updated The coordinate value of the first direction in the relative position is the processed coordinate value; it is determined that the updated relative position is the relative position of the symmetrical point.

In this embodiment of the present disclosure, the determination module can be used to determine the first position information of at least two face key points in the face image displayed on the display page at the current moment, and at least one trajectory point in the movement trajectory input by the user on the display page. The position information of the trajectory is determined, and the relative position of each trajectory point relative to the face image is determined. In order to make the image special effect processing module repeatedly perform the conversion according to the second position information of the face key points in the face image displayed on the display page after the current moment, the relative positions are converted to obtain the first absolute position of each track point on the display screen. , and on the display page, the process of displaying the first special effect line formed by connecting the trajectory points located at the first absolute positions. In the above technical solution, the first special effect line is drawn according to the movement trajectory input by the user, which realizes the function of the user to draw the special effect independently. Moreover, after obtaining the relative positions of each track point in the moving track and the current face image, the position information of the key points of the face and the relative position of each track point in the face image displayed in real time on the display page can be used to determine each track. point at the first absolute position on the display screen, so that the first special effect line is generated and displayed after connecting the track points located at the first absolute positions. In this way, the display position of the generated first special effect line will change with the change of the display position of the face image displayed in real time on the display page, realizing the special effect that the first special effect line moves with the face, and enriching the special effect display effect.

Fig. 10 is a block diagram of an electronic device according to an exemplary embodiment. The electronic device may be a terminal. The electronic device 1000 can be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III, a moving picture expert compression standard Audio Layer 3), MP4 (Moving Picture Experts Group Audio Layer IV, a moving picture expert compression standard Audio Layer 4) Player, Laptop or Desktop. Electronic device 1000 may also be called user equipment, portable terminal, laptop terminal, desktop terminal, and the like by other names.

Generally, the electronic device 1000 includes: a processor 1001 and a memory 1002 .

The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 1001 can use at least one hardware form among DSP (Digital Signal Processing, digital signal processing), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array, programmable logic array) accomplish. The processor 1001 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake-up state, also called CPU (Central Processing Unit, central processing unit); the coprocessor is A low-power processor for processing data in a standby state. In some embodiments, the processor 1001 may be integrated with a GPU (Graphics Processing Unit, image processor), and the GPU is used for rendering and drawing the content that needs to be displayed on the display screen. In some embodiments, the processor 1001 may further include an AI (Artificial Intelligence, artificial intelligence) processor, where the AI processor is used to process computing operations related to machine learning.

Memory 1002 may include one or more non-volatile computer-readable storage media, which may be non-transitory. Memory 1002 may also include high-speed random access memory, as well as non-volatile memory, such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one instruction, and the at least one instruction is used to be executed by the processor 1001 to realize the information display provided by the method embodiments in this application. method.

In some embodiments, the electronic device 1000 may further include: a peripheral device interface 1003 and at least one peripheral device. The processor 1001, the memory 1002 and the peripheral device interface 1003 may be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 1003 through a bus, a signal line or a circuit board. In some embodiments, the peripheral device includes at least one of a radio frequency circuit 1004 , a display screen 1005 , a camera 1006 , an audio circuit 1007 , a positioning component 1008 and a power supply 1009 .

The peripheral device interface 1003 may be used to connect at least one peripheral device related to I/O (Input/Output) to the processor 1001 and the memory 1002 . In some embodiments, processor 1001, memory 1002, and peripherals interface 1003 are integrated on the same chip or circuit board; in some other embodiments, any one of processor 1001, memory 1002, and peripherals interface 1003 or The two may be implemented on a separate chip or circuit board, which is not limited by the embodiments of the present disclosure.

The radio frequency circuit 1004 is used for receiving and transmitting RF (Radio Frequency, radio frequency) signals, also called electromagnetic signals. The radio frequency circuit 1004 communicates with the communication network and other communication devices through electromagnetic signals. The radio frequency circuit 1004 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. In some embodiments, radio frequency circuitry 1004 includes: an antenna system, an RF transceiver, one or more amplifiers, tuners, oscillators, digital signal processors, codec chipsets, subscriber identity module cards, and the like. The radio frequency circuit 1004 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocols include, but are not limited to, metropolitan area networks, mobile communication networks of various generations (2G, 3G, 4G and 5G), wireless local area networks and/or WiFi (Wireless Fidelity, wireless fidelity) networks. In some embodiments, the radio frequency circuit 1004 may further include a circuit related to NFC (Near Field Communication, short-range wireless communication), which is not limited in this application.

The display screen 1005 is used for displaying UI (User Interface, user interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 1005 is a touch display screen, the display screen 1005 also has the ability to acquire touch signals on or above the surface of the display screen 1005 . The touch signal can be input to the processor 1001 as a control signal for processing. At this time, the display screen 1005 may also be used to provide virtual buttons and/or virtual keyboards, also referred to as soft buttons and/or soft keyboards. In some embodiments, there may be one display screen 1005, which is arranged on the front panel of the electronic device 1000; in other embodiments, there may be at least two display screens 1005, which are respectively arranged on different surfaces of the electronic device 1000 or in a folded design. ; In still other embodiments, the display screen 1005 may be a flexible display screen, disposed on a curved surface or a folding surface of the electronic device 1000 . Even, the display screen 1005 can also be set as a non-rectangular irregular figure, that is, a special-shaped screen. The display screen 1005 can be prepared by using materials such as LCD (Liquid Crystal Display, liquid crystal display), OLED (Organic Light-Emitting Diode, organic light emitting diode).

The camera assembly 1006 is used to capture images or video. In some embodiments, camera assembly 1006 includes a front-facing camera and a rear-facing camera. Usually, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth-of-field camera to realize the background blur function, the main camera It is integrated with the wide-angle camera to achieve panoramic shooting and VR (Virtual Reality, virtual reality) shooting functions or other integrated shooting functions. In some embodiments, the camera assembly 1006 may also include a flash. The flash can be a single color temperature flash or a dual color temperature flash. Dual color temperature flash refers to the combination of warm light flash and cold light flash, which can be used for light compensation under different color temperatures.

Audio circuitry 1007 may include a microphone and speakers. The microphone is used to collect the sound waves of the user and the environment, convert the sound waves into electrical signals, and input them to the processor 1001 for processing, or to the radio frequency circuit 1004 to realize voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively disposed in different parts of the electronic device 1000 . The microphone may also be an array microphone or an omnidirectional collection microphone. The speaker is used to convert the electrical signal from the processor 1001 or the radio frequency circuit 1004 into sound waves. The loudspeaker can be a traditional thin-film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert electrical signals into sound waves audible to humans, but also convert electrical signals into sound waves inaudible to humans for distance measurement and other purposes. In some embodiments, the audio circuit 1007 may also include a headphone jack.

The positioning component 1008 is used to locate the current geographic location of the electronic device 1000 to implement navigation or LBS (Location Based Service). The positioning component 1008 may be a positioning component based on the GPS (Global Positioning System, global positioning system) of the United States, the Beidou system of China, the Grenas system of Russia, or the Galileo system of the European Union.

Power supply 1009 is used to power various components in electronic device 1000 . The power source 1009 may be alternating current, direct current, disposable batteries or rechargeable batteries. When the power source 1009 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

In some embodiments, the electronic device 1000 also includes one or more sensors 1010 . The one or more sensors 1010 include, but are not limited to, an acceleration sensor 1011, a gyro sensor 1012, a pressure sensor 1013, a fingerprint sensor 1014, an optical sensor 1015, and a proximity sensor 1016.

The acceleration sensor 1011 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the electronic device 1000 . For example, the acceleration sensor 1011 can be used to detect the components of the gravitational acceleration on the three coordinate axes. The processor 1001 can control the display screen 1005 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 1011 . The acceleration sensor 1011 can also be used for game or user movement data collection.

The gyroscope sensor 1012 can detect the body direction and rotation angle of the electronic device 1000 , and the gyroscope sensor 1012 can cooperate with the acceleration sensor 1011 to collect the 3D actions of the user on the electronic device 1000 . The processor 1001 can implement the following functions according to the data collected by the gyro sensor 1012: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

The pressure sensor 1013 may be disposed on the side frame of the electronic device 1000 and/or the lower layer of the display screen 1005 . When the pressure sensor 1013 is disposed on the side frame of the electronic device 1000 , the user's holding signal of the electronic device 1000 can be detected, and the processor 1001 can perform left and right hand recognition or quick operation according to the holding signal collected by the pressure sensor 1013 . When the pressure sensor 1013 is disposed on the lower layer of the display screen 1005, the processor 1001 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 1005. The operability controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

The fingerprint sensor 1014 is used to collect the user's fingerprint, and the processor 1001 identifies the user's identity according to the fingerprint collected by the fingerprint sensor 1014, or the fingerprint sensor 1014 identifies the user's identity according to the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 1001 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 1014 may be provided on the front, back, or side of the electronic device 1000 . When the electronic device 1000 is provided with physical buttons or a manufacturer's logo, the fingerprint sensor 1014 can be integrated with the physical buttons or the manufacturer's logo.

The optical sensor 1015 is used to collect ambient light intensity. In one embodiment, the processor 1001 can control the display brightness of the display screen 1005 according to the ambient light intensity collected by the optical sensor 1015 . In some embodiments, when the ambient light intensity is high, the display brightness of the display screen 1005 is increased; when the ambient light intensity is low, the display brightness of the display screen 1005 is decreased. In another embodiment, the processor 1001 may also dynamically adjust the shooting parameters of the camera assembly 1006 according to the ambient light intensity collected by the optical sensor 1015 .

A proximity sensor 1016 , also called a distance sensor, is usually provided on the front panel of the electronic device 1000 . The proximity sensor 1016 is used to collect the distance between the user and the front of the electronic device 1000 . In one embodiment, when the proximity sensor 1016 detects that the distance between the user and the front of the electronic device 1000 is gradually decreasing, the processor 1001 controls the display screen 1005 to switch from the bright screen state to the off screen state; when the proximity sensor 1016 detects When the distance between the user and the front of the electronic device 1000 gradually increases, the processor 1001 controls the display screen 1005 to switch from the off-screen state to the bright-screen state.

Those skilled in the art can understand that the structure shown in FIG. 10 does not constitute a limitation on the electronic device 1000, and may include more or less components than the one shown, or combine some components, or adopt different component arrangements.

In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, when the instructions in the storage medium are executed by the processor of the electronic device, the electronic device can execute the methods provided by the above method embodiments. image processing methods. For example, the computer-readable storage medium can be ROM (Read-Only Memory, read-only memory), RAM (Random Access Memory, random access memory), CD-ROM (Compact Disc Read-Only Memory, read-only optical disk), Tape, floppy disk, and optical data storage devices, etc.

In an exemplary embodiment, there is also provided a computer program product, including a computer program. When the computer program is executed by the processor, the image processing methods provided by the above method embodiments can be executed.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

All the embodiments of the present disclosure can be implemented alone or in combination with other embodiments, which are all regarded as the protection scope required by the present disclosure.

Claims (35)

An image processing method, wherein the method comprises: In response to the special effect display instruction, acquiring the movement track input by the user in the display page including the face image; According to the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the movement trajectory, determine the relative position of each trajectory point relative to all the trajectory points. relative position of the face image; The image special effect processing is repeatedly performed, and the image special effect processing includes: Convert the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current time, to obtain the first position of each track point on the display screen. absolute position; Connect the trajectory points located at each of the first absolute positions to generate a first special effect line; In the display page, the first special effect line is displayed. The method according to claim 1, wherein the at least two face key points include: a first face key point, a second face key point and a target face key point, and the first face key point is the same as the The second face key point is symmetrical with respect to the target face key point, and the target face key point is any face key point on the symmetry axis of the face image. The method according to claim 2, wherein in the face image displayed at the current moment according to the display page, the first position information of at least two face key points and at least one track point in the movement track The position information of the trajectory, determine the relative position of each of the trajectory points relative to the face image, including: For each of the track points, according to the first position information of the target face key point and the track position information of the track point, determine the translation vector of the track point pointing to the target face key point; According to the first position information of the first face key point and the second face key point, determine the first rotation matrix and the first scaling matrix of the current face posture in the face image; According to the first rotation matrix, the first scaling matrix and the translation vector, the relative position of the trajectory point relative to the face image is obtained. The method according to claim 3, wherein the first position information and the track position information both include absolute coordinates on the display screen, Determining the first rotation matrix and the first scaling matrix of the current face in the face image according to the first position information of the first face key point and the second face key point, including: According to the first position information of the first face key point and the second face key point, and the first length, the first rotation matrix is obtained, and the first length is the first face key point to the length of the first vector of the second face key point; According to the reference length of the connecting line between the first face key point and the second face key point, and the first length, the first scaling matrix is obtained, and the reference length is for the face image The first length set by the face in the facing posture. The method according to claim 3, wherein the obtaining the relative position of the trajectory point relative to the face image according to the first rotation matrix, the first scaling matrix and the translation vector comprises: The relative position is obtained according to the first scaling matrix, the first rotation matrix, the translation vector and a first formula, where the first formula includes:

Wherein, Q represents the relative position, Ms ₁ represents the first scaling matrix, Mr ₁ represents the first rotation matrix,

represents the translation vector. The method according to claim 2, wherein the relative position is converted according to the second position information of the face key point in the face image displayed on the display page after the current moment, Obtain the first absolute position of each track point on the display screen, including: For each of the trajectory points, according to the second position information of the first face key point and the second face key point, determine the second rotation matrix and the second rotation matrix of the current face pose in the face image scaling matrix; The first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, and the relative position. The method according to claim 6, wherein the second position information includes absolute coordinates on a display screen, and the second position according to the first face key point and the second face key point information, determine the second rotation matrix and the second scaling matrix of the current face posture in the face image, including: According to the second position information of the first face key point and the second face key point, and the second length, the second rotation matrix is obtained, and the second length is the first face key point point to the length of the second vector of the second face key point; According to the reference length of the line connecting the first face key point and the second face key point, and the second length, the second scaling matrix is obtained, and the reference length is for the face image The second length set by the face in the facing posture. The method according to claim 6, wherein the trajectory is obtained according to the second rotation matrix, the second scaling matrix, the second position information of the target face key point and the relative position The first absolute position of the point, including: According to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, the relative position and the second formula, the first absolute position of the trajectory point is obtained, and the The second formula includes: R=Mr ₂ ·Ms ₂ ·(x _q , y _q ) ^T +(x _c , y _c ) ^T ; Wherein, R represents the first absolute position, Mr ₂ represents the second rotation matrix, Ms ₂ represents the second scaling matrix, (x _q , y _q ) represents the relative position, (x _c , y _{c )} ) represents the second position information of the target face key point, and T represents the transposition process. The method according to any one of claims 1-8, wherein the image special effect processing further comprises: generating a second special effect line symmetrical to the first special effect line according to the first special effect line, where the second special effect line and the first special effect line are left-right symmetrical with respect to the face in the face image; The second special effect line is displayed in the display page. The method of claim 9, wherein the method further comprises: According to the relative position of each track point relative to the face image, determine the relative position of the symmetry point of each track point relative to the face image, and the symmetry point and the track point take the face as the The benchmark is left and right symmetrical; The generating a second special effect line symmetrical with the first special effect line according to the first special effect line, comprising: Determine the second absolute position of each of the symmetrical points on the display screen according to the second position information of the face key points and the relative positions of each of the symmetrical points; Connecting the symmetrical points located at each of the second absolute positions to generate the second special effect line. The method according to claim 10, wherein the relative positions include relative coordinates relative to the face image, and the trajectory points are determined according to the relative positions of the trajectory points relative to the face image. The relative position of the symmetry point relative to the face image, including: Perform positive and negative conversion processing on the coordinate value of the first direction in the relative coordinates of the trajectory point to obtain the processed coordinate value, and the first direction is perpendicular to the symmetry axis of the face image; updating the relative position of the trajectory point, so that the coordinate value of the first direction in the updated relative position is the processed coordinate value; The updated relative position is determined as the relative position of the symmetry point. An image processing device, wherein the device comprises: an acquisition module, used for acquiring the movement track input by the user in the display page including the face image in response to the special effect display instruction; The determining module is configured to determine the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the moving trajectory. the relative position of the trajectory point relative to the face image; An image special effect processing module, configured to repeatedly perform image special effect processing, wherein the image special effect processing includes: Convert the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current time, to obtain the first position of each track point on the display screen. absolute position; Connect the trajectory points located at each of the first absolute positions to generate a first special effect line; In the display page, the first special effect line is displayed. The device according to claim 12, wherein the at least two face key points comprise: a first face key point, a second face key point and a target face key point, the first face key point being the same as the The second face key point is symmetrical with respect to the target face key point, and the target face key point is any face key point on the symmetry axis of the face image. The apparatus according to claim 13, wherein the determining module is further configured to: For each of the track points, according to the first position information of the target face key point and the track position information of the track point, determine the translation vector of the track point pointing to the target face key point; According to the first position information of the first face key point and the second face key point, determine the first rotation matrix and the first scaling matrix of the current face posture in the face image; According to the first rotation matrix, the first scaling matrix and the translation vector, the relative position of the trajectory point relative to the face image is obtained. The device according to claim 14, wherein the first position information and the track position information both include absolute coordinates on the display screen, and the determining module is further configured to: According to the first position information of the first face key point and the second face key point, and the first length, the first rotation matrix is obtained, and the first length is the first face key point to the length of the first vector of the second face key point; According to the reference length of the connecting line between the first face key point and the second face key point, and the first length, the first scaling matrix is obtained, and the reference length is for the face image The first length set by the face in the facing posture. The apparatus according to claim 14, wherein the determining module is further configured to: The relative position is obtained according to the first scaling matrix, the first rotation matrix, the translation vector and a first formula, where the first formula includes:

Wherein, Q represents the relative position, Ms ₁ represents the first scaling matrix, Mr ₁ represents the first rotation matrix,

represents the translation vector. The device according to claim 13, wherein the image special effect processing module is further configured to: For each of the trajectory points, according to the second position information of the first face key point and the second face key point, determine the second rotation matrix and the second rotation matrix of the current face pose in the face image scaling matrix; The first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, and the relative position. The device according to claim 17, wherein the second position information includes absolute coordinates on the display screen, and the image special effect processing module is further configured to: According to the second position information of the first face key point and the second face key point, and the second length, the second rotation matrix is obtained, and the second length is the first face key point point to the length of the second vector of the second face key point; According to the reference length of the line connecting the first face key point and the second face key point, and the second length, the second scaling matrix is obtained, and the reference length is for the face image The second length set by the face in the facing posture. The device according to claim 17, wherein the image special effect processing module is further configured to: According to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, the relative position and the second formula, the first absolute position of the trajectory point is obtained, and the The second formula includes: R=Mr ₂ ·Ms ₂ ·(x _q , y _q ) ^T +(x _c , y _c ) ^T ; Wherein, R represents the first absolute position, Mr ₂ represents the second rotation matrix, Ms ₂ represents the second scaling matrix, (x _q , y _q ) represents the relative position, (x _c , y _{c )} ) represents the second position information of the target face key point, and T represents the transposition process. The device according to any one of claims 12-19, wherein the image special effect processing further comprises: generating a second special effect line symmetrical to the first special effect line according to the first special effect line, where the second special effect line and the first special effect line are left-right symmetrical with respect to the face in the face image; The second special effect line is displayed in the display page. The apparatus according to claim 20, wherein the determining module is further configured to: According to the relative position of each track point relative to the face image, determine the relative position of the symmetry point of each track point relative to the face image, and the symmetry point and the track point take the face as the The benchmark is left and right symmetrical; The image special effect processing module is also used for: Determine the second absolute position of each of the symmetrical points on the display screen according to the second position information of the face key points and the relative positions of each of the symmetrical points; Connecting the symmetrical points located at each of the second absolute positions to generate the second special effect line. The apparatus according to claim 21, wherein the relative position includes relative coordinates relative to the face image, and the determining module is further configured to: Perform positive and negative conversion processing on the coordinate value of the first direction in the relative coordinates of the trajectory point to obtain the processed coordinate value, and the first direction is perpendicular to the symmetry axis of the face image; updating the relative position of the trajectory point, so that the coordinate value of the first direction in the updated relative position is the processed coordinate value; The updated relative position is determined as the relative position of the symmetry point. An electronic device comprising: one or more processors; one or more memories for storing the one or more processor-executable instructions; wherein the one or more processors are configured to execute the executable instructions to implement the following steps: In response to the special effect display instruction, acquiring the movement track input by the user in the display page including the face image; According to the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the movement trajectory, determine the relative position of each trajectory point relative to all the trajectory points. relative position of the face image; The image special effect processing is repeatedly performed, and the image special effect processing includes: Convert the relative position according to the second position information of the key points of the face in the face image displayed on the display page after the current time, to obtain the first position of each track point on the display screen. absolute position; Connect the trajectory points located at each of the first absolute positions to generate a first special effect line; In the display page, the first special effect line is displayed. The electronic device according to claim 23, wherein the at least two face key points include: a first face key point, a second face key point and a target face key point, the first face key point Symmetrical with the second face key point about the target face key point, the target face key point is any face key point on the symmetry axis of the face image. The electronic device according to claim 24, wherein in the face image displayed at the current moment according to the display page, the first position information of at least two face key points, and at least one trajectory in the movement trajectory The track position information of the point, to determine the relative position of each of the track points relative to the face image, including: For each of the track points, according to the first position information of the target face key point and the track position information of the track point, determine the translation vector of the track point pointing to the target face key point; According to the first position information of the first face key point and the second face key point, determine the first rotation matrix and the first scaling matrix of the current face posture in the face image; According to the first rotation matrix, the first scaling matrix and the translation vector, the relative position of the trajectory point relative to the face image is obtained. The electronic device according to claim 25, wherein the first position information and the track position information both include absolute coordinates on the display screen, Determining the first rotation matrix and the first scaling matrix of the current face in the face image according to the first position information of the first face key point and the second face key point, including: According to the first position information of the first face key point and the second face key point, and the first length, the first rotation matrix is obtained, and the first length is the first face key point to the length of the first vector of the second face key point; According to the reference length of the connecting line between the first face key point and the second face key point, and the first length, the first scaling matrix is obtained, and the reference length is for the face image The first length set by the face in the facing posture. The electronic device according to claim 25, wherein the obtaining the relative position of the trajectory point relative to the face image according to the first rotation matrix, the first scaling matrix and the translation vector, comprising: : The relative position is obtained according to the first scaling matrix, the first rotation matrix, the translation vector and a first formula, where the first formula includes:

Wherein, Q represents the relative position, Ms ₁ represents the first scaling matrix, Mr ₁ represents the first rotation matrix,

represents the translation vector. The electronic device according to claim 25, wherein the relative position is converted according to the second position information of the face key point in the face image displayed on the display page after the current time , to obtain the first absolute position of each track point on the display screen, including: For each of the trajectory points, according to the second position information of the first face key point and the second face key point, determine the second rotation matrix and the second rotation matrix of the current face pose in the face image scaling matrix; The first absolute position of the trajectory point is obtained according to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, and the relative position. The electronic device according to claim 28, wherein the second position information includes absolute coordinates on a display screen, and the second position based on the first face key point and the second face key point The position information determines the second rotation matrix and the second scaling matrix of the current face posture in the face image, including: According to the second position information of the first face key point and the second face key point, and the second length, the second rotation matrix is obtained, and the second length is the first face key point point to the length of the second vector of the second face key point; According to the reference length of the line connecting the first face key point and the second face key point, and the second length, the second scaling matrix is obtained, and the reference length is for the face image The second length set by the face in the facing posture. The electronic device according to claim 28, wherein the obtaining of the The first absolute position of the trajectory point, including: According to the second rotation matrix, the second scaling matrix, the second position information of the target face key point, the relative position and the second formula, the first absolute position of the trajectory point is obtained, and the The second formula includes: R=Mr ₂ ·Ms ₂ ·(x _q , y _q ) ^T +(x _c , y _c ) ^T ; Wherein, R represents the first absolute position, Mr ₂ represents the second rotation matrix, Ms ₂ represents the second scaling matrix, (x _q , y _q ) represents the relative position, (x _c , y _{c )} ) represents the second position information of the target face key point, and T represents the transposition process. The electronic device according to any one of claims 23-30, wherein the image special effect processing further comprises: generating a second special effect line symmetrical to the first special effect line according to the first special effect line, where the second special effect line and the first special effect line are left-right symmetrical with respect to the face in the face image; The second special effect line is displayed in the display page. The electronic device of claim 31 , wherein the one or more processors are configured to execute the executable instructions to: According to the relative position of each track point with respect to the face image, the relative position of the symmetry point of each track point with respect to the face image is determined, and the symmetry point and the track point take the face as the The benchmark is left and right symmetrical; The generating a second special effect line symmetrical with the first special effect line according to the first special effect line, comprising: Determine the second absolute position of each of the symmetrical points on the display screen according to the second position information of the face key points and the relative positions of each of the symmetrical points; Connecting the symmetrical points located at each of the second absolute positions to generate the second special effect line. The electronic device according to claim 32, wherein the relative position includes relative coordinates relative to the face image, and the trajectory is determined according to the relative position of each trajectory point relative to the face image. The relative position of the symmetrical point of the point relative to the face image, including: Perform positive and negative conversion processing on the coordinate value of the first direction in the relative coordinates of the trajectory point to obtain the processed coordinate value, and the first direction is perpendicular to the symmetry axis of the face image; updating the relative position of the trajectory point, so that the coordinate value of the first direction in the updated relative position is the processed coordinate value; The updated relative position is determined as the relative position of the symmetry point. A non-volatile computer-readable storage medium, when the instructions in the non-volatile computer-readable storage medium are executed by a processor of an electronic device, the electronic device can implement the following steps: In response to the special effect display instruction, acquiring the movement track input by the user in the display page including the face image; According to the first position information of at least two face key points in the face image displayed on the display page at the current moment, and the trajectory position information of at least one trajectory point in the movement trajectory, determine the relative position of each trajectory point relative to all the trajectory points. relative position of the face image; The image special effect processing is repeatedly performed, and the image special effect processing includes: Convert the relative position according to the second position information of the face key points in the face image displayed on the display page after the current time, to obtain the first position of each track point on the display screen. absolute position; Connect the trajectory points located at each of the first absolute positions to generate a first special effect line; In the display page, the first special effect line is displayed. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the image processing method according to any one of claims 1-11 is implemented.

Images