[go: up one dir, main page]

WO2023066120A1 - Procédé et appareil de traitement d'image, dispositif électronique et support de stockage - Google Patents

Procédé et appareil de traitement d'image, dispositif électronique et support de stockage Download PDF

Info

Publication number
WO2023066120A1
WO2023066120A1 PCT/CN2022/125036 CN2022125036W WO2023066120A1 WO 2023066120 A1 WO2023066120 A1 WO 2023066120A1 CN 2022125036 W CN2022125036 W CN 2022125036W WO 2023066120 A1 WO2023066120 A1 WO 2023066120A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
adjustment
point
dimensional
processed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/125036
Other languages
English (en)
Chinese (zh)
Inventor
杨瑞健
陶然
赵代平
许亲亲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Sensetime Intelligent Technology Co Ltd
Original Assignee
Shanghai Sensetime Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Sensetime Intelligent Technology Co Ltd filed Critical Shanghai Sensetime Intelligent Technology Co Ltd
Publication of WO2023066120A1 publication Critical patent/WO2023066120A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks

Definitions

  • the present disclosure relates to the technical field of image processing, and in particular, to an image processing method, device, electronic equipment, and storage medium.
  • an image processing method including: acquiring an image to be processed, wherein the image to be processed includes a target object; extracting 3D key points of the target object, and determining the 3D A first correspondence between key points and vertices in the three-dimensional adjustment model; determining an adjustment result of the three-dimensional key points according to an adjustment instruction, preset deformation data of vertices in the three-dimensional adjustment model, and the first correspondence; A target image is determined according to the adjustment result of the three-dimensional key points and the image to be processed.
  • an image processing device including: an acquisition module, configured to acquire an image to be processed, wherein the image to be processed includes a target object; a first corresponding module, configured to extract the The three-dimensional key points of the target object, and determine the first corresponding relationship between the three-dimensional key points and the vertices in the three-dimensional adjustment model; the adjustment module is used to adjust according to the adjustment instruction, the preset deformation data of the vertices in the three-dimensional adjustment model and The first correspondence is used to determine an adjustment result of the 3D key point; a target module is configured to determine a target image according to the adjustment result of the 3D key point and the image to be processed.
  • an electronic device the device includes a memory and a processor, the memory is used to store computer instructions that can be run on the processor, and the processor is used to execute the The computer instructions implement the method described in the first aspect.
  • a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in the first aspect is implemented.
  • a computer program product includes a computer program, and when the program is executed by a processor, the method described in the first aspect is implemented.
  • FIG. 1 is a flowchart of an image processing method according to an embodiment of the present disclosure
  • Fig. 2 is a schematic diagram of an interactive interface shown according to an embodiment of the present disclosure
  • Fig. 3 is a schematic diagram of a face standard model shown according to an embodiment of the present disclosure.
  • Fig. 4 is a flow chart of determining a first correspondence according to an embodiment of the present disclosure
  • Fig. 5 is a flowchart showing a method of determining a target image according to an embodiment of the present disclosure
  • FIG. 6 is a flowchart of an image processing method according to another embodiment of the present disclosure.
  • Fig. 7 is a schematic diagram of an integrated model shown according to an embodiment of the present disclosure.
  • Fig. 8 is a schematic structural diagram of an image processing device according to an embodiment of the present disclosure.
  • Fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.
  • first, second, third, etc. may be used in the present disclosure to describe various information, the information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of the present disclosure, first information may also be called second information, and similarly, second information may also be called first information. Depending on the context, the word “if” as used herein may be interpreted as “at” or “when” or “in response to a determination.”
  • the product applying the disclosed technical solution has clearly informed the user of the information processing rules and obtained the user's independent consent before processing the user information.
  • the product applying the disclosed technical solution has obtained the separate consent of the user before processing the sensitive user information, and at the same time meets the requirement of "express consent". For example, at the user information collection device such as a camera, set a clear and prominent sign to inform that the user information has entered the collection range, and the user information will be collected.
  • the user information processing rules may include user Information processor, user information processing purpose, processing method, type of user information processed and other information.
  • At least one embodiment of the present disclosure provides an image processing method. Please refer to FIG. 1 , which shows the flow of the method, including steps S101 to S104.
  • the method can be used to process the image to be processed, for example, the target object in the image to be processed can be adjusted, and the target object can be, for example, at least one of the following objects in the image to be processed: face, hand, limb, adjusted
  • the items can be shape, size, position, etc.
  • the facial information of the face that is, the shape, size, proportion, etc. of the facial features, can be adjusted, so as to complete the enhancement processing such as face beautification, micro-shaping, and body shaping.
  • the image to be processed may be an image captured by the image acquisition device, or a frame of image in a video recorded by the image acquisition device. It can be understood that, by using the method provided by the embodiment of the present disclosure to process each frame of image in the video recorded by the image acquisition device, the processing of the video can be completed.
  • the method can be executed by electronic devices such as terminal equipment or servers, and the terminal equipment can be user equipment (User Equipment, UE), mobile equipment, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA) handheld device, computing device, vehicle-mounted device, wearable device, etc.
  • the method can be realized by calling a computer-readable instruction stored in a memory by a processor.
  • the method may be executed by a server, and the server may be a local server, a cloud server, or the like.
  • step S101 an image to be processed is acquired, wherein the image to be processed includes a target object.
  • the image to be processed may be an image taken by an image acquisition device, or a frame image in a video recorded by the image acquisition device, wherein the image acquisition device may be an electronic device with an image acquisition function such as a mobile phone or a camera.
  • the target object is the image of a specific object in the real scene in the image, for example, it can be the face in the image to be processed (that is, the image of the face in the image), the hand (that is, the hand in the image The resulting image), etc.
  • the image captured by the image acquisition device can be identified. If the image contains the target object, the image is acquired as the image to be processed. If the image does not contain the target object, the image is not regarded as the image to be processed; correspondingly, the The video recorded by the image acquisition device is identified. If a frame of image contains the target object, the frame of image is acquired as the image to be processed. If a frame of image does not contain the target object, the frame of image is not regarded as the image to be processed.
  • the user uses a mobile phone to record a video.
  • a face sometimes appears in the screen, and sometimes there is no face.
  • Image frames with no faces in the video frame are not treated as images to be processed for beautification processing, thereby ensuring real-time beautification processing of faces in the video screen, and reducing processor load when there are no faces in the video screen , save energy consumption, and avoid mishandling of video images.
  • step S102 the 3D key points of the target object are extracted, and a first corresponding relationship between the 3D key points and vertices in the 3D adjustment model is determined.
  • the depth image corresponding to the image to be processed can be obtained, and the image after the image to be processed is aligned with the depth image can be input into the pre-trained deep neural network.
  • Perform 3D keypoint detection While the image acquisition device acquires the image to be processed, it can start the structured light sensor of the camera of the image acquisition device to acquire the depth image, or use Time of Flight (ToF) sensing technology to acquire the depth image.
  • TOF Time of Flight
  • each detected 3D key point has an identifier.
  • the identifier of each 3D key point may be a sequence identifier.
  • the detection can be performed according to the characteristics of the target object or detection requirements, that is, whether to detect a certain position of the target object, and to detect several 3D key points can be preset in advance.
  • three-dimensional key point detection can be performed on facial features and other key positions. These key positions have signs such as eyes, mouth, eyebrows, nose, etc., so the detected three-dimensional key points is identified.
  • the three-dimensional adjustment model is used to represent the adjustment method of each adjustment item of the target object.
  • the three-dimensional adjustment model of the face may include a beauty model, which is used to represent various beauty items (such as thin face, big eyes, small mouth, etc.) way of beauty.
  • the 3D adjustment model can be composed of multiple vertices and the mesh structure between the vertices.
  • the vertices of the 3D adjustment model can also correspond to the 3D key points of the target object, and the multiple vertices of the 3D adjustment model are also identified.
  • the 3D adjustment Multiple vertices of the model have order, and the identification of each vertex can be order identification.
  • the order of the multiple vertices may be the same as or different from the order of the three-dimensional key points detected above.
  • the relationship between the 3D key point and the vertices in the 3D adjustment model is established.
  • the first correspondence is to correspond the 3D key points at the same key position with the vertices in the 3D adjustment model.
  • step S103 the adjustment result of the three-dimensional key points is determined according to the adjustment instruction, the preset deformation data of vertices in the three-dimensional adjustment model, and the first corresponding relationship.
  • the three-dimensional adjustment model may also store preset deformation data of each vertex, such as displacement data of the vertex when each adjustment item is performed.
  • preset deformation data can be recorded in the form of an array, and an array is created for each vertex, and the three-dimensional coordinates of the vertex are first recorded in the array, and then the displacement data of the vertex under each adjustment item is sequentially recorded.
  • the coordinate information of the 3D key points corresponding to the vertices in the 3D adjustment model can be converted into the coordinate system of the 3D adjustment model, and the converted 3D key points The coordinates of the vertex are added to the array of the vertex, so as to facilitate the subsequent calculation when adjusting the 3D key points.
  • the adjustment instruction can be generated according to the user's operation, for example, if the user selects at least one adjustment item, an adjustment instruction for the at least one adjustment item is generated; the adjustment instruction can also be automatically generated by the terminal executing the method.
  • the target object is a face
  • the user may select at least one adjustment item for the face, so as to generate an adjustment instruction corresponding to the at least one adjustment item.
  • at least one adjustment item may be set in advance, and then an adjustment instruction corresponding to the at least one adjustment item is automatically generated each time an image to be processed is acquired.
  • the adjustment process of the three-dimensional key points includes the position adjustment process of the three-dimensional key points, so the adjustment result of the three-dimensional key points includes the adjusted three-dimensional key points.
  • the adjustment item of the target object may be determined according to the adjustment instruction, and the preset deformation data of the vertices in the three-dimensional adjustment model corresponding to the adjustment item may be obtained.
  • the first deformation data According to the first deformation data, the three-dimensional key point corresponding to at least one vertex in the three-dimensional adjustment model is displaced, so as to obtain the adjustment result of the three-dimensional key point.
  • the adjustment items include adjustment position and adjustment type.
  • the adjustment position refers to the local position of the target object, such as a facial feature in the face
  • the adjustment type refers to the expected adjustment result, such as reducing, increasing, and rounding.
  • the target object is a face
  • the adjustment position is eyes
  • the adjustment type is enlarged
  • the adjustment item is small mouth
  • the adjustment position is mouth
  • the adjustment type is reduced, etc.
  • FIG. 2 it has shown the interactive interface of a face adjustment, has adjustment items such as thin face, big eyes, small head, rhinoplasty and small mouth etc. in this interface, after the user selects at least one adjustment item among them, then generate Contains the adjustment command for this adjustment item.
  • one adjustment item is determined, and if there are multiple adjustment items in the adjustment instruction, multiple adjustment items are determined.
  • the first deformation data includes displacement data of each vertex of the three-dimensional adjustment model under the adjustment item.
  • Each vertex can be traversed to obtain the deformation data of each vertex corresponding to the adjustment item in the adjustment instruction, that is, the displacement data of all vertices under the adjustment item.
  • the adjustment instruction includes, for example, the big eye item 202 in FIG. 2
  • traverse each vertex corresponding to the adjustment item and obtain the deformation data of the eye vertices (for example, the number of eye vertices of E1, E2, E3, and E4) in the three-dimensional adjustment model of the big eye project
  • the three-dimensional adjustment Other vertices in the model such as mouth vertices and nose vertices, do not have deformation data.
  • each vertex stores the displacement data under each adjustment item in the form of an array
  • the displacement data can be obtained according to the corresponding position of the adjustment item to the array.
  • the array of E1 is [0 ,1,0.2,0.1,0]
  • the first to fifth data positions in the array are the displacement data 0 of E1 under the thin face item 201, the displacement data 1 of E1 under the big eye item 202, and the small head
  • the displacement data of E1 under item 203 is 0.2
  • the displacement data of E1 under item 204 of rhinoplasty is 0.1
  • the displacement data of E1 under item 205 of small mouth is 0, then when the adjustment item is big eyes, it can be directly obtained from the second part of the E1 array.
  • the displacement data 0.2 corresponding to the big-eye item 202 is obtained from the position.
  • the displacement data corresponding to the big-eye item can be obtained from the second positions of the E2, E3, and E4 arrays respectively.
  • the deformation data corresponding to the adjustment item is obtained directly.
  • the deformation data (displacement data) of vertices E1-E4 corresponding to the big eye item are directly obtained.
  • the deformation data corresponding to each adjustment item may be acquired.
  • the three-dimensional adjustment model also stores displacement data corresponding to various combinations of adjustment items, so when there are multiple adjustment items in the adjustment instruction, multiple adjustment items can be directly obtained.
  • the displacement data corresponding to the combination of adjustment items is obtained directly.
  • the deformation data corresponding to the face-lifting item 201 is obtained directly;
  • the deformation data corresponding to the deformation data of the large-eye item 202 and the deformation data corresponding to the combination of the thin-face item 201 and the large-eye item 202 can also be obtained.
  • the adjustment command includes the adjustment items, it can also include the adjustment parameter values of each adjustment item.
  • the adjustment parameter value can be the adjustment degree of the adjustment item. Different adjustment degrees can correspond to different deformation data. For example, when the adjustment degree is 100%, the deformation data is the maximum deformation data corresponding to the adjustment item, and the adjustment degree is 60%. , the deformation data is 60% of the maximum deformation data corresponding to the adjustment item.
  • the adjustment of the adjustment items can be further determined according to the adjustment instruction. parameter value, and then adjust the first deformation data according to the adjusted parameter value.
  • the degree of the adjustment item can be adjusted by adjusting the parameter value, which increases the flexibility and diversity of the adjustment.
  • the coordinates of the three-dimensional key points corresponding to the vertices of the three-dimensional adjustment model and the displacement data corresponding to the vertices of the three-dimensional adjustment model can be calculated to obtain the adjustment result of the three-dimensional key points. Under one or more adjustment items, if the displacement data of at least one vertex is not 0, the 3D key points corresponding to these vertices will be displaced after the operation.
  • the preset deformation data is recorded in the form of an array, and after the first corresponding relationship is determined, the coordinate information of the 3D key point corresponding to the vertex is converted into the coordinate system of the 3D adjustment model, and the converted 3D When the coordinate information of the key point is added to the array of the vertex, the coordinate and displacement data of the three-dimensional key point can be directly extracted from the array of each vertex.
  • each displacement data can be used in sequence according to the three-dimensional key point corresponding to the vertex. Coordinates are calculated.
  • the displacement data of the eye vertex E1 is [0.2,0.1,1]
  • you can follow the order in the adjustment command such as small head item, nose augmentation
  • the displacement data of E1 corresponding to the project and the big-eye project can be used to calculate the coordinates of the three-dimensional key points corresponding to E1; or the displacement data corresponding to the big-eye project, small-head project and nose augmentation project can also be calculated according to the order in the E1 array.
  • the coordinates of the 3D key points corresponding to E1 are calculated.
  • the positions of the three-dimensional key points are adjusted, and the three-dimensional key points at the new positions are the adjusted three-dimensional key points in the adjustment result of the three-dimensional key points.
  • step S104 a target image is determined according to the adjustment result of the 3D key points and the image to be processed.
  • the same adjustment is performed on the corresponding position of the image to be processed, that is, the adjustment result is mapped to the image to be processed, thereby completing the adjustment of the target object in the image to be processed.
  • the target object is a facial image
  • the beautification process on the facial image can be completed in this step.
  • the vertices in the three-dimensional adjustment model by acquiring the image to be processed, and extracting the three-dimensional key points of the target object in the image to be processed, determining the first corresponding relationship between the three-dimensional key points and the vertices in the three-dimensional adjustment model, according to the adjustment instruction, the vertices in the three-dimensional adjustment model.
  • the preset deformation data and the first corresponding relationship determine the adjustment result of the three-dimensional key point, and determine the target image according to the adjustment result of the three-dimensional key point and the image to be processed.
  • the adjustment of the target object is to adjust the 3D key points according to the adjustment instruction and the preset deformation data, so the adjustment process is more precise and controllable , and the three-dimensional key points are extracted from the image to be processed, and finally the adjustment result obtained by adjusting the three-dimensional key points is returned to the image to be processed, so the adjustment effect of the target object in the obtained target image is natural and real, which improves the user satisfaction.
  • the image processing method further includes: acquiring a second corresponding relationship between vertices in the three-dimensional adjustment model and vertices in the three-dimensional standard model.
  • the three-dimensional standard model is the standard model of the target object, which is composed of multiple vertices and meshes between the vertices.
  • the three-dimensional standard model may be a standard facial model as shown in FIG. 3 . If the 3D key points are extracted from the 3D standard model, all vertices of the 3D standard model can be obtained.
  • the three-dimensional adjustment model is used to characterize the specific adjustment manner of at least one adjustment item of the three-dimensional standard model, that is, the three-dimensional adjustment model includes deformation data of each vertex of the three-dimensional standard model in at least one adjustment item.
  • the 3D adjustment model has a dedicated model coordinate system.
  • the coordinates of each vertex in the 3D adjustment model are the coordinates in the coordinate system of the 3D adjustment model.
  • the 3D adjustment model can also use the coordinate transformation matrix to coordinate the coordinates of the 3D adjustment model in the coordinate system Transform to the world coordinate system.
  • the three-dimensional adjustment model may be obtained by performing preset deformations respectively corresponding to at least one adjustment item on the three-dimensional standard model.
  • the coordinates of the vertices in the three-dimensional adjustment model and the coordinates of the vertices in the three-dimensional standard model can be converted into the same coordinate system, for example, the coordinates of the vertices in the three-dimensional standard model are coordinates in the world coordinate system, then
  • the coordinate transformation matrix of the three-dimensional adjustment model can be multiplied by the coordinates of the vertices in the three-dimensional adjustment model, so as to convert the coordinates of the vertices of the three-dimensional adjustment model to the world coordinate system.
  • the coordinates of the vertices in the converted three-dimensional adjustment model and the three-dimensional standard model The coordinates of vertices are all in the world coordinate system.
  • the distance threshold is to determine whether the two vertices are corresponding, that is, if the Euclidean distance is less than the distance threshold, then the two points are corresponding, otherwise they are not.
  • the first corresponding relationship between the 3D key points and the vertices in the 3D adjustment model may be determined as shown in FIG. 4 , including steps S401 to S402.
  • step S401 according to the identification of the 3D key point and the identification of the vertex in the 3D standard model, a third corresponding relationship between the 3D key point and the vertex in the 3D standard model is determined.
  • the setting logic of the key points of the standard object and the target object in the 3D standard model is the same, for example, they all include eyebrows, eyes, nose, mouth , therefore, when the position of the vertex in the target object in the 3D standard model generated based on the standard face image is the same as the position of the 3D key point in the target object, then the vertex in the 3D standard model and the identity of the 3D key point are the same , so the same three-dimensional key point and the vertices of the three-dimensional standard model are determined as pairs of vertices corresponding to each other, and the third corresponding relationship is obtained.
  • the order of the vertices of the three-dimensional standard model is consistent with the order of the three-dimensional key points of the target object, and the order of the same three-dimensional key points and the vertices of the three-dimensional standard model can be determined as a pair of vertices corresponding to each other, and the described third correspondence.
  • step S402 the first correspondence is determined according to the second correspondence and the third correspondence.
  • the vertices of the 3D adjustment model corresponding to the same vertex of the 3D standard model and the 3D key points are determined as corresponding to each other.
  • the 3D key is determined according to the third corresponding relationship between the 3D key point and each vertex of the 3D standard model.
  • the first corresponding relationship between the point and each vertex of the three-dimensional adjustment model can also be directly used to determine the first corresponding relationship between the 3D key points and the vertices of the 3D adjustment model, which is simple, convenient, accurate and reliable, and avoids the For the 3D key points of the target object of each image to be processed, complex operations such as Euclidean distance calculation are performed.
  • the target image may be determined according to the adjustment result of the three-dimensional key points and the image to be processed as shown in FIG. 5 , including steps S501 to S502.
  • step S501 a first grid is constructed between the adjustment results of the three-dimensional key points according to a pre-established first topology structure.
  • the pre-established first topology refers to the connection relationship of the vertices in the grid among the vertices of the 3D standard model, for example, which other vertices need to be connected to each vertex of the 3D standard model.
  • the first topological structure is recorded with the identification of vertices.
  • the identification of the three-dimensional key point corresponds to the identification of the vertices of the three-dimensional standard model one by one, and the third corresponding relationship between the three-dimensional key point and the vertices of the three-dimensional standard model is established in step S401, when at least one three-dimensional key point is displaced Finally, the identification of each 3D key point is still clear, and the corresponding relationship between each 3D key point and each vertex of the 3D standard model is still clear, and then the adjustment results of different 3D key points can be connected according to the first topology structure to construct the second a grid.
  • a target model of the target object is formed.
  • step S502 according to the pixel information of the projection point of the 3D key point on the image to be processed, the coordinate information of the projection point of the adjustment result of the 3D key point on the image to be processed, and the first A grid, rendering the adjustment result of the target object on the image to be processed to obtain a target image.
  • the pixel information of the projection point of the three-dimensional key point on the image to be processed can be extracted as the pixel information of the three-dimensional key point, wherein the pixel information includes, for example, RGB values, etc.
  • the three-dimensional key point is on the image to be processed
  • the pixel information of the projection point of the three-dimensional key point is unchanged before and after the three-dimensional key point adjustment, so the pixel information of the projection point of the three-dimensional key point on the image to be processed can also be used as the pixel of the three-dimensional key point adjustment result information; according to the pixel information of the adjustment result of the three-dimensional key point, render the first grid to obtain the pixel information in the first grid; according to the adjustment result of the three-dimensional key point in the image to be processed
  • the coordinate information of the projected point on the above, and the pixel information of the three-dimensional key point and the pixel information in the first grid are projected onto the image to be processed to obtain the target image. For example, if the target object is a facial image, then the
  • certain positions of the target object in the target image are adjusted, for example, the eyes in the face in the image to be processed become larger, the mouth becomes smaller, etc.
  • These adjustments change the pixels in the image, such as eyes When it becomes larger, the pixel points of the eye part are displaced, the number increases, and the shape composed of all the pixel points of the eye changes.
  • the pixel points are not directly adjusted on the two-dimensional image to be processed, but the pixel points on the image to be processed are adjusted by adjusting the three-dimensional key points extracted from the two-dimensional image to be processed. .
  • the 3D key point corresponds to its projection point on the image to be processed
  • the adjustment result of the 3D key point can be used to represent the adjustment result of the 3D key point corresponding to the projection point on the image to be processed, that is, the projection point of the projection point
  • the position adjustment of the pixel point can be, for example, extracting the pixel information of the projection point of the three-dimensional key point on the image to be processed, and re-projecting the pixel information to the adjustment result of the three-dimensional key point on the projection point of the image to be processed.
  • the first grid includes a plurality of closed sub-grids, and these sub-grids are surrounded by at least three three-dimensional key points.
  • the pixel information in each sub-grid can be rendered as the Pixel information of three 3D keypoints.
  • the image to be processed can be rendered on the output image first, for example, through an image copy operation of a rendering API such as OpenGL, the image to be processed can be copied to the output image, that is, as the background of the target image, wherein the output image includes The pixel information of the three-dimensional key points and the projection of the pixel information in the first grid on the two-dimensional plane; the image to be processed is used as the input image of the rendering process, and the coordinates of the projection points of each three-dimensional key point on the image to be processed are used as Input the texture sampling coordinates of the image, so that the pixel information of the projection point of the 3D key point on the image to be processed can be extracted.
  • a rendering API such as OpenGL
  • the pixel information of the 3D key point can be used as the pixel information of the adjustment result of the 3D key point; through the model transformation matrix of the 3D adjustment model, the coordinates of the adjustment result of the 3D key point are converted to the world coordinate system, and then through the transformation matrix from the world coordinate system to the clipping space, Convert the coordinates of the adjustment results of the 3D key points in the world coordinate system to the clipping coordinate system, and send them to the subsequent fragment rendering stage, that is, render the pixel information in the first grid according to the pixel information of the adjustment results of the 3D key points, And rendering the adjustment result of the three-dimensional key point and the pixel information in the first grid on the output image, so as to render the adjustment result of the target object on the image to be processed, and obtain the target image.
  • the first grid is constructed between the adjustment results of each three-dimensional key point to form the target model of the target object; and then the pixel information of the adjustment result of the three-dimensional key point is extracted, and the first grid
  • the rendering of pixel information is to paste pixel information on the target model, and finally render the target model with pixel information to the output image to obtain the final target image.
  • the image may also be processed in the manner shown in FIG. 6 , including steps S601 to S602.
  • step S601 generate a 3D extended model surrounding the 3D key point, wherein the vertices in the 3D extended model include the adjustment result of the first edge point and the second edge point obtained based on the extension of the first edge point , the first edge point is a three-dimensional key point corresponding to the boundary of the target object.
  • the first edge point corresponding to the boundary of the target object among the three-dimensional key points may be obtained first.
  • the projection point of the three-dimensional key point in the image to be processed is obtained; according to the projection point of the target object and the three-dimensional key point in the image to be processed, determine the target object A projection point on the boundary: determining a 3D key point corresponding to the projection point on the boundary of the target object as the first edge point.
  • a plurality of straight lines can be set in the preset direction in the image to be processed, for example, a plurality of horizontal straight lines can be set in advance, and then projected on the basis of the three-dimensional key points The points on the plurality of straight lines are determined to determine the projected points on the boundary of the target object.
  • this step it may be determined on the extension line of the line between the central point of all the three-dimensional key points (for example, the central point of the target object, for example, located on the nose) and the first edge point, and determine the connection with the first edge point.
  • a point at which the edge points are separated by a preset distance is a second edge point corresponding to the first edge point.
  • the calculation may be performed according to the coordinate information of each 3D key point, so as to obtain the center point among the multiple 3D key points, that is, the 3D key point located at the center of the target object. Then construct a straight line between the center point and each edge point of the target object, and further extend the preset distance outward on the straight line (the distance can be preset in advance and can be set according to the size of the target object, for example, for the target object 1/10 of the width), then the vertex of the extension line is the second edge point corresponding to the edge point.
  • a second grid may be constructed between the adjustment result of the first edge point and the second edge point according to the second pre-established topological structure, so as to obtain the three-dimensional extended model.
  • the pre-established second topology refers to the connection relationship of points in the grid between multiple first edge points and multiple second edge points, for example, which other points each point needs to be connected to.
  • all the first edge points are connected sequentially to form the inner circle of the 3D extended model; all the second edge points are connected sequentially to form the outer circle of the 3D extended model; then each first edge point and its corresponding second
  • the edge points are connected with a line, and the first edge point and the second edge point next to the corresponding second edge point are also connected with a line, thereby forming a second grid in the three-dimensional extended model.
  • the second topology is recorded in the order of the first edge point and the second edge point, so after the first edge point is adjusted, the second grid can still be constructed according to the topology.
  • the three-dimensional extended model is a model that surrounds the three-dimensional key points located on the contour of the target object.
  • the outer circle of the model ie, the circle surrounded by the second edge point
  • the inner circle of the model ie, The circle surrounded by the first edge point
  • the target model of the target object is formed.
  • a 3D extended model is added around the target model, and the target model and the 3D extended model pass through the first edge point
  • FIG. 7 shows a schematic diagram of an integrated model when the target object is a face.
  • step S602 the 3D extended model is rendered on the image to be processed according to the 3D extended model and the image to be processed.
  • the pixel information of the projection point of the first edge point on the image to be processed can be extracted. Since the pixel information of the first edge point is unchanged before and after the adjustment of the first edge point, the first edge point is The pixel information of the projection point on the image to be processed may also be used as the pixel information of the adjustment result of the first edge point; the pixel information of the projection point of the second edge point on the image to be processed is extracted as The pixel information of the second edge point; according to the pixel information of the adjustment result of the first edge point and the pixel information of the second edge point, render the second grid to obtain the second grid pixel information; according to the adjustment result of the first edge point and the coordinate information of the projection point of the second edge point on the image to be processed, the pixel information of the adjustment result of the first edge point, the The pixel information of the second edge point and the pixel information in the second grid are projected onto the image to be processed.
  • step S602 can be executed with reference to the operation in step S502.
  • a three-dimensional extended model can be constructed around the target model according to the method provided in this embodiment to obtain a comprehensive model, and then the comprehensive model can be operated according to the operation method introduced in step S502, so that the The target object and surrounding images are projected onto the image to be processed to obtain the target image.
  • the three-dimensional extension by determining the second edge points surrounding the target model composed of three-dimensional key points, and further determining the three-dimensional extension model, the three-dimensional extension The adjusted results of the model are projected onto the image to be processed. Since the outer ring of the 3D extended model has not been adjusted, the connection between the area corresponding to the 3D extended model and the surrounding area in the target image is relatively natural, and there is no abrupt connection; and although the inner ring of the 3D extended model has been adjusted Yes, but the pixel information in the second grid in the 3D extended model is rendered, so the connection between the area corresponding to the 3D extended model in the target image and the area corresponding to the target object is relatively natural, and there will be no abrupt connection .
  • the outside of the integrated model including the target model and the 3D extended model has not been adjusted, so the edge area of the entire integrated model on the target image is the same as the corresponding area on the image to be processed; while the inside of the integrated model (target model) After adjustment, the adjusted target model can overcome the uncoordinated and unnatural problems caused by the adjustment because the pixel information of the pixels in each sub-grid is the same during the rendering process, so this part (comprehensive model The corresponding region on the target object is also natural.
  • the vertices in the three-dimensional adjustment model and the vertices in the three-dimensional standard model all include, for example, the identification of the reflection order, and each vertex of the three-dimensional adjustment model also records an array representing preset deformation data corresponding to different adjustment items; extraction to be processed
  • the three-dimensional key points of the face in the image wherein each three-dimensional key point includes, for example, an identification reflecting the order, and a third corresponding relationship is established between the three-dimensional key points with the same identification and the vertices in the three-dimensional standard model, according to the second corresponding relationship and the third
  • the corresponding relationship is to establish a first corresponding relationship between the 3D key point and the vertex in the 3D adjustment model
  • At least one 3D key point is moved according to currently set adjustment items (such as big eyes, thin face, etc.) and adjustment parameter values (ie, adjustment degree), that is, deformation data of vertices in the 3D adjustment model.
  • adjustment items such as big eyes, thin face, etc.
  • adjustment parameter values ie, adjustment degree
  • the array of each vertex in the three-dimensional adjustment model can be traversed, and the deformation data of the vertex under the adjustment item is multiplied by the adjustment parameter value to obtain the displacement data of the vertex, and the displacement data of the vertex is used to compare
  • the three-dimensional key points corresponding to the array are displaced, and the displacement results of the three-dimensional key points are obtained. Then construct the first grid between the displacement results of the three-dimensional key points, and connect the first grid and the second grid as a whole through the first edge point to obtain a comprehensive model.
  • the image to be processed is pasted on the integrated model, and then the integrated model is rendered on the image to be processed to obtain the target image for presenting the face adjustment effect.
  • the image to be processed can be rendered on the output image first, for example, the image to be processed can be copied to the output image through an image copy operation of a rendering API such as OpenGL; then the image to be processed can be used as the input image of the rendering process , the coordinates of the projection points of each 3D key point on the image to be processed are taken as the texture sampling coordinates of the input image; the coordinates of the adjustment results of the 3D key points are converted to the world coordinate system through the model transformation matrix of the 3D adjustment model, and the world The transformation matrix from the coordinate system to the clipping space converts the coordinates of the adjustment results of the 3D key points in the world coordinate system to the clipping coordinate system, and sends them to the subsequent fragment rendering stage to realize the presentation of the real-time 3D adjustment effect.
  • a rendering API such as OpenGL
  • an image processing device is provided. Please refer to FIG.
  • the processed image includes the target object; the first correspondence module 802 is used to extract the 3D key points of the target object, and determine the first correspondence between the 3D key points and the vertices in the 3D adjustment model; the adjustment module 803 uses Determine the adjustment result of the 3D key point according to the adjustment instruction, the preset deformation data of the vertices in the 3D adjustment model and the first corresponding relationship; the target module 804 is used to determine the adjustment result of the 3D key point according to the adjustment result and the image to be processed to determine a target image.
  • a second corresponding module is also included, configured to: obtain a second corresponding relationship between the vertices in the 3D adjustment model and the vertices in the 3D standard model; the first corresponding module is configured to: According to the identification of the 3D key point and the identification of the vertex in the 3D standard model, determine the third corresponding relationship between the 3D key point and the vertex in the 3D standard model; according to the second corresponding relationship and the vertex in the 3D standard model The third corresponding relationship is determined, and the first corresponding relationship is determined.
  • the second corresponding module is used to: convert the coordinates of vertices in the three-dimensional adjustment model and the coordinates of vertices in the three-dimensional standard model into the same coordinate system; according to the The Euclidean distance between each vertex in the three-dimensional adjustment model and each vertex in the three-dimensional standard model is used to determine the second corresponding relationship.
  • the first corresponding module is configured to determine the 3D key point and the vertex in the 3D standard model according to the identification of the 3D key point and the identification of the vertex in the 3D standard model For the third corresponding relationship, it is used to: determine the same 3D key point and the vertices of the 3D standard model as a pair of vertices corresponding to each other, so as to obtain the third corresponding relationship.
  • the adjustment module is configured to: determine the adjustment item of the target object according to the adjustment instruction; obtain the preset deformation data of the vertex in the three-dimensional adjustment model, which corresponds to the adjustment item First deformation data: Displacing the 3D key point corresponding to at least one vertex in the 3D adjustment model according to the first deformation data, to obtain an adjustment result of the 3D key point.
  • the adjustment module is further configured to: determine the adjustment parameter value of the adjustment item according to the adjustment instruction before obtaining the adjustment result of the three-dimensional key point; The first deformation data is adjusted.
  • the target module is configured to: construct a first grid between the adjustment results of the 3D key points according to a pre-established first topology structure; The pixel information of the projection point on the image to be processed, the coordinate information of the projection point of the adjustment result of the three-dimensional key point on the image to be processed, and the first grid, and render the image on the image to be processed The adjustment result of the target object is obtained to obtain the target image.
  • the target module is further configured to: extract the pixel information of the projection point of the 3D key point on the image to be processed, as the pixel information of the adjustment result of the 3D key point; Rendering the first grid according to the pixel information of the adjustment result of the three-dimensional key point to obtain pixel information in the first grid; according to the adjustment result of the three-dimensional key point on the image to be processed Projecting the coordinate information of the point, projecting the pixel information of the three-dimensional key point and the pixel information in the first grid onto the image to be processed to obtain the target image.
  • an extension module configured to: generate a 3D extended model surrounding the 3D key point, wherein the vertices in the 3D extended model include the adjustment result of the first edge point and based on the The second edge point obtained by extending the first edge point, the first edge point is a three-dimensional key point corresponding to the boundary of the target object; according to the three-dimensional extended model and the image to be processed, in the to-be-processed The three-dimensional extended model is rendered on the image.
  • the expansion module is further configured to: obtain the first edge point; on an extension line of a line connecting the center point of the three-dimensional key point and the first edge point, Determining a point at a preset distance from the first edge point as the second edge point; according to a pre-established second topology, between the adjustment result of the first edge point and the second edge point Construct the second grid to obtain the 3D extended model.
  • the extension module is further configured to: obtain the projection point of the 3D key point in the image to be processed; according to the target object and the 3D key point in the image to be processed The projection point in the image is determined as the projection point on the boundary of the target object; and the three-dimensional key point corresponding to the projection point on the boundary of the target object is determined as the first edge point.
  • the expansion module is further configured to: set a plurality of straight lines in preset directions in the image to be processed; points, determine the projected points that lie on the boundary of the target object.
  • the extension module is further configured to: extract pixel information of a projection point of the first edge point on the image to be processed, as a pixel of the adjustment result of the first edge point information, and extract the pixel information of the projection point of the second edge point on the image to be processed as the pixel information of the second edge point; according to the pixel information of the adjustment result of the first edge point and the pixel information of the second edge point, render the second grid, and obtain pixel information in the second grid; according to the adjustment result of the first edge point and the second edge point in the Processing the coordinate information of the projection point on the image, projecting the pixel information of the adjustment result of the first edge point, the pixel information of the second edge point, and the pixel information in the second grid to the waiting processing image.
  • the target object includes at least one of the following objects in the image to be processed: face, hand, and limb.
  • At least one embodiment of the present disclosure provides a device. Please refer to FIG. 9 , which shows the structure of the device.
  • the device includes a memory 904, a processor 901, and a network interface 903. Coupled with 902, the memory is used to store computer instructions that can be run on a processor, and the processor is used to process the image based on the method described in any one of the first aspect when executing the computer instructions.
  • At least one embodiment of the present disclosure provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the method described in any one of the first aspect is implemented.
  • At least one embodiment of the present disclosure provides a computer program product, the product includes a computer program, and when the computer program is executed by a processor, the method described in any one of the first aspect is implemented
  • This disclosure relates to the field of augmented reality.
  • acquiring the image information of the target object in the real environment and then using various visual correlation algorithms to detect or identify the relevant features, states and attributes of the target object, and thus obtain the image information that matches the specific application.
  • AR effect combining virtual and reality.
  • the target object may involve faces, limbs, gestures, actions, etc. related to the human body, or markers and markers related to objects, or sand tables, display areas or display items related to venues or places.
  • Vision-related algorithms can involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, object key point extraction and tracking, object pose or depth detection, etc.
  • Specific applications can not only involve interactive scenes such as guided tours, navigation, explanations, reconstructions, virtual effect overlays and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, body beautification, special effect display, virtual Interactive scenarios such as model display.
  • the relevant features, states and attributes of the target object can be detected or identified through the convolutional neural network.
  • the above-mentioned convolutional neural network is a network model obtained by performing model training based on a deep learning framework.
  • first and second are used for descriptive purposes only, and should not be understood as indicating or implying relative importance.
  • plurality means two or more, unless otherwise clearly defined.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Processing Or Creating Images (AREA)

Abstract

La présente invention concerne un procédé et un appareil de traitement d'image, un dispositif électronique et un support de stockage. Le procédé de traitement d'image comprend les étapes suivantes: l'obtention d'une image à traiter, l'image à traiter comprenant un objet cible; l'extraction de points clés tridimensionnels de l'objet cible, et la détermination de premières correspondances entre les points clés tridimensionnels et les sommets dans un modèle de réglage tridimensionnel; la détermination des résultats de réglage des points clés tridimensionnels selon une instruction de réglage, des données de déformation préétablies des sommets dans le modèle de réglage tridimensionnel, et des premières correspondances; et la détermination d'une image cible en fonction des résultats de réglage des points clés tridimensionnels et de l'image à traiter.
PCT/CN2022/125036 2021-10-20 2022-10-13 Procédé et appareil de traitement d'image, dispositif électronique et support de stockage Ceased WO2023066120A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111222848.5 2021-10-20
CN202111222848.5A CN113657357B (zh) 2021-10-20 2021-10-20 图像处理方法、装置、电子设备及存储介质

Publications (1)

Publication Number Publication Date
WO2023066120A1 true WO2023066120A1 (fr) 2023-04-27

Family

ID=78494756

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/125036 Ceased WO2023066120A1 (fr) 2021-10-20 2022-10-13 Procédé et appareil de traitement d'image, dispositif électronique et support de stockage

Country Status (2)

Country Link
CN (3) CN114581986A (fr)
WO (1) WO2023066120A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116894870A (zh) * 2023-08-03 2023-10-17 成都纵横大鹏无人机科技有限公司 一种图像目标定位方法、系统、电子设备及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114581986A (zh) * 2021-10-20 2022-06-03 北京市商汤科技开发有限公司 图像处理方法、装置、电子设备及存储介质
CN114648762B (zh) * 2022-03-18 2024-11-26 腾讯科技（深圳）有限公司 语义分割方法、装置、电子设备和计算机可读存储介质
CN115019021B (zh) * 2022-06-02 2025-04-18 北京字跳网络技术有限公司 图像处理方法、装置、设备及存储介质
CN115239860B (zh) * 2022-09-01 2023-08-01 北京达佳互联信息技术有限公司 表情数据生成方法、装置、电子设备及存储介质
CN115409951B (zh) * 2022-10-28 2023-03-24 北京百度网讯科技有限公司 图像处理方法、装置、电子设备及存储介质
CN119559361A (zh) * 2023-08-31 2025-03-04 北京字跳网络技术有限公司 图像处理方法及相关设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108447017A (zh) * 2018-05-31 2018-08-24 Oppo广东移动通信有限公司 人脸虚拟整容方法和装置
CN108550185A (zh) * 2018-05-31 2018-09-18 Oppo广东移动通信有限公司 人脸美化处理方法和装置
CN108876708A (zh) * 2018-05-31 2018-11-23 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和存储介质
CN109190503A (zh) * 2018-08-10 2019-01-11 珠海格力电器股份有限公司 美颜方法、装置、计算装置和存储介质
CN113657357A (zh) * 2021-10-20 2021-11-16 北京市商汤科技开发有限公司 图像处理方法、装置、电子设备及存储介质

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108765273B (zh) * 2018-05-31 2021-03-09 Oppo广东移动通信有限公司 人脸拍照的虚拟整容方法和装置
CN109191552A (zh) * 2018-08-16 2019-01-11 Oppo广东移动通信有限公司 三维模型处理方法、装置、电子设备和存储介质
CN109584151B (zh) * 2018-11-30 2022-12-13 腾讯科技（深圳）有限公司 人脸美化方法、装置、终端及存储介质
CN111985265B (zh) * 2019-05-21 2024-04-12 华为技术有限公司 图像处理方法和装置
CN110675489B (zh) * 2019-09-25 2024-01-23 北京达佳互联信息技术有限公司 一种图像处理方法、装置、电子设备和存储介质
CN111368678B (zh) * 2020-02-26 2023-08-25 Oppo广东移动通信有限公司 图像处理方法及相关装置
CN112700533B (zh) * 2020-12-28 2023-10-03 北京达佳互联信息技术有限公司 三维重建方法、装置、电子设备及存储介质
CN113379623B (zh) * 2021-05-31 2023-12-19 北京达佳互联信息技术有限公司 图像处理方法、装置、电子设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108447017A (zh) * 2018-05-31 2018-08-24 Oppo广东移动通信有限公司 人脸虚拟整容方法和装置
CN108550185A (zh) * 2018-05-31 2018-09-18 Oppo广东移动通信有限公司 人脸美化处理方法和装置
CN108876708A (zh) * 2018-05-31 2018-11-23 Oppo广东移动通信有限公司 图像处理方法、装置、电子设备和存储介质
CN109190503A (zh) * 2018-08-10 2019-01-11 珠海格力电器股份有限公司 美颜方法、装置、计算装置和存储介质
CN113657357A (zh) * 2021-10-20 2021-11-16 北京市商汤科技开发有限公司 图像处理方法、装置、电子设备及存储介质

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116894870A (zh) * 2023-08-03 2023-10-17 成都纵横大鹏无人机科技有限公司 一种图像目标定位方法、系统、电子设备及存储介质

Also Published As

Publication number Publication date
CN114581986A (zh) 2022-06-03
CN114581987A (zh) 2022-06-03
CN113657357A (zh) 2021-11-16
CN113657357B (zh) 2022-02-25

Similar Documents

Publication Publication Date Title
CN114972632B (zh) 基于神经辐射场的图像处理方法及装置
WO2023066120A1 (fr) Procédé et appareil de traitement d'image, dispositif électronique et support de stockage
EP3992919B1 (fr) Procédé et appareil de génération de modèle facial tridimensionnel, dispositif et support
US12067662B2 (en) Advanced automatic rig creation processes
CN110675487B (zh) 基于多角度二维人脸的三维人脸建模、识别方法及装置
WO2022095721A1 (fr) Procédé et appareil de formation de modèle d'estimation de paramètre, dispositif, et support de stockage
WO2021093453A1 (fr) Procédé de génération de base d'expression 3d, procédé interactif vocal, appareil et support
CN113628327B (zh) 一种头部三维重建方法及设备
CN111553284B (zh) 人脸图像处理方法、装置、计算机设备和存储介质
US12400475B2 (en) Face detection to generate parameters for a neural network that determines facial landmarks used for facial synthesis for head turns in augmented reality content
CN115668300A (zh) 利用纹理解析的对象重建
CN113516755B (zh) 图像处理方法、图像处理装置、电子设备和存储介质
CN114998514B (zh) 一种虚拟角色的生成方法及设备
CN114821675B (zh) 对象的处理方法、系统和处理器
WO2022237081A1 (fr) Procédé et appareil de transfert de style de maquillage, et dispositif et support de stockage lisible par ordinateur
WO2021244172A1 (fr) Procédé de traitement d'image et procédé de synthèse d'image, appareil de traitement d'image et appareil de synthèse d'image, et support de stockage
CN108111911B (zh) 基于自适应跟踪框分割的视频数据实时处理方法及装置
CN113822965A (zh) 图像渲染处理方法、装置和设备及计算机存储介质
CN114693853A (zh) 对象渲染方法、装置、电子设备及存储介质
US11080920B2 (en) Method of displaying an object
CN114820907B (zh) 人脸图像卡通化处理方法、装置、计算机设备和存储介质
WO2024174422A1 (fr) Procédé et appareil de génération de modèle, dispositif électronique et support de stockage
CN120339528B (zh) 风格化三维虚拟场景生成方法、电子设备和存储介质
CN119832193A (zh) 一种虚拟现实交互系统、虚拟现实交互方法及其相关设备
JP6341540B2 (ja) 情報端末装置、方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22882730

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22882730

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 27.11.2024)

122 Ep: pct application non-entry in european phase

Ref document number: 22882730

Country of ref document: EP

Kind code of ref document: A1