WO2018164327A1 - Method and system for estimating anatomical layer of face, and non-transitory computer-readable recording medium - Google Patents

Abstract

The present invention relates to a method and system for estimating an anatomical layer of a face, and to a non-transitory computer-readable recording medium. An aspect of the present invention provides the method for estimating an anatomical layer of a face, comprising: a step of acquiring three-dimensional measurement data of the shape of a face of a user; and a step of analyzing the acquired three-dimensional measurement data by comparing same against at least one anatomical face model associated with the user data, thereby estimating the shape and location of at least one kind of an anatomical layer included in soft tissue of the face of the user, wherein the anatomical face model includes modeled data of the shape and location of the at least one kind of the anatomical layer included in the soft tissue of the face.

Description

Method, system and non-transitory computer readable recording medium for estimating the anatomical layer on the face

The present invention relates to a method, system and non-transitory computer readable recording medium for estimating an anatomical layer on a face.

As interest in appearance increases, interest in services that help users to virtually decorate their appearance through plastic surgery, orthodontics, dermatology, cosmetics, accessories, and the like is exploding. In addition, a lot of researches have been conducted on the technology of virtually realizing the user's appearance virtually like the actual appearance, and the technology of accurately and precisely implementing the expected changes in the user's appearance.

As an example of the related art, a face image is obtained from a user, an outline of a face component such as an eye, a nose, a mouth, and a chin is extracted from the acquired face image, and based on each feature point of the component A method of generating a user's face and performing a virtual shaping simulation on the user's face generated through a mesh warping technique has been introduced.

However, according to the above-described prior arts and the related arts, the whole face of the user is estimated or only the face or bone of the user is imaged based on a local feature obtained from the image of the face or bone of the user. Based on this, only the simulation service such as virtual molding was provided, so there was no difference between the actual result and the virtual result.

The present invention aims to solve all of the above-mentioned problems of the prior art.

In addition, another object of the present invention is to estimate the shape and position of at least one type of anatomical layer included in soft tissue of a user's face based on user information.

It is another object of the present invention to provide a simulation service for at least one of plastic surgery, skin treatment, growth, aging and obesity to the user based on the estimated anatomical layer.

Representative configuration of the present invention for achieving the above object is as follows.

According to an aspect of the present invention, there is provided a method for estimating an anatomical layer of a face, the method comprising: acquiring three-dimensional measurement data of a shape of a user's face; Estimating the shape and location of at least one type of anatomical layer included in the soft tissue of the user's face by analyzing it relative to at least one associated anatomical face model, wherein the anatomical face model includes: A method is provided that includes modeled data regarding the shape and location of at least one type of anatomical layer included in soft tissues of the face.

According to another aspect of the present invention, in the system for estimating the anatomical layer of the face, the measurement data acquisition unit for obtaining the three-dimensional measurement data about the shape of the user's face, and the obtained three-dimensional measurement data And a layer estimator for estimating the shape and position of at least one kind of anatomical layer included in the soft tissue of the user's face by analyzing the at least one anatomical face model associated with user information. The face model is provided with a system that includes modeled data regarding the shape and location of at least one type of anatomical layer included in soft tissues of the face.

In addition, there is further provided a non-transitory computer readable recording medium for recording another method for implementing the present invention, another system, and a computer program for executing the method.

According to the present invention, it is possible to estimate the shape and position of at least one kind of anatomical layer included in the soft tissue of the user's face based on the user information.

According to the present invention, it is possible to provide a simulation service for at least one of plastic surgery, skin treatment, growth, aging and obesity to the user based on the estimated anatomical layer.

1 is a diagram illustrating a schematic configuration of an entire system for estimating an anatomical layer about a face according to an embodiment of the present invention.

2 is a diagram showing in detail the internal configuration of the estimation system according to an embodiment of the present invention.

3 is a diagram exemplarily illustrating a process of estimating an anatomical layer included in soft tissue of a user's face according to one embodiment of the present invention.

<Description of the code>

100: network

200: estimation system

210: measurement data acquisition unit

220: layer estimator

230: service provider

240: communication unit

250: control unit

300: device

DETAILED DESCRIPTION The following detailed description of the invention refers to the accompanying drawings that show, by way of illustration, specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be implemented with changes from one embodiment to another without departing from the spirit and scope of the invention. In addition, it is to be understood that the location or arrangement of individual components within each embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention should be taken as encompassing the scope of the claims of the claims and all equivalents thereto. Like reference numerals in the drawings indicate the same or similar elements throughout the several aspects.

Hereinafter, various preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art can easily implement the present invention.

Configuration of the entire system

1 is a diagram illustrating a schematic configuration of an entire system for estimating an anatomical layer about a face according to an embodiment of the present invention.

As shown in FIG. 1, an entire system according to an embodiment of the present invention may include a communication network 100, an estimation system 200, and a device 300.

First, the communication network 100 according to an embodiment of the present invention may be configured regardless of a communication mode such as wired communication or wireless communication, and includes a local area network (LAN) and a metropolitan area network (MAN). ), And various communication networks such as a wide area network (WAN). Preferably, the communication network 100 as used herein may be a known Internet or World Wide Web (WWW). However, the communication network 100 may include, at least in part, a known wired / wireless data communication network, a known telephone network, or a known wired / wireless television communication network without being limited thereto.

For example, the communication network 100 is a wireless data communication network, and includes Wi-Fi communication, Wi-Fi Direct communication, Long Term Evolution (LTE) communication, Bluetooth communication (for example, low power Bluetooth BLE (Bluetooth Low Energy), infrared communication, ultrasonic communication, etc. may be implemented at least in part.

Next, the estimation system 200 according to an embodiment of the present invention may perform communication with the device 300, which will be described later, through the communication network 100, and obtain 3D measurement data regarding the shape of the face of the user. And analyzing the obtained three-dimensional measurement data by comparing the obtained three-dimensional measurement data with at least one anatomical face model associated with the user information, thereby obtaining at least one kind of anatomical layer included in the soft tissue of the user's face. The function of estimating the shape and the position can be performed.

In addition, the estimation system 200 according to an embodiment of the present invention refers to the shape and location of at least one type of anatomical layer included in the soft tissue of the user's face, which may include cosmetic surgery, skin treatment, and growth to the user. It may also perform a function of providing a simulation service relating to at least one of aging, obesity, and the like.

The configuration and function of the estimation system 200 according to the present invention will be described in detail through the following detailed description. On the other hand, as described above with respect to the estimation system 200, this description is exemplary, and at least some of the functions or components required for the estimation system 200 will be described later as necessary, the device 300 or an external system It will be apparent to those skilled in the art that they may be implemented within (not shown) or incorporated into device 300 or an external system (not shown).

Next, according to an embodiment of the present invention, the device 300 is a digital device including a function of enabling communication after connecting to the estimation system 200 through the communication network 100, such as a smartphone, tablet PC, etc. As long as a portable digital device having a memory means and a microprocessor equipped with arithmetic capability can be adopted as the device 300 according to the present invention.

Meanwhile, according to an embodiment of the present invention, the device 300 may include an application for estimating an anatomical layer of a face and supporting a function according to the present invention in which a simulation service is provided to a user. have. Such an application may be downloaded from the estimation system 200 or an external application distribution server (not shown).

Configuration of the estimation system

Hereinafter, the internal configuration of the estimation system 200 performing important functions for the implementation of the present invention and the function of each component will be described.

2 is a diagram showing in detail the internal configuration of the estimation system 200 according to an embodiment of the present invention.

Estimating system 200 according to an embodiment of the present invention may be a digital device having a computing capability by mounting a microprocessor and a memory means. Such estimation system 200 may be a server system. As shown in FIG. 2, the estimation system 200 may include a measurement data acquisition unit 210, a layer estimator 220, a service provider 230, a communication unit 240, and a controller 250. Can be. According to an embodiment of the present invention, the measurement data acquisition unit 210, the layer estimator 220, the service provider 230, the communication unit 240 and the control unit 250 is at least a part of the external system and It may be a program module that communicates. Such program modules may be included in the estimation system 200 in the form of operating systems, application modules or other program modules, and may be physically stored in various known storage devices. Such program modules may also be stored in a remote storage device capable of communicating with the estimation system 200. On the other hand, such program modules include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc. that perform particular tasks or execute particular abstract data types, described below, in accordance with the present invention.

First, according to an embodiment of the present invention, the measurement data acquisition unit 210 may perform a function of acquiring three-dimensional measurement data regarding a shape of a user's face. The three-dimensional measurement data according to an embodiment of the present invention may include at least one of three-dimensional scanning data of the user's face and three-dimensional imaging data of the hard tissue of the user's face, such three-dimensional scanning Alternatively, the imaging data may include x-rays, ultrasound waves, computer tomography (CT), magnetic resonance images (MRI), and positron emission tomography (PET). And a three-dimensional scanner.

Next, the layer estimator 220 according to an embodiment of the present invention at least one anatomy associated with the user information, the three-dimensional measurement data about the shape of the user's face obtained from the measurement data acquisition unit 210 By comparing with the enemy face model, a function of estimating the shape and position of at least one kind of anatomical layer included in the soft tissue of the user's face may be performed. According to an embodiment of the present invention, the anatomical face model is obtained by scanning the peeled (or unpeeled) layer while peeling off the anatomical layers of the soft tissue of the face of the dead human body (ie, the dead body). (For example, the position and shape of a layer) and data obtained by transmitting at least one of ultrasonic waves, radiation, a magnetic field, and a positron to a living human facial soft tissue (for example, data regarding the position and shape of a layer). Based on at least one, may be facial modeling data obtained by normalizing, averaging, or normalizing the anatomical layers of the face according to demographic variables (eg, race, gender, age, etc.), such anatomy Red face models relate to the shape and location of at least one type of anatomical layer that may be included in the soft tissues of the face. The modeled data may be included. On the other hand, according to an embodiment of the present invention, the above-described obtained data is supported by a support vector machine (SVM) algorithm, a multivariate adaptive regression spline (MARS) algorithm, a nearest neighbor (KNN) algorithm, a neural network (NN) algorithm, and the like. By analyzing through known algorithms for deep learning or big data, an anatomical face model may be obtained that patterns the anatomical layers of the face according to demographic indicators. In addition, according to an embodiment of the present invention, the above-described anatomical layer may include layers relating to at least one of muscle, fat, blood vessels, nerves and lymphatic vessels, and the user information may include race, ethnicity, gender, and age of the user. Information may be included. On the other hand, the type of the anatomical layer and the user information according to an embodiment of the present invention is not necessarily limited to those listed above, periosteum, fascia (faiocia) within the scope that can achieve the object of the present invention , Anatomic layers such as ligaments, or user information, such as a country or a residential area.

Specifically, according to an embodiment of the present invention, the layer estimator 220 may include at least one feature related to the face of the user, which is extracted from the 3D measurement data regarding the shape of the face of the user, associated with the corresponding user information. Compared to the anatomical face model, the user's face by matching or transforming the shape and position of at least one type of anatomical layer included in the anatomical face model to conform to three-dimensional measurement data about the shape of the user's face It is possible to estimate the shape or position of at least one type of anatomical layer included in the soft tissue of.

In this case, according to an embodiment of the present invention, the layer estimator 220 is active to compare and analyze three-dimensional measurement data regarding the shape of the user's face and at least one anatomical face model associated with the user information. Active appearance model (AAM) algorithms, active shape model (ASM) algorithms, composite constraint AAM (Composite Constraint AAM) algorithms, iterative closest points (ICP) algorithms, and non-rigid matching At least one of an algorithm (non-rigid registration) may be used.

For example, according to an embodiment of the present invention, the layer estimator 220 may extract an active appearance model (AAM) to extract features of the face of the user from three-dimensional measurement data regarding the shape of the user's face. At least one of an algorithm, an active shape model (ASM) algorithm, and a composite constraint active expression model (Composite Constraint AAM) algorithm may be used, and iterative closest points (ICP) algorithms and non-rigid matching algorithms ( by matching or transforming the shape and position of at least one type of anatomical layer included in the anatomical face model with at least one of non-rigid registration to conform to three-dimensional measurement data about the shape of the user's face, The shape or position of at least one type of anatomical layer included in the soft tissue of the face may be estimated. Meanwhile, according to an embodiment of the present invention, an algorithm for comparing and analyzing three-dimensional measurement data about a shape of a face and at least one anatomical face model associated with user information is not necessarily limited to those listed above. It is noted that other algorithms may be utilized within the scope of the object of the present invention.

Next, according to an embodiment of the present invention, the service provider 230 refers to the shape and position of at least one type of anatomical layer included in the soft tissue of the face of the user estimated by the layer estimator 220. , To provide a user with a simulation service relating to at least one of plastic surgery, skin treatment, growth, aging and obesity. In this case, the layer estimator 220 according to an embodiment of the present invention virtually deforms, injects, and extracts the user's face according to a simulation service regarding at least one of plastic surgery, skin treatment, growth, aging, and obesity. When at least one is determined, the shape and position of at least one type of anatomical layer included in the soft tissue of the user's face may be reestimated.

For example, according to an embodiment of the present invention, if the service provider 230 is determined to inject a virtual implant or drug to the face of the user for plastic surgery or skin treatment during the above-described simulation service, With reference to the shape and position of the layer and the other layer that changes (ie, re-estimated) depending on which one of the at least one anatomical layer included in the soft tissue of the user's face is located the implant or drug, A virtual simulation service about a user's face that changes after implantation or drug injection can be provided to the user. That is, according to an embodiment of the present invention, it is possible to accurately predict and provide a face of a user expected to change according to a plastic surgery or a skin procedure based on an anatomical layer.

For another example, according to an embodiment of the present invention, when it is determined that the service provider 230 provides a simulation service according to growth or aging among simulation services, the service provider 230 is included in the soft tissue of the face of the estimated user. Virtual simulation service for a user's face that changes due to growth or aging, referring to the shape and location that changes (ie, reestimated) as at least one type of anatomical layer grows or ages. Can be provided. That is, according to an embodiment of the present invention, it is possible to accurately predict and provide a face of a user expected to change due to growth or aging based on an anatomical layer. On the other hand, according to an embodiment of the present invention, the shape and position that is changed (ie, re-estimated) as the anatomical layer included in the soft tissue of the user's face is grown or aged are obtained from the above-described anatomical face model. It may be obtained from the modeled data with respect to the average shape and position of the anatomical layer by age.

Next, the communication unit 240 according to an embodiment of the present invention performs a function to enable data transmission and reception from and to the measurement data acquisition unit 210, the layer estimator 220, and the service provider 230. can do.

Finally, the controller 250 according to an embodiment of the present invention controls the flow of data between the measurement data acquisition unit 210, the layer estimator 220, the service provider 230, and the communication unit 240. Can be performed. That is, the controller 250 according to the present invention controls the data flow from / to the outside of the estimation system 200 or the data flow between each component of the estimation system 200, thereby measuring the measurement data acquisition unit 210 and the layer. The estimator 220, the service provider 230, and the communicator 240 may control each to perform a unique function.

3 is a diagram exemplarily illustrating a process of estimating an anatomical layer included in soft tissue of a user's face according to one embodiment of the present invention.

Referring to FIG. 3, in accordance with an embodiment of the present invention, the estimation system 200 may acquire 3D computed tomography data of hard tissue of a user's face or 3D scanning data of a user's face (301).

Next, at least one anatomical face model associated with at least one of the race, gender and age of the user is selected 302 and at least included in the selected anatomical face model according to one embodiment of the invention. At least one type of anatomy included in the soft tissue of the user's face by matching or transforming the shape or position of one type of anatomical layer to match three-dimensional data about the shape of the user's face using a non-rigid registration algorithm. The shape or position of the enemy layer may be estimated (303).

Next, the estimation system 200 according to an embodiment of the present invention refers to the shape and location of the estimated at least one type of anatomical layer, and to the user through the device 200, a plastic surgery, skin treatment It may provide a simulation service (304) regarding at least one of growth, aging and obesity.

On the other hand, the present invention, in providing a simulation service for at least one of plastic surgery, skin surgery, growth, aging and obesity, the shape of the various anatomical layers included in the soft tissue of the user's face according to the simulation service And re-estimating the change in position can provide more accurate results than estimating the simulation results of at least one of cosmetic surgery, skin treatment, growth, aging and obesity based solely on facial appearance.

Embodiments according to the present invention described above can be implemented in the form of program instructions that can be executed by various computer components and recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs and DVDs, and magneto-optical media such as floptical disks. medium) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be modified with one or more software modules to perform the processing according to the present invention, and vice versa.

Although the present invention has been described by specific matters such as specific components and limited embodiments and drawings, it is provided only to help a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Those skilled in the art may make various modifications and changes from this description.

Therefore, the spirit of the present invention should not be limited to the above-described embodiments, and the scope of the spirit of the present invention is defined not only in the claims below, but also in the ranges equivalent to or equivalent to the claims. Will belong to.

Claims (8)

A method for estimating an anatomical layer on a face, Obtaining three-dimensional measurement data about the shape of the user's face, and By analyzing the obtained three-dimensional measurement data by comparing with the at least one anatomical face model associated with the user information, the shape of at least one kind of anatomical layer included in the soft tissue of the user's face and Estimating the location Including, The anatomical face model includes modeled data regarding the shape and position of at least one type of anatomical layer included in the soft tissue of the face. Way. The method of claim 1, In the acquiring step, the acquired three-dimensional measurement data includes at least one of three-dimensional scanning data of a face of the user and three-dimensional imaging data of a hard tissue of the user's face. Way. The method of claim 1, In the estimating step, the anatomical layer includes at least one of muscle, fat, blood vessels, nerves and lymphatic vessels. Way. The method of claim 1, In the estimating step, an active appearance model (AAM) algorithm, an active shape model (ASM) algorithm, a composite constraint active expression model (Composite Constraint AAM) algorithm, and iterative closest points (ICP) Comparing and analyzing the obtained three-dimensional measurement data and the anatomical face model using at least one of an algorithm and a non-rigid registration algorithm. Way. The method of claim 1, Providing the user with a simulation service relating to at least one of plastic surgery, skin surgery, growth, aging, and obesity with reference to the estimated shape and location of the at least one type of anatomical layer; Way. The method of claim 5, In the providing step, when at least one of virtual deformation, injection, and extraction of the shape of the user's face is determined according to the simulation service, the shape and position of at least one kind of anatomical layer included in the soft tissue of the user's face To reestimate each Way. A non-transitory computer readable recording medium for recording a computer program for executing the method according to any one of claims 1 to 6. In the system for estimating the anatomical layer on the face, A measurement data acquisition unit for obtaining 3D measurement data regarding a shape of a user's face, and By analyzing the obtained three-dimensional measurement data compared to at least one anatomical face model associated with the user information, to estimate the shape and position of at least one type of anatomical layer included in the soft tissue of the user's face Layer estimator Including, The anatomical face model includes modeled data regarding the shape and position of at least one type of anatomical layer included in the soft tissue of the face. system.

Images