WO2024183760A1 - Scanning data splicing method and apparatus, and device and medium - Google Patents

Abstract

The embodiments of the present disclosure relate to a scanning data splicing method and apparatus, and a device and a medium. The method comprises: acquiring a facial pose corresponding to a facial scanning mesh result of a user; acquiring a tooth pose corresponding to a mouth scanning mesh result of the user; and splicing the facial scanning mesh result and the mouth scanning mesh result on the basis of the facial pose and the tooth pose, so as to obtain an initial splicing result. By means of the technical solution, a facial pose and a tooth pose assist in splicing, such that the accuracy of an initial splicing result is improved while the splicing efficiency is ensured, thereby improving a final splicing success rate.

Description

Scanning data splicing method, device, equipment and medium

This disclosure claims the priority of the Chinese patent application filed with the Chinese Patent Office on March 7, 2023, with application number 202310229276.6 and invention name “A method, device, equipment and medium for stitching scanned data”, the entire contents of which are incorporated by reference in this disclosure.

Technical Field

The present disclosure relates to the field of data processing technology, and in particular to a scan data splicing method, device, equipment and medium.

Background Art

Usually, in application scenarios such as smile design and simulated orthodontics, it is necessary to accurately splice the mouth scan data with the face scan data to achieve the effect of linking the face scan data with the mouth scan data, which can serve as a reference for later orthodontic diagnosis and treatment, medical aesthetics and other fields. However, the resolution difference between the mouth scan data and the face scan data is relatively large, the proportion of the splicable area is low, the face scan data is affected by the face, and the tooth data in the face scan data is greatly affected by noise, resulting in difficulty in splicing the mouth scan data with the tooth data in the face scan data.

In the related art, the stitching accuracy of the mouth scan data and the face scan data is relatively poor.

Summary of the invention

In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides a scan data stitching method, device, equipment and medium.

The present disclosure provides a scan data stitching method, the method comprising:

Get the facial pose corresponding to the user's facial scan mesh result;

Obtaining a tooth posture corresponding to a mesh result of a mouth scan of the user;

Based on the facial pose and the tooth pose, the facial scanning mesh result and the The above-mentioned mouth scanning mesh results are spliced to obtain an initial splicing result.

The present disclosure also provides a scan data splicing device, the device comprising:

The first acquisition module is used to obtain the facial posture corresponding to the user's facial scanning grid result;

A second acquisition module is used to acquire the tooth posture corresponding to the user's mouth scanning grid result;

A splicing module is used to splice the facial scan grid result and the mouth scan grid result based on the facial posture and the tooth posture to obtain an initial splicing result.

An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the scan data stitching method provided in the embodiment of the present disclosure.

The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the scan data stitching method provided by the embodiment of the present disclosure.

The scanning data stitching solution provided by the embodiment of the present disclosure obtains the facial pose corresponding to the user's facial scanning grid result, and obtains the tooth pose corresponding to the user's mouth scanning grid result, and stitches the facial scanning grid result and the mouth scanning grid result based on the facial pose and the tooth pose to obtain an initial stitching result.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

FIG1 is a schematic diagram of a flow chart of a scan data stitching method provided by an embodiment of the present disclosure;

FIG. 2 is a schematic diagram of another scanning data stitching method provided by an embodiment of the present disclosure. picture;

FIG3 is a schematic diagram of the structure of a scan data splicing device provided by an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure.

DETAILED DESCRIPTION

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present disclosure is not limited in this respect.

The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

At present, the proportion of dental data in facial scan data is very low, the stitching area is small, and the dental data of the mouth scan is relatively fine. The resolution of facial scan data is low, and the dental data in facial scan data is often interfered by noise. There is also the problem of incomplete data. These situations have greatly increased the difficulty of stitching. The existing stitching methods have a large amount of calculation, weak anti-noise ability, and low stitching accuracy.

In view of the above problems, the embodiments of the present disclosure propose a scan data stitching method, which improves the existing rough stitching method and increases the stitching success rate for the tooth stitching scenario of oral scan data and facial scan data.

Specifically, FIG1 is a flow chart of a scan data stitching method provided by an embodiment of the present disclosure, and the method can be performed by a scan data stitching device, wherein the device can be implemented by software and/or hardware, and can generally be integrated in an electronic device. As shown in FIG1 , the method includes:

Step 101: Obtain the facial pose corresponding to the user's facial scanning grid result.

Among them, the face scanning mesh result refers to the mesh data of the user's face. The face point cloud data can be obtained by scanning the user's face with a face scanner or other equipment, and then meshed to obtain the face scanning mesh result. Facial postures, such as the frontal face orientation, the side face orientation, etc.

In an embodiment of the present disclosure, after obtaining a facial scan grid result, the facial pose corresponding to the facial scan grid result can be obtained. In some implementations, a facial image of the user is obtained, and the facial image is recognized to obtain a dental area and key facial feature points. The dental area and key facial feature points are projected to the facial scan grid result to obtain a facial scan grid result to be processed. Calculations are performed based on the facial scan grid result to obtain the facial pose corresponding to the facial scan grid result.

In other embodiments, facial feature points corresponding to the facial scan grid result are obtained, and the facial pose corresponding to the facial scan grid result is determined by calculation based on the facial feature points. The above two methods are only examples of obtaining the facial pose corresponding to the user's facial scan grid result, and the present disclosure does not specifically limit the implementation method of obtaining the facial pose corresponding to the user's facial scan grid result.

Step 102: Obtain the tooth posture corresponding to the user's mouth scan grid result.

The mouth scanning grid result refers to the grid data of the user's mouth. The mouth of the user can be scanned by a device such as a mouth scanner to obtain the mouth point cloud data, and then the mouth scanning grid result can be obtained by gridding.

In the embodiments of the present disclosure, tooth posture refers to the direction of the front teeth area. There are many ways to obtain the tooth posture corresponding to the mouth scanning grid result. In some embodiments, the mouth scanning grid result is projected onto a two-dimensional plane to obtain two-dimensional coordinate points of the mouth. The two-dimensional coordinate points of the mouth are processed according to a preset processing method to obtain a two-dimensional curve. The parameters corresponding to the two-dimensional curve are fitted based on the dental arch line fitting method to obtain the tooth posture; in other embodiments, the mouth scanning grid result is directly calculated based on a preset posture estimation formula to obtain the tooth posture; the above two methods are only examples of obtaining the tooth posture corresponding to the mouth scanning grid result. The embodiments of the present disclosure do not specifically limit the method of obtaining the tooth posture corresponding to the mouth scanning grid result.

Step 103: splice the facial scan mesh result and the mouth scan mesh result based on the facial pose and the tooth pose to obtain an initial splicing result.

In an embodiment of the present disclosure, after obtaining the facial pose and the tooth pose, the facial scan grid result and the mouth scan grid result are spliced based on the facial pose and the tooth pose to obtain an initial splicing result. In some embodiments, the mouth scan grid result is converted to a coordinate system corresponding to the facial scan grid result, and feature points of the same target number and the same position are respectively obtained from the facial scan grid result and the mouth scan grid result, and the rigid body transformation matrix between the feature points is determined. The facial scan grid result and the mouth scan grid result to be processed are spliced according to the rigid body transformation matrix to obtain an initial splicing result. In other embodiments, the facial scan grid result is converted to a coordinate system corresponding to the mouth scan grid result, the upper teeth area is cut from the facial scan grid result, and the front teeth area of a certain width is cut from the mouth scan grid result. Based on a feature descriptor of a fixed direction (such as a point on the tooth cusp, etc.), the cut front teeth area is spliced to the upper teeth area to obtain an initial splicing result. The above two methods are only examples. The embodiment of the present disclosure does not perform the method of splicing the facial scan mesh result and the mouth scan mesh result based on the facial posture and the tooth posture to obtain the initial splicing result. Specific restrictions.

The scan data stitching solution provided by the embodiment of the present disclosure obtains the facial pose corresponding to the user's facial scan grid result, and obtains the tooth pose corresponding to the user's mouth scan grid result, and stitches the facial scan grid result and the mouth scan grid result based on the facial pose and the tooth pose to obtain an initial stitching result. The above technical solution is adopted, and the facial pose and the tooth pose are used to assist the stitching, so as to improve the accuracy of the initial stitching result while ensuring the stitching efficiency, thereby improving the final stitching success rate and meeting the user's accuracy requirements for the stitching of scan data.

FIG2 is a flow chart of another scan data stitching method provided by an embodiment of the present disclosure. Based on the above embodiment, this embodiment further optimizes the above scan data stitching method. As shown in FIG2 , the method includes:

Step 201: Obtain a user's facial image, and recognize the facial image to obtain the tooth area and key facial feature points.

Among them, the face image refers to a two-dimensional picture including the user's face, which can be obtained by scanning the user's face through a device such as a face scanner.

In the embodiments of the present disclosure, there are many ways to recognize facial images. In some implementations, facial images can be recognized by a facial intelligent recognition algorithm in an artificial intelligence recognition algorithm to obtain various areas corresponding to the face, such as the eye area, nose area, and tooth area, etc., and obtain key facial feature points, such as the tip of the nose, the root of the nose, the chin, the center of the mouth, the left corner of the mouth, the outer corner of the left eye, and the inner corner of the right eye, etc. 75 facial feature points.

The tooth area may include the upper tooth area and the lower tooth area or the entire tooth area. In the disclosed embodiment, the facial scanning grid result mainly exposes the upper teeth, with less exposure of the lower teeth. The tooth area generally refers to the upper tooth area. It should be noted that after the upper tooth area is spliced, the lower tooth area can be determined according to the tooth occlusion state, thereby further improving the splicing efficiency.

Step 202: Project the tooth area and key feature points of the face to the facial scan grid result to obtain the facial scan grid result to be processed.

In the disclosed embodiment, after obtaining the tooth area and key facial feature points based on the facial image, it is necessary to project the tooth area and the key facial feature points onto the facial scanning grid result to obtain the facial scanning grid result to be processed, that is, map the pixel coordinate points of the tooth area and the two-dimensional coordinate points of the key facial feature points to the facial scanning grid result, thereby obtaining the facial scanning grid result to be processed including the tooth area and the key facial feature points.

It can be understood that the facial image is a two-dimensional image, so the tooth area and the key feature points of the face are both two-dimensional coordinate points in the two-dimensional image coordinate system. The tooth area and the key feature points of the face in the image coordinate system can be converted to three-dimensional coordinate points in the scanning camera coordinate system by obtaining the coordinate transformation matrix between the image coordinate system and the scanning camera coordinate system. It can also be understood that the tooth area and the key feature points of the face are projected onto the facial scanning grid result based on the camera projection principle to obtain the facial scanning grid result to be processed.

In some embodiments, the tooth area and key feature points of the face are projected onto the facial scanning grid result to obtain the facial scanning grid result to be processed, including: obtaining pixel coordinate points corresponding to the tooth area; obtaining feature coordinate points corresponding to the key feature points of the face; projecting the pixel coordinate points and feature coordinate points onto the facial scanning grid result according to the transformation relationship between the image coordinate system and the scanning camera coordinate system to obtain the facial scanning grid result to be processed.

Specifically, after identifying the dental area (usually referring to the pixel point set area corresponding to the upper dental area) and the key facial feature points on the face image (a two-dimensional face image taken of a user including the face) through the image's artificial intelligence recognition algorithm, the dental area and the key facial feature points are projected onto the facial scanning grid result based on the camera projection principle.

Specifically, the pixel coordinate points corresponding to the tooth area are obtained, the feature coordinate points corresponding to the key feature points of the face are obtained, and the pixel coordinate points and the feature coordinate points are projected to the facial scanning grid result according to the transformation relationship between the image coordinate system and the scanning camera coordinate system to obtain the facial scanning grid result to be processed.

Step 203: Calculate based on the face scan mesh result to be processed to obtain the face pose corresponding to the face scan mesh result.

In the embodiment of the present disclosure, after obtaining the face scanning grid result to be processed, it means that each facial feature point in the face scanning grid result to be processed can be obtained, that is, facial feature points, and estimate the facial pose based on each facial feature point. For example, 75 facial feature points are input into a preset facial pose calculation formula for calculation to obtain facial poses such as the frontal face orientation and the side face orientation. For example, multiple facial directions are determined through each facial feature point, and the facial pose is determined based on the multiple facial directions. The specific setting is selected according to the application scenario, and the embodiments of the present disclosure do not impose specific restrictions.

In some embodiments, a facial contour, a first feature area, and a second feature area are determined based on facial feature points, a facial plane is determined based on the facial contour and the first feature area, a first midline feature point and a second midline feature point are determined based on the facial feature points, and a longitudinal direction of the facial plane is determined based on the first midline feature point and the second midline feature point, and the longitudinal direction of the facial plane is adjusted based on the center of gravity direction of the facial contour and the center of gravity direction of the second feature area to obtain a facial pose corresponding to the facial scanning grid result.

Among them, the first characteristic area may refer to the eye and eyebrow area; the second characteristic area refers to the mouth area; the first midline refers to the nose midline; and the second midline refers to the mouth midline.

Specifically, the facial pose corresponding to the facial scanning grid result is calculated based on the facial feature points in three-dimensional space. Usually, facial pose estimation can be completed based on the facial feature points. For example, the facial plane is estimated by taking the vertices of the facial contour and the eye and eyebrow areas, the feature points of the midline of the nose and the midline of the mouth are taken to estimate the longitudinal direction, and the center of gravity of the mouth area and the center of gravity of the facial contour are taken to calibrate the facial orientation, thereby determining the facial pose.

Step 204: Project the mouth scanning grid result onto a two-dimensional plane to obtain the two-dimensional coordinate points of the mouth, process the two-dimensional coordinate points of the mouth according to a preset processing method to obtain a two-dimensional curve, fit the parameters corresponding to the two-dimensional curve based on the dental arch line fitting method to obtain the tooth posture.

Specifically, the mouth scanning grid results are projected onto a two-dimensional plane to obtain the two-dimensional coordinate points of the mouth, and the two-dimensional coordinate points of the mouth are converted into a two-dimensional curve (expressed by the two-dimensional curve formula) on the two-dimensional plane through diffusion corrosion. Finally, the parameters corresponding to the two-dimensional curve are fitted by the dental arch line fitting method to obtain the tooth posture.

Step 205: Based on the facial posture and the tooth posture, the facial scan mesh result and the mouth scan mesh result are unified into the same coordinate system to obtain the facial scan mesh result to be spliced and the mouth scan mesh result to be spliced. Scan mesh results of the joint surface.

In an embodiment of the present disclosure, the facial scan grid result and the oral scan grid result are unified into the same coordinate system based on the facial posture and the tooth posture, including: determining the coordinate transformation matrix between the facial scan grid result and the oral scan grid result based on the facial posture and the tooth posture, and transforming the oral scan grid result into the coordinate system corresponding to the facial scan grid result according to the coordinate transformation matrix; or, transforming the facial scan grid result into the coordinate system corresponding to the oral scan grid result according to the coordinate transformation matrix.

Step 206: determine a first tooth region to be spliced from the face scan grid result to be spliced, and determine a second tooth region to be spliced of a target width from the face scan grid result to be spliced.

Step 207: Acquire feature points of the same target quantity and position in the first tooth region to be spliced and the second tooth region to be spliced, respectively, and determine the rigid body transformation matrix between the feature points in the first tooth region to be spliced and the feature points in the second tooth region to be spliced, and splice the face scan mesh result to be processed and the mouth scan mesh result according to the rigid body transformation matrix to obtain an initial splicing result.

Specifically, based on the facial posture and the tooth posture, the mouth scanning grid result and the mouth scanning grid result are aligned to the same coordinate system, and a preset number of feature points are selected from the facial scanning grid result and the mouth scanning grid result respectively, wherein the number of feature points in the facial scanning grid result is equal to the number of feature points in the mouth scanning grid result and the positions are the same, and the rigid body transformation matrix between the feature points in the facial scanning grid result and the feature points in the mouth scanning grid result is determined, and the facial scanning grid result and the mouth scanning grid result are spliced according to the rigid body transformation matrix to obtain an initial splicing result.

For example, the upper teeth area is cut from the facial scan grid result, and the front teeth area of a certain width (set according to the empirical value) is cut from the mouth scan grid result. Based on the fixed direction (pre-set) feature descriptor (such as the point on the tooth tip), the cut mouth scan grid result is spliced to the facial scan grid result to obtain the initial splicing result. It can be understood that under the premise that the pose has been confirmed, the feature descriptor no longer needs the estimation of the local coordinate system. The local coordinate system with a fixed direction can ensure that the pose changes during the splicing process are as small as possible, further improving the accuracy of the splicing process. High splicing success rate.

Step 208: Process the overlapping source scan point cloud and target scan point cloud in the initial stitching result to obtain a rotation and translation matrix, transform the source scan point cloud to the coordinate system of the target scan point cloud according to the rotation and translation matrix, determine the error between the transformed source scan point cloud and the target scan point cloud, until the error is less than or equal to a preset error threshold, and obtain the target stitching result.

Specifically, the ICP (Iterative Closest Point) algorithm is used to perform stitching adjustment on the initial stitching result, that is, the overlapping source scan point cloud and target scan point cloud in the initial stitching result are processed to obtain a rotation and translation matrix, and the source scan point cloud is transformed into the coordinate system of the target scan point cloud according to the rotation and translation matrix, and the error between the transformed source scan point cloud and the target scan point cloud is determined until the error is less than or equal to a preset error threshold, and the target stitching result is obtained. The error threshold is selected and set according to the application scenario.

The scanning data splicing solution provided by the embodiment of the present disclosure obtains the user's face image, recognizes the face image, obtains the tooth area and the key feature points of the face, projects the tooth area and the key feature points of the face to the face scanning grid result, obtains the face scanning grid result to be processed, performs calculation based on the face scanning grid result to obtain the facial posture corresponding to the facial scanning grid result, projects the mouth scanning grid result to a two-dimensional plane, obtains the two-dimensional coordinate points of the mouth, processes the two-dimensional coordinate points of the mouth according to a preset processing method, obtains a two-dimensional curve, fits the parameters corresponding to the two-dimensional curve based on the dental arch line fitting method, obtains the tooth posture, unifies the face scanning grid result and the mouth scanning grid result into the same one based on the face posture and the tooth posture. A coordinate system is provided, and the facial scanning grid result to be spliced and the mouth scanning grid result to be spliced are obtained, a first tooth region to be spliced is determined from the facial scanning grid result to be spliced, and a second tooth region to be spliced with a target width is determined from the mouth scanning grid result to be spliced, feature points of the same target number and the same position are obtained in the first tooth region to be spliced and the second tooth region to be spliced, respectively, and a rigid body transformation matrix between the feature points in the first tooth region to be spliced and the feature points in the second tooth region to be spliced is determined, and the facial scanning grid result to be processed and the mouth scanning grid result are spliced according to the rigid body transformation matrix to obtain an initial splicing result, and the overlapping source scanning point cloud and target scanning point cloud in the initial splicing result are processed to obtain a rotation and translation matrix, and a rotation and translation matrix is obtained according to the rotation and translation matrix. The source scan point cloud is transformed into the coordinate system of the target scan point cloud, and the error between the transformed source scan point cloud and the target scan point cloud is determined until the error is less than or equal to the preset error threshold, and the target stitching result is obtained. The above technical solution solves the technical problem of difficult tooth stitching of mouth scan data and facial scan data. Through auxiliary stitching of facial posture and tooth posture, the accuracy of the initial stitching result is improved while ensuring the stitching efficiency, thereby improving the accuracy of the target stitching result, ensuring the success rate of the final scan data stitching, meeting the user's requirements for the accuracy of scan data stitching, and improving the user's scanning stitching experience.

FIG3 is a schematic diagram of the structure of a scan data splicing device provided by an embodiment of the present disclosure. The device can be implemented by software and/or hardware and can generally be integrated in an electronic device. As shown in FIG3 , the device includes:

The first acquisition module 301 is configured to acquire the facial posture corresponding to the user's facial scanning grid result;

The second acquisition module 303 is configured to acquire the tooth posture corresponding to the mouth scanning grid result;

The stitching module 303 is configured to stitch the facial scan mesh result and the mouth scan mesh result based on the facial pose and the tooth pose to obtain an initial stitching result.

Optionally, the first acquisition module 301 includes:

An acquisition unit is configured to acquire a facial image of a user, and recognize the facial image to obtain a tooth region and key facial feature points;

A projection unit is configured to project the tooth area and the key feature points of the face onto the facial scanning grid result to obtain the facial scanning grid result to be processed;

The calculation unit is configured to perform calculation based on the face scanning grid result to be processed to obtain the face position and pose corresponding to the face scanning grid result.

Optionally, the projection unit is specifically configured as:

Get the pixel coordinate points corresponding to the tooth area;

Get the feature coordinate points corresponding to the key feature points of the face;

According to the transformation relationship between the image coordinate system and the scanning camera coordinate system, the pixel coordinate point The feature coordinate points are projected onto the facial scanning grid result to obtain the facial scanning grid result to be processed.

Optionally, the computing unit is specifically configured as:

Determine a facial contour, a first feature area, and a second feature area based on the facial feature points;

determining a facial plane based on the facial contour and the first feature area;

Determine a first midline feature point and a second midline feature point based on the facial feature points, and determine a longitudinal direction of the facial plane based on the first midline feature point and the second midline feature point;

The longitudinal direction of the facial plane is adjusted based on the centroid direction of the facial contour and the centroid direction of the second feature area to obtain the facial pose corresponding to the facial scanning grid result.

Optionally, the second acquisition module 302 is specifically configured to:

Project the mouth scanning grid result onto a two-dimensional plane to obtain the two-dimensional coordinate points of the mouth;

Process the two-dimensional coordinate points of the mouth according to a preset processing method to obtain a two-dimensional curve;

Based on the dental arch line fitting method, the parameters corresponding to the two-dimensional curve are fitted to obtain the tooth posture.

Optionally, the splicing module 303 includes:

A coordinate unification unit is configured to unify the facial scanning mesh result and the oral scanning mesh result into the same coordinate system based on the facial posture and the tooth posture, so as to obtain the facial scanning mesh result to be spliced and the oral scanning mesh result to be spliced;

A determination unit is configured to determine a first tooth region to be spliced from a mesh result of a face scan to be spliced, and to determine a second tooth region to be spliced of a target width from a mesh result of a face scan to be spliced;

an acquisition and determination unit, configured to acquire the same target number and the same position of feature points in the first tooth region to be spliced and the second tooth region to be spliced, respectively, and determine a rigid body transformation matrix between the feature points in the first tooth region to be spliced and the feature points in the second tooth region to be spliced;

The splicing unit is configured to splice the face scanning mesh result to be processed and the mouth scanning mesh result according to the rigid body transformation matrix to obtain an initial splicing result.

Optionally, the coordinate unification unit is specifically configured as:

The facial pose and tooth pose determine the coordinate transformation matrix between the facial scan mesh result and the mouth scan mesh result;

The mouth scan grid result is converted to the coordinate system corresponding to the face scan grid result according to the coordinate conversion matrix; or, the face scan grid result is converted to the coordinate system corresponding to the mouth scan grid result according to the coordinate conversion matrix to obtain the face scan grid result to be spliced and the mouth scan grid result to be spliced.

Optionally, after obtaining the initial stitching result, it also includes:

A processing module is configured to process the overlapping source scanning point cloud and target scanning point cloud in the initial stitching result to obtain a rotation and translation matrix;

The stitching adjustment module is configured to transform the source scanning point cloud into the coordinate system of the target scanning point cloud according to the rotation and translation matrix, determine the error between the transformed source scanning point cloud and the target scanning point cloud, until the error is less than or equal to a preset error threshold, and obtain the target stitching result.

The scan data stitching device provided in the embodiment of the present disclosure can execute the scan data stitching method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

The embodiments of the present disclosure also provide a computer program product, including a computer program/instruction, which, when executed by a processor, implements the scan data stitching method provided by any embodiment of the present disclosure.

FIG4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. Referring specifically to FIG4 below, it shows a schematic diagram of the structure of an electronic device 400 suitable for implementing the embodiment of the present disclosure. The electronic device 400 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG4 is merely an example and should not impose any limitations on the functions and scope of use of the embodiment of the present disclosure.

As shown in FIG. 4 , the electronic device 400 may include a processing device (eg, a central processing unit, The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input/output (I/O) interface 405 is also connected to the bus 404.

Typically, the following devices may be connected to the I/O interface 405: input devices 406 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 408 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 409. The communication device 409 may allow the electronic device 400 to communicate wirelessly or wired with other devices to exchange data. Although FIG. 4 shows an electronic device 400 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the scan data stitching method of the embodiment of the present disclosure are executed.

It should be noted that the computer-readable medium of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a A computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains the facial posture corresponding to the user's facial scanning grid result, and obtains the tooth posture corresponding to the user's mouth scanning grid result, and splices the facial scanning grid result and the mouth scanning grid result based on the facial posture and the tooth posture to obtain an initial splicing result.

Computer program code for performing operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including but not limited to object-oriented programming languages. Programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages are also included. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and/or flow chart, and the combination of the square boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, constitute a limitation on the unit itself.

The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that can A program containing or storing for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

processor;

a memory for storing processor-executable instructions;

The processor is used to read executable instructions from the memory and execute the instructions to implement any scan data stitching method provided in the present disclosure.

According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium, the storage medium stores a computer program, and the computer program is used to execute any scan data stitching method provided by the present disclosure.

The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other.

Furthermore, although the operations are depicted in a particular order, this should not be construed as requiring that These operations may be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Certain features described in the context of a separate embodiment may also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment may also be implemented in multiple embodiments individually or in any suitable sub-combination.

Although the subject matter has been described in language specific to structural features and/or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Industrial Applicability

The scan data stitching method provided by the present invention improves the accuracy of the initial stitching result while ensuring the stitching efficiency through the auxiliary stitching of facial posture and tooth posture, thereby improving the final stitching success rate and having strong industrial practicability.

Claims (11)

A scanning data splicing method, comprising: Get the facial pose corresponding to the user's facial scan mesh result; Obtaining a tooth posture corresponding to a mesh result of a mouth scan of the user; The facial scan mesh result and the mouth scan mesh result are spliced based on the facial pose and the tooth pose to obtain an initial splicing result. According to the scan data stitching method of claim 1, the step of obtaining the facial pose corresponding to the facial scan grid result of the user comprises: Acquire a facial image of the user, and identify the facial image to obtain a tooth area and key facial feature points; Projecting the tooth area and the key feature points of the face onto the facial scanning grid result to obtain a facial scanning grid result to be processed; Calculation is performed based on the face scan grid result to be processed to obtain the face pose corresponding to the face scan grid result. According to the scan data stitching method of claim 2, projecting the tooth area and the key feature points of the face onto the facial scan grid result to obtain the facial scan grid result to be processed comprises: Obtaining pixel coordinate points corresponding to the tooth area; Obtaining feature coordinate points corresponding to the key feature points of the face; The pixel coordinate points and the feature coordinate points are projected onto the facial scanning grid result according to the transformation relationship between the image coordinate system and the scanning camera coordinate system to obtain the facial scanning grid result to be processed. According to the scan data stitching method of claim 2 or 3, the face scan grid result to be processed includes: facial feature points, and the calculation based on the face scan grid result to obtain the facial position pose corresponding to the face scan grid result includes: Determine a facial contour, a first feature area, and a second feature area based on the facial feature points; determining a facial plane based on the facial contour and the first feature area; Determine a first midline feature point and a second midline feature point based on the facial feature points, and determine a longitudinal direction of the facial plane based on the first midline feature point and the second midline feature point; The longitudinal direction of the facial plane is adjusted based on the centroid direction of the facial contour and the centroid direction of the second feature area to obtain a facial pose corresponding to the facial scanning grid result. According to the scan data stitching method according to any one of claims 1 to 4, the step of obtaining the tooth posture corresponding to the mouth scan grid result of the user comprises: Projecting the mouth scanning grid result onto a two-dimensional plane to obtain two-dimensional coordinate points of the mouth; Processing the two-dimensional coordinate points of the mouth according to a preset processing method to obtain a two-dimensional curve; The parameters corresponding to the two-dimensional curve are fitted based on the dental arch line fitting method to obtain the tooth posture. According to the scan data stitching method according to any one of claims 1 to 5, stitching the facial scan mesh result and the mouth scan mesh result based on the facial pose and the tooth pose to obtain an initial stitching result comprises: Based on the facial posture and the tooth posture, the facial scanning grid result and the mouth scanning grid result are unified into the same coordinate system to obtain the facial scanning grid result to be spliced and the mouth scanning grid result to be spliced; Determine a first tooth region to be spliced from the face scanning grid result to be spliced, and determine a second tooth region to be spliced with a target width from the face scanning grid result to be spliced; Acquire the same target number and the same position of feature points in the first tooth region to be spliced and the second tooth region to be spliced, respectively, and determine a rigid body transformation matrix between the feature points in the first tooth region to be spliced and the feature points in the second tooth region to be spliced; The facial scanning mesh result and the mouth scanning mesh result are spliced according to the rigid body transformation matrix to obtain an initial splicing result. According to the scan data stitching method of claim 6, unifying the facial scan mesh result and the mouth scan mesh result into the same coordinate system based on the facial pose and the tooth pose, comprises: Determine a coordinate transformation matrix between the facial scan grid result and the oral scan grid result based on the facial pose and the tooth pose; The mouth scan grid result is converted into a coordinate system corresponding to the face scan grid result according to the coordinate conversion matrix; or, the face scan grid result is converted into a coordinate system corresponding to the mouth scan grid result according to the coordinate conversion matrix. According to the scan data stitching method according to any one of claims 1 to 7, after obtaining the initial stitching result, the scan data stitching method further comprises: Processing the overlapping source scanning point cloud and target scanning point cloud in the initial stitching result to obtain a rotation and translation matrix; The source scan point cloud is transformed into the coordinate system of the target scan point cloud according to the rotation and translation matrix, and the error between the transformed source scan point cloud and the target scan point cloud is determined until the error is less than or equal to a preset error threshold, thereby obtaining a target stitching result. A scanning data splicing device, characterized by comprising: A first acquisition module is configured to acquire the facial posture corresponding to the user's facial scanning grid result; A second acquisition module is configured to acquire a tooth posture corresponding to a mesh result of the user's mouth scan; The splicing module is configured to splice the facial scan grid result and the mouth scan grid result based on the facial pose and the tooth pose to obtain an initial splicing result. An electronic device, characterized in that the electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the scan data stitching method described in any one of claims 1-8. A computer-readable storage medium, characterized in that the storage medium stores a computer program, and the computer program is used to execute the scan data stitching method described in any one of claims 1 to 8.