WO2025233370A1

WO2025233370A1 - A computer-implemented method for digital tooth representation

Info

Publication number: WO2025233370A1
Application number: PCT/EP2025/062409
Authority: WO
Inventors: Florin-Nicolae COFAR; Titus NICOLAE; Cristian-Nicolae PERJESCU; Nicolae-Marcel SURDU; Cristian-Florin DIACONESCU; Radu-Florin SARGHE; Mihai SIMONIA
Original assignee: Smilecloud Srl
Current assignee: Smilecloud Srl
Priority date: 2024-05-06
Filing date: 2025-05-06
Publication date: 2025-11-13
Anticipated expiration: 2026-11-06
Also published as: WO2025233357A1

Abstract

A computer-implemented method involving obtaining digital representations of a patient's teeth through facial imaging and intraoral scanning. The first segmentation involves obtaining digital representations of teeth from facial images and storing them as a first digital file. The second segmentation involves obtaining digital representations of upper and/or lower teeth from intraoral scans and storing them as a second digital file. These files are aligned and overlaid to create an overlay image that can be modified by a user. The modification is applied to the overlay image while maintaining the coordinates of the modified tooth in both the first and second digital files. This method allows for more efficient and accurate digital representation of a patient's teeth, which can be used for various dental procedures.

Description

A computer-implemented method for digital tooth representation

Field of Invention

The present invention relates to the field of digital dentistry, and more particularly to techniques for digitally representing and modifying a patient's teeth.

Background

In the field of digital dentistry, several software tools are currently available that allow dental technicians to digitally modify an orthodontic region of a patient. These tools start with a single digital file, which is modified to reflect the desired changes to the patient's teeth. The modified file is then output for use in treatment planning and other purposes.

Despite the benefits of these software tools, they have several significant limitations. For one, they lack flexibility in that they do not allow dental technicians and dentists to operate on the same digital file or modify the file once it has been output. This lack of flexibility can be particularly problematic when it comes to making changes to the patient's treatment plan. Additionally, the current software tools only allow for starting anew from the first single file, rather than allowing for modifications to be made to existing files. Finally, the current software tools lack interoperability, which can make it difficult to share files between different software applications.

Summary

It is therefore an objective of the present invention to provide a computer-implemented method that is more flexible, allows a greater degree of interoperability, and is computationally more efficient.

According to a first aspect there is provided a computer-implemented method for digital tooth representation, the method comprising:

- obtaining at least one facial image of a patient, said at least one facial image comprising a head of the patient and a smile comprising one or more teeth;

- obtaining at least one first digital representation of the one or more teeth comprised by the smile, by performing a first segmentation on the obtained at least one facial image, wherein during the first segmentation each first digital representation of the one or more teeth comprises one or more first parameters including a parameter representative for a coordinate of the tooth;

- storing the obtained at least one first digital representation of the one or more teeth as a first digital file; - obtaining at least one intraoral scan of the patient, the at least one intraoral scan comprising at least one of a portion of an upper jaw comprising one or more upper teeth of the patient and a portion of a lower jaw comprising one or more lower teeth of the patient;

- obtaining at least one second digital representation of the plurality of teeth comprised by the smile, by performing a second segmentation on the obtained at least one intraoral scan of the patient, wherein during the second segmenting each second digital representation of the one or more upper and/or one or more lower teeth comprises one or more second parameters including a parameter representative for a coordinate of the tooth;

- storing the obtained at least one second digital representation of the plurality of teeth comprised by the smile as a second digital file, wherein the second digital file is distinct from the first digital file;

- aligning the at least one first digital representation and the at least one second digital representation;

- overlaying the aligned at least one first digital representation and at least one second digital representation to obtain an overlay image comprising the aligned at least one facial image and the at least one intraoral scan;

- allowing a user to modify at least a dimension of at least one tooth in the at least one first digital representation or the at least one second digital representation;

- applying the modification to the overlay image while maintaining the coordinates of the modified tooth in at least the first digital file and the second digital file.

A computer-implemented method for digital tooth representation refers to a process for creating a digital model or representation of a tooth using a computer system. This method can be used in dentistry to create 3D models of teeth for a variety of purposes, such as designing dental prosthetics, planning orthodontic treatments, or analyzing tooth wear patterns. The first step involves obtaining at least one facial image of the patient, which includes a view of the head and the smile, including one or more teeth. The facial image is used to obtain at least one first digital representation of the one or more teeth by performing a first segmentation. During this process, the software analyzes the facial image and identifies the teeth within the smile. Each tooth can then be assigned a first digital representation, which includes one or more first parameters that describe the tooth's characteristics, such as its size, shape, and position within the smile. When obtaining the first digital representation of the teeth, each tooth can be independently represented in the digital representation. This means that each tooth is assigned a separate digital representation that includes its unique characteristics, such as its size, shape, and position within the mouth or that a plurality of teeth are represented in a digital representation. In this context it is preferred that each tooth is assigned a first digital representation. Having each tooth represented independently in the first digital representation can provide dental professionals with a more detailed and accurate view of the patient's teeth. This can be particularly useful when planning treatments such as orthodontics or dental implants, where the position and alignment of each tooth is critical. One particularly important first parameter that is included in the first digital representation of each tooth is a coordinate of the tooth. This parameter represents the location of the tooth within the facial image, and it is used to track the position of the tooth throughout the treatment planning process.

After obtaining the first digital representation of the teeth, the next step is to store this information as a first digital file. This first digital file contains the digital model of the patient's teeth, including the individual digital representations of each tooth and any associated parameters.

Storing the first digital representation of the teeth as a first digital file is important because it allows dental professionals to access and reference the information at any time. The first digital representation can be saved on a computer or in a cloud-based system and can be easily shared and transferred between different dental professionals involved in the patient's treatment.

After obtaining the first digital representation of the teeth from the facial image, the next step involves obtaining at least one intraoral scan of the patient. The intraoral scan may include a portion of the upper jaw comprising one or more upper teeth, or a portion of the lower jaw comprising one or more lower teeth. The intraoral scan may include both a portion of the upper jaw comprising one or more upper teeth, and a portion of the lower jaw comprising one or more lower teeth. Using the intraoral scan, a second segmentation is performed to obtain at least one second digital representation of the plurality of teeth comprised by the smile. During the second segmentation, the software analyzes the intraoral scan and identifies the teeth within the scanned area. Each tooth is then assigned a second digital representation, which includes one or more second parameters. These parameters at least include a parameter representative for a coordinate of the tooth but the parameters may also include the size, or shape of the tooth. By obtaining a second digital representation of the teeth from the intraoral scan, dental professionals can create a more comprehensive digital model of the patient's teeth. The second digital representation can provide additional information about the structure and alignment of the teeth and can also help identify any issues or conditions that may not have been visible in the facial image. After obtaining the at least one second digital representation of the teeth from the intraoral scan, the next step is to store this information as a second digital file. This file contains the digital model of the patient's teeth from the intraoral scan, including the individual digital representations of each tooth and any associated parameters. Storing the second digital representation of the teeth as a second digital file is beneficial because it allows dental professionals to access and reference the information at any time. The second digital file can be saved on a computer or in a cloud-based system. It is important to note that the second digital file is distinct from the first digital file. This means that the information obtained from the facial image and the intraoral scan are kept separate. By storing the information in separate files, dental professionals can more easily analyze and compare the two sets of data to create a more accurate and comprehensive digital model of the patient's teeth.

After obtaining the first digital file and the second digital file, the next step is to align them. This involves matching the two sets of data such that they are aligned with each other. This step is important to create a comprehensive digital model of the patient's teeth that incorporates information from both the facial image and the intraoral scan. Once the first and second digital representations are aligned, the method overlays them to create an overlay image. This overlay image combines the digital models from the first and second digital files and provides a more detailed and accurate representation of the patient's teeth. In this way, the overlay image comprises the aligned at least one first digital representation and at least one second digital representation and may also include additional information such as the patient's facial features or the relevant anatomical structures of the orthodontic region.

After creating the overlay image, the next step is to allow a user, such as a dental professional, to modify at least a dimension of at least one tooth in the first digital representation or the second digital representation. This modification may involve adjusting the size, shape, or position of the tooth to better align it with the surrounding teeth or to correct any issues or conditions.

Once the modification is made, it is applied to the overlay image, which allows the dental professional to visualize the impact of the modification on the overall structure of the teeth and surrounding structures. The computer-implemented method maintains the coordinates of the modified tooth in at least the first digital file and the second digital file. This ensures that any modifications made to the digital representation of the tooth are accurately applied to both files and that the digital model remains accurate and consistent throughout the treatment planning process.

By allowing a user to modify at least a dimension of at least one tooth in the digital representations and applying the modification to the overlay image, dental professionals can more accurately plan and execute dental treatments. This approach can improve the accuracy and precision of treatment, as well as reduce the risk of errors or complications.

The separation of the digital files obtained from the facial image and the intraoral scan is important from a computational standpoint because it allows for more efficient data processing and analysis. Facial images and intraoral scans are typically large and complex data sets that require significant computational resources to analyze and process. By separating the data into different files, the computational load can be distributed across multiple processors or systems, allowing for faster and more efficient analysis or simply reduce the computational load by limiting the computational action to the file on which the modification is performed.

Furthermore, separating the digital files can also improve the interoperability of different tools and technologies used in dental practice. For example, the first digital file may be used with CAD software to design dental restorations or orthodontic appliances, while the second digital file may be used with 3D printing technology to create physical models for treatment planning. By separating the data, dental professionals can more easily integrate different tools and technologies to create a more comprehensive and accurate digital model of the patient's teeth.

In summary, a computer-implemented method for digital tooth representation is provided that is flexible because it allows dental professionals to modify at least one dimension of at least one tooth in the digital representations and apply the modification to the overlay image while maintaining the coordinates of the modified tooth in both digital files. This enables dental professionals to more accurately plan and execute dental treatments, improving the precision and efficiency of the process.

Additionally, the method allows for a greater degree of interoperability between different tools and technologies used in dental practice. The separation of the digital files obtained from the facial image and the intraoral scan enables dental professionals to use different tools and technologies to create a more comprehensive and accurate digital model of the patient's teeth.

Finally, the method is computationally more efficient because it separates the large and complex data sets obtained from the facial image and the intraoral scan into different files, enabling the computational load to be distributed across multiple processors or systems, and reducing the computational load by limiting the computational action to the file on which the modification is performed. This makes the process faster and more efficient, enabling dental professionals to analyze and process the data more quickly and accurately.

Preferably, the computer-implemented method comprises showing the obtained at least one facial image of the patient as a first image and showing the obtained at least one intraoral scan of the patient as a second image, wherein the first image and the second image are distinct images from each other.

It may be provided that the first image is a two-dimensional image of the patient's head and smile, and the second image is a three-dimensional image of the patient's upper and/or lower jaw. In this arrangement, one advantage is that dental professionals can more easily visualize and analyze the patient's teeth and surrounding structures from both the facial image and the intraoral scan. This can improve the accuracy and precision of treatment planning, as well as reduce the risk of errors or complications. It may be further provided that the first image and the second image are displayed side-by-side to create a more comprehensive and detailed view of the patient's teeth. In this arrangement, one advantage is that dental professionals can more easily compare and contrast the two sets of data to identify any issues or conditions that may not have been visible in one image alone. This can lead to more accurate and comprehensive treatment planning and execution.

It may be provided that the computer-implemented method further comprises displaying the first image, the second image, and the overlay image as distinct images from each other. In this arrangement, one advantage is that dental professionals can more easily compare and contrast the different images to identify any issues or conditions that may require attention and can make more informed treatment decisions. It may be further provided that the first image, the second image, and the overlay image are displayed side-by-side or in separate windows on a computer monitor or other display device. In this arrangement, one advantage is that dental professionals can more easily analyze and compare the different images to create a more accurate and comprehensive digital model of the patient's teeth.

Preferably, the computer-implemented method comprises obtaining at least one intraoral scan of the patient, which includes obtaining an intraoral scan of a portion of an upper jaw comprising one or more upper teeth of the patient and an intraoral scan of a portion of a lower jaw comprising one or more lower teeth of the patient. It may be provided that the step of obtaining at least one second digital representation comprises performing a separate second segmentation on the intraoral scan of the portion of the upper jaw and a separate second segmentation on the intraoral scan of the portion of the lower jaw. In this arrangement, one advantage is that dental professionals can create a more comprehensive digital model of the patient's teeth by obtaining separate second digital representations for the upper and lower teeth. This can improve the accuracy and precision of treatment planning, as well as reduce the risk of errors or complications.

Preferably, the computer-implemented method further comprises displaying the intraoral scan of the portion of the upper jaw as the second image and the intraoral scan of the portion of the lower jaw as a third image. In this arrangement, one advantage is that dental professionals can more easily visualize and analyze the patient's upper and lower teeth separately, which can lead to more accurate and comprehensive treatment planning and execution. It may be further provided that the first image, the second image, the third image, and the overlay image are displayed as distinct images from each other, such as side-by-side or in separate windows on a computer monitor or other display device. In this arrangement, one advantage is that dental professionals can more easily compare and contrast the different images to identify any issues or conditions that may require attention and can make more informed treatment decisions.

Preferably, the computer-implemented method stores each modification as an independent modification. In this arrangement, one advantage is that dental professionals can more easily track and analyze the modifications made to the digital model of the patient's teeth over time. This can provide valuable information about the effectiveness of the treatment plan and can help identify any issues or conditions that may require attention. It may be further provided that each independent modification includes information about the modified tooth, such as the modified dimension, the date of the modification, and the identity of the dental professional who made the modification. In this arrangement, one advantage is that dental professionals can more easily track and analyze the modifications made to the digital model of the patient's teeth and can more easily collaborate with other dental professionals involved in the patient's treatment.

Preferably, each modification is revertible. In this arrangement, one advantage is that dental professionals can more easily undo or revert a modification if it is found to be unnecessary or if it negatively impacts the overall structure or function of the teeth. This can help reduce the risk of errors or complications and improve the accuracy and precision of treatment planning and execution. It may be further provided that the revertible modifications are stored in a separate file or database, which can be accessed and updated as needed. In this arrangement, one advantage is that dental professionals can more easily track and analyze the modifications made to the digital model of the patient's teeth over time and can make informed decisions about the treatment plan.

Preferably, the computer-implemented method further comprises obtaining at least one Computed Tomography, CT, image of the orthodontic region of the patient. It may be provided that the CT image is used to obtain at least one third digital representation of the one or more teeth by performing a third segmentation. During this segmentation, each third digital representation of the one or more teeth comprises one or more third parameters, including a parameter representative for a coordinate of the tooth. It may be further provided that the third digital representation is stored as a third digital file, which is distinct from the first digital file and the second digital file. In this arrangement, one advantage is that dental professionals can create a more comprehensive and detailed digital model of the patient's teeth by incorporating information from the CT image into the digital model. The third digital file can provide additional information about the structure and alignment of the teeth and can also help identify any issues or conditions that may not have been visible in the facial image or intraoral scan.

More preferably, the computer-implemented method further comprises generating a root structure in the overlay image by aligning the at least one first digital representation, the at least one second digital representation and the at least one third digital representation. The root structure can provide important information about the position and alignment of the teeth, as well as any underlying conditions or issues that may alfect the treatment plan. During the alignment, the crown shown in the at least one third digital representation is aligned with a crown shown in the at least one first digital representation or the at least one second digital representation. The root structure is generated by aligning the at least one first digital representation, the at least one second digital representation, and the at least one third digital representation. The root structure of the at least one third digital representation shown in the aligned at least one first digital representation, the at least one second digital representation, and the at least one third digital representation is used as the root structure of the patient. In this arrangement, one advantage is that dental professionals can more accurately visualize and analyze the position and alignment of the teeth and surrounding structures, which can lead to more accurate and comprehensive treatment planning and execution.

Again, it may be provided that the computer-implemented method maintains the coordinates of the modified tooth in the first digital file, the second digital file, and the third digital file during the step of applying the modification to the overlay image. In this arrangement, one advantage is that dental professionals can ensure that any modifications made to the digital representation of the tooth are accurately applied to all three digital files, which can provide a more comprehensive and accurate digital model of the patient's teeth and surrounding structures. This can improve the accuracy and precision of treatment planning, reduce the risk of errors or complications, and ultimately lead to better patient outcomes.

Preferably, the computer-implemented method further comprises showing the obtained at least one CT image as a fourth image. One advantage is that dental professionals can more easily visualize and analyze the patient's teeth and surrounding structures from different perspectives and with different levels of detail. The CT image can provide additional information about the bone structure and density of the teeth, which can be useful in planning treatments such as dental implants or orthognathic surgery. It may be further provided that the first image, the second image, optionally the third image, the fourth image, and the overlay image are displayed as distinct images from each other, such as side-by-side or in separate windows on a computer monitor or other display device. In this arrangement, one advantage is that dental professionals can more easily compare and contrast the different images to identify any issues or conditions that may require attention and can make more informed treatment decisions.

Preferably, the computer-implemented method comprises aligning the at least one first digital representation and the at least one second digital representation by determining a center point of each tooth in the at least one first digital representation, rotating the at least one second digital representation to substantially match the perspective rotation of the at least one first digital representation, projecting a center point of each tooth in the rotated at least one second digital representation on a plane in line with the at least one first digital representation, and point matching the determined center point of each tooth in the at least one first digital representation to a projected center point of each tooth on the plane to align the at least one first digital representation and the at least one second digital representation. The determination of the center point of each tooth in the at least one first digital representation and the subsequent rotation of the at least one second digital representation to substantially match the perspective rotation of the at least one first digital representation can provide a more accurate alignment of the two digital representations. This can lead to a more precise and comprehensive digital model of the patient's teeth, enabling dental professionals to more accurately plan and execute dental treatments. The projection of a center point of each tooth in the rotated at least one second digital representation on a plane in line with the at least one first digital representation and the subsequent point matching of the determined center point of each tooth in the at least one first digital representation to a projected center point of each tooth on the plane can further enhance the accuracy of the alignment process. This approach leads to a substantially easier computational calculation than aligning the center points of a 3D model to those of a 2D model, enabling faster and more efficient processing of the data. This can lead to a more efficient and streamlined treatment planning process, improving the overall quality and accuracy of the dental treatment.

Preferably, the computer-implemented method further comprises simulating a movement of the patient's jaw using the aligned at least one first digital representation and the at least one second digital representation based on anatomical averages of jaw and tooth moment to position the at least one first digital representation and the at least one second digital representation with respect to each other. This simulation can provide valuable information about the movement of the patient's jaw, which can be used to plan and execute dental treatments such as orthodontics or dental implants. By using anatomical averages of jaw and tooth moment to position the at least one first digital representation and the at least one second digital representation with respect to each other, dental professionals can create a more accurate and precise digital model of the patient's teeth and surrounding structures. This approach can improve the accuracy and precision of dental treatments and reduce the risk of errors or complications. By simulating the movement of the patient's jaw, dental professionals can more accurately predict the impact of various treatments on the overall structure and function of the teeth and surrounding structures, enabling more informed treatment decisions.

Preferably, the computer-implemented method further comprises deriving one or more occlusion parameters based on the simulated movement of the patient's jaw. The occlusion parameters derived from the simulated movement of the patient's jaw can provide valuable information about the alignment and function of the patient's teeth and surrounding structures. By using the simulated movement of the patient's jaw to derive occlusion parameters, dental professionals can more accurately identify any issues or conditions that may require attention and can make more informed treatment decisions. This approach can improve the accuracy and precision of dental treatments and reduce the risk of errors or complications. By deriving occlusion parameters based on the simulated movement of the patient's jaw, dental professionals can more accurately predict the impact of various treatments on the overall structure and function of the teeth and surrounding structures, enabling more informed treatment decisions.

Preferably, the computer-implemented method further comprises simulating a movement of the patient's jaw using the aligned at least one first digital representation, the at least one second digital representation, and the at least one third digital representation. The simulation of the movement of the patient's jaw using the aligned at least one first digital representation, the at least one second digital representation, and the at least one third digital representation can provide a more comprehensive and accurate digital model of the patient's teeth and surrounding structures. By incorporating information from the CT image into the simulation, dental professionals can create a more detailed and accurate model of the patient's teeth and surrounding structures, enabling more informed treatment decisions. This approach can improve the accuracy and precision of dental treatments and reduce the risk of errors or complications. By simulating the movement of the patient's jaw using the aligned at least one first digital representation, the at least one second digital representation, and the at least one third digital representation, dental professionals can more accurately predict the impact of various treatments on the overall structure and function of the teeth and surrounding structures.

According to yet another aspect of the present invention, there is provided a computer program product comprising a computer-executable program of instructions for performing, when executed on a computer, the steps of the method of any one of the method embodiments described above.

It will be understood by the skilled person that the features and advantages disclosed hereinabove with respect to embodiments of the method may also apply, mutatis mutandis, to embodiments of the computer program product.

According to yet another aspect of the present invention, there is provided a digital storage medium encoding a computer-executable program of instructions to perform, when executed on a computer, the steps of the method of any one of the method embodiments described above.

It will be understood by the skilled person that the features and advantages disclosed hereinabove with respect to embodiments of the method may also apply, mutatis mutandis, to embodiments of the digital storage medium.

According to yet another aspect of the present invention, there is provided a device programmed to perform a method comprising the steps of any one of the methods of the method embodiments described above.

According to yet another aspect of the present invention, there is provided a method for downloading to a digital storage medium a computer-executable program of instructions to perform, when executed on a computer, the steps of the method of any one of the method embodiments described above.

It will be understood by the skilled person that the features and advantages disclosed hereinabove with respect to embodiments of the method may also apply, mutatis mutandis, to embodiments of the method for downloading.

Brief description of the figures The accompanying drawings are used to illustrate presently preferred non-limiting exemplary embodiments of devices of the present invention. The above and other advantages of the features and objects of the present invention will become more apparent and the present invention will be better understood from the following detailed description when read in conjunction with the accompanying drawings, in which:

Figure 1 shows a photograph of an exemplary embodiment of an interface for a computer- implemented method 1000 for digital tooth representation

Figure 2 schematically illustrates a flow diagram of an exemplary embodiment of the first and second digital representation;

Figure 3 shows a photograph of an exemplary embodiment of modifying a teeth in a digital representation;

Figure 4 is a front view of a display screen or portion thereof with graphical user interface showing a design of the primary user interface display;

Figure 5 is a front view of a display screen or portion thereof with graphical user interface, GUI, showing a design of a secondary user interface display;

Figure 6 is a front view of a display screen or portion thereof with graphical user interface, GUI, showing a design of a tertiary user interface display;

Figure 7 is a front view of a display screen or portion thereof with graphical user interface including a drop down panel describing one or more user workflows such as smile design, blueprint creation, review, and comparison;

Figure 8A, 8B and 8C shows front view of a display screen or portion thereof with graphical user interface illustrating a compare workflow;

Figure 9A, 9B and 9C shows front view of a display screen or portion thereof with graphical user interface illustrating a smile design;

Figures 10A-10K illustrates a front view of GUIs designed for creating a 3D blueprint of dental structures using selected teeth shapes;

Figures 11A-F illustrate respective line drawings of the GUI renders shown in figures 10A, 10B, 10F, 10J and 10K.

Description of embodiments

The patent application may contain at least one drawing executed in color. Copies of this patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

Figure 1 shows an exemplary embodiment of an interface for a computer-implemented method 1000 for digital tooth representation. A computer-implemented method 1000 for digital tooth representation refers to a technique that uses computer software to create a virtual model of a patient's teeth. This digital representation can be used for various purposes, such as creating dental restorations, orthodontic treatments, and dental implants. The digital model can be manipulated to simulate dilferent treatment options and to help the dentist or orthodontist plan and execute the treatment more accurately and efficiently as will be elaborated here below.

The computer-implemented method 1000 comprises obtaining 100 at least one facial image of a patient, said at least one facial image comprising a head of the patient and a smile comprising one or more teeth. A facial image is a visual representation of a person's face, which may include the head and neck, and can also include the upper portion of the torso. Facial images can be captured using various methods such as photography, digital scanning, or video. Photography is the most common method and involves taking a still image of the patient's face using a camera. The method may use more than one facial image, for example multiple different perspectives of the same patients face or a plurality of frames of a video. A video captures a moving visual representation of the patient's face and may provide additional information about the patient's facial movements and expressions. A video can provide a more comprehensive view of the patient's face and smile, allowing for a more accurate analysis of their dental condition or to create a simulation of motion as will be elaborated below. In figure 1 a photograph of a young woman is used as a facial image in which the facial image shown the young woman smiling and showing her upper and lower teeth from the front is shown. The image is taken from a frontal perspective, with the young woman's face and smile directly facing the camera. This perspective is helpful in accurately assessing the alignment and spacing of the teeth, as well as the shape and contour of the lips and gums.

The method 1000 further comprises obtaining 200 at least one first digital representation of the one or more teeth comprised by the smile, by performing a first segmentation on the obtained at least one facial image, wherein during the first segmentation each first digital representation of the one or more teeth comprises one or more first parameters including a parameter representative for a coordinate of the tooth. The first digital representation obtained by performing a first segmentation on the facial image could be a 2D or 3D digital model of the patient's teeth. In a 2D digital model, the segmentation process would identify the boundaries of each tooth in the facial image and create a digital representation of each tooth. This digital representation could include information about the shape, size, and position of each tooth.

The method 1000 involves performing a first segmentation on the facial image to isolate the teeth and create a digital representation of each tooth. During this segmentation, one or more first parameters are identified and assigned to each tooth, including at least a parameter that represents the coordinate of the tooth within the overall image. A parameter that represents the coordinate of the tooth within the overall image is a specific value or set of values that identify the location of the tooth within the digital representation of the facial image. The coordinate can include two or more values that indicate the position of the tooth in relation to the X, Y, and Z axes of the image. For example, the X coordinate may represent the horizontal position of the tooth within the image, while the Y coordinate may represent the vertical position. The Z coordinate may represent the depth or thickness of the tooth, depending on the type of digital representation being used. The coordinate parameter is important because it allows for accurate identification and analysis of each tooth within the digital representation. By knowing the precise location of each tooth, dentists and orthodontists can assess its alignment, spacing, and other characteristics and develop a treatment plan that is tailored to the specific needs and goals of the patient. In addition to the parameter that represents the coordinate of the tooth within the overall image, there may be other parameters that can be used to describe the characteristics of the teeth and gums in the digital representation. Some of these parameters may include: Tooth size and shape which are parameters that describe the size and shape of each tooth, such as length, width, and curvature. Tooth color and texture which are parameters that describe the color and texture of each tooth, such as hue, saturation, and brightness. Gum health which are parameters that describe the health and condition of the gums, such as gum line height, gum tissue thickness, and presence of gum disease. Alignment and spacing which are parameters that describe the alignment and spacing of the teeth, such as the distance between each tooth and the angle at which they are positioned. Occlusion which is a parameter that describe the way the upper and lower teeth come together when the patient bites down, such as overbite or underbite.

It is noted that the segmentation process is performed using a method that can identify and isolate specific features of the facial image, such as the teeth and gums. This method uses algorithms to analyze the image and identify patterns that correspond to the teeth and other dental structures. Steps of executing the segmentation of the teeth are known as such and will only be briefly mentioned for the sake of conciseness. There are several methods for performing segmentation, including manual segmentation, semi-automated segmentation, and fully automated segmentation. Manual segmentation involves manually selecting the teeth and other dental structures using specialized software or tools. This process is time-consuming and labor-intensive but can be very precise and accurate when performed by a skilled technician or dentist. Semiautomated segmentation involves using software that can identify potential dental structures in the image but requires input and guidance from a technician or dentist to ensure accuracy and precision. Fully automated segmentation involves using software that can automatically identify and isolate the teeth and other dental structures without any human intervention. This method is typically faster and more efficient than manual or semi-automated segmentation but may be less precise or accurate in certain cases. The method 1000 preferably uses the fully automated segmentation method to segment the facial image. This process allows for a more precise analysis of the teeth and gums, as each tooth can be analyzed and evaluated independently of the others. The first parameters can be used to accurately identify and locate each tooth within the image, and to assess its alignment, spacing, and other characteristics.

Storing the first digital representation of the teeth as a first digital file is important because it allows dental professionals to access and reference the information at any time. The first digital file can be saved on a computer or in a cloud-based system, and can be easily shared and transferred between dilferent dental professionals involved in the patient's treatment.

The method 1000 further comprises obtaining 300, 300A, 300B at least one intraoral scan of the patient, the at least one intraoral scan comprising at least one of a portion of an upper jaw comprising one or more upper teeth of the patient and a portion of a lower jaw comprising one or more lower teeth of the patient. An intraoral scan is a digital scan of the teeth and gums taken from inside the patient's mouth using specialized equipment. Intraoral scanners typically use a wand-like device that is inserted into the patient's mouth and moved around to capture images of the teeth and gums from various angles. The scanner uses a combination of light and cameras to capture images of the teeth and gums, which are then processed by specialized software to create a digital model of the patient's mouth. This digital model can be used to create a highly detailed and accurate representation of the teeth and gums, which can be used for further analysis and treatment planning. Intraoral scans are commonly used in dentistry and orthodontics for a variety of purposes, including creating digital impressions for restorations such as crowns and bridges, planning and monitoring orthodontic treatment, and assessing the overall health and condition of the teeth and gums. The step of obtaining 300 at least one intraoral scan of the patient does not need to be performed after the step of obtaining 100 at least one facial image or obtaining 200 at least one first digital representation of the one or more teeth. The step of obtaining 300 at least one intraoral scan of the patient can be performed simultaneously or before the steps of obtaining 100 at least one facial image or obtaining 200 at least one first digital representation of the one or more teeth. In figure 1 , the intraoral scan of the patient comprises at least one of a portion of an upper jaw comprising one or more upper teeth shown as 300A and the intraoral scan of the patient comprises a portion of a lower jaw comprising one or more lower teeth of the patient shown as 300B. Intraoral scans can be obtained by scanning the patient at the time of the appointment, but they can also be uploaded from a previous scan date. In the context of the method, it is preferred that the facial image and the intraoral scan are taken at the same time. This is because the digital models created from the facial image and the intraoral scan are more accurate and precise when taken at the same time. When the facial image and the intraoral scan are taken at the same time, they can be combined to create a more comprehensive digital representation of the patient's teeth and gums. This can provide a more accurate and detailed view of the patient's dental condition, which can be used for further analysis and treatment planning. In addition, taking the facial image and intraoral scan at the same time can help to reduce the risk of discrepancies or errors that may occur when the scans are taken at dilferent times. This can help to ensure that the digital models accurately reflect the patient's dental condition and provide a more effective basis for treatment planning. It is noted that the facial image is a 2D image that captures the patient's face and smile from a frontal perspective, while the intraoral scan is a 3D image that captures the teeth and gums from inside the patient's mouth. The difference between 2D and 3D imaging is important because it affects the level of detail and accuracy that can be obtained from the image. A 2D image provides a flat representation of the patient's face and smile, while a 3D image provides a more detailed and accurate representation of the teeth and gums, allowing for more precise analysis and treatment planning. While the facial image and intraoral scan provide different types of information, they can be combined to create a more comprehensive digital representation of the patient's teeth and gums, as will be elaborated below.

Next, the method 1000 comprises obtaining (not shown) at least one second digital representation of the plurality of teeth comprised by the smile, by performing a second segmentation on the obtained at least one intraoral scan of the patient, wherein during the second segmenting each second digital representation of the one or more upper and/or one or more lower teeth comprises one or more second parameters including a parameter representative for a coordinate of the tooth. Similar to the first segmentation performed on the facial image, the second segmentation is performed using a method that can identify and isolate specific features of the intraoral scan, such as the teeth and gums. The method uses algorithms and machine learning techniques to analyze the intraoral scan and identify patterns that correspond to the teeth and other dental structures. Again, the method for segmenting the intraoral scan is known to the skilled person and will not be elaborated further. The second parameter that represents the coordinate of the tooth is an important aspect of the second digital representation, as it allows for accurate identification and analysis of each tooth within the digital representation of the intraoral scan. The coordinate typically includes one or more values that indicate the position of the tooth in relation to the X, Y, and Z axes of the intraoral scan.

The step of obtaining at least one second digital representation may involve performing a separate second segmentation on the intraoral scan of the portion of the upper jaw and a separate second segmentation on the intraoral scan of the portion of the lower jaw. In this arrangement, dental professionals can create a more comprehensive digital model of the patient's teeth by obtaining separate second digital representations for the upper and lower teeth. This allows for a more detailed analysis and treatment planning, as the digital representations of the upper and lower teeth can be analyzed independently and in relation to each other. Separating the second segmentation of the intraoral scan into separate segments for the upper and lower teeth also reduces the risk of errors or complications that may occur when analyzing both portions together. This also allows dental professionals to focus on specific areas of the mouth, which can lead to more accurate and precise treatment planning.

After the second segmentation is performed on the intraoral scan, the resulting digital representation of the teeth and gums is stored as a second digital file. This file is distinct from the first digital file, which contains the digital representation of the teeth and gums created from the facial image. Storing the second digital file separately from the first digital file is important because it allows for easier access and manipulation of the digital representations of the teeth and gums. Separating the files also allows dental professionals to analyze and compare the digital representations of the teeth and gums separately, which can be helpful in developing a treatment plan that addresses all aspects of the patient's dental condition. The separation of the digital files obtained from the facial image and the intraoral scan is important from a computational standpoint because it allows for more efficient data processing and analysis. Facial images and intraoral scans are typically large and complex data sets that require significant computational resources to analyze and process. By separating the data into different files, the computational load can be distributed across multiple processors or systems, allowing for faster and more efficient analysis or simply reduce the computational load by limiting the computational action to the file on which the modification is performed. Furthermore, separating the digital files can also improve the interoperability of different tools and technologies used in dental practice. For example, the first digital file may be used with CAD software to design dental restorations or orthodontic appliances, while the second digital file may be used with 3D printing technology to create physical models for treatment planning. By separating the data, dental professionals can more easily integrate different tools and technologies to create a more comprehensive and accurate digital model of the patient's teeth.

After obtaining the first digital file and the second digital file, the next step is aligning the at least one first digital representation and the at least one second digital representation. This involves matching the first digital representation and the second digital representation such that they are aligned with each other. The step of aligning the at least one first digital representation and the at least one second digital representation will be further elaborated with respect to figure 2. This step creates a comprehensive digital model of the patient's teeth that incorporates information from both the facial image and the intraoral scan. Once the first and second digital representations are aligned, the method overlays 400 them to create an overlay image. In figure 1 , the overlay image 400 combines the digital models from the first and second digital files and provides a more detailed and accurate representation of the patient's teeth. In this way, the overlay image comprises the aligned at least one first digital representation and at least one second digital representation and may also include additional information such as the patient's facial features or the relevant anatomical structures of the orthodontic region. The step of aligning the at least one first digital representation and the at least one second digital representation will be further elaborated with respect to figure 2.

After creating the overlay image, the next step is to allow a user, such as a dental professional, to modify at least a dimension of at least one tooth in the first digital representation or the second digital representation. This modification, shown in figure 3, may involve adjusting the size, shape, or position of the tooth to better align it with the surrounding teeth or to correct any issues or conditions as an example. Once the modification is made, it is applied to the overlay image, which allows the dental professional to visualize the impact of the modification on the overall structure of the teeth and surrounding structures. The computer-implemented method maintains the coordinates of the modified tooth in at least the first digital file and the second digital file. In the context of the computer-implemented method, maintaining the coordinates of the modified tooth in at least the first digital file and the second digital file means that the changes made to the digital model are saved and recorded in a way that preserves the accuracy and integrity of the original digital files. This means that when modifications are made to the digital model of the patient's teeth, the coordinates of the modified tooth are saved and recorded in both the first and second digital files. This ensures that any changes made to the digital model are accurately reflected in both digital files. Maintaining the coordinates of the modified tooth is important for accurate treatment planning and execution. By preserving the accuracy and integrity of the original digital files, dental professionals can more effectively plan and execute dental treatments, leading to better outcomes for the patient. This ensures that any modifications made to the digital representation of the tooth are accurately applied to both files and that the digital model remains accurate and consistent throughout the treatment planning process. By allowing a user to modify at least a dimension of at least one tooth in the digital representations and applying the modification to the overlay image, see figure 3, dental professionals can more accurately plan and execute dental treatments. This approach can improve the accuracy and precision of treatment, as well as reduce the risk of errors or complications. Because the first and second digital representations are stored as separate files and each tooth is at least represented by a set of coordinates, the method allows dental professionals to flexibly modify one or more teeth to better align with the surrounding teeth or to correct any issues or conditions. The separation of the files and maintenance of the coordinates allows dental professionals to make precise modifications to the digital representations of the teeth and gums, while ensuring that the modifications are accurately represented in the final treatment plan.

For example, if a dental professional determines that a patient needs orthodontic treatment to correct a malocclusion, they can use the digital representations of the teeth and gums to make precise adjustments to the positions of individual teeth. These adjustments can be made flexibly, and without affecting the overall structure of the teeth and gums, ensuring that the patient receives the most accurate and effective treatment possible.

Preferably, the computer-implemented method 1000 comprises showing the obtained at least one facial image of the patient as a first image, see the image referred to under reference number 100, and showing the obtained at least one intraoral scan of the patient as a second image, see the images referred to under reference numbers 300A, 300B. In this case, the first image and the second image are distinct images from each other. For example, the method obtains at least one facial image of the patient, e.g. a front-facing photo of the patient's face shown in 100 and at least one intraoral scan, e.g. a 3D scan of the patient's teeth and gums, referenced to by reference numbers 300A and 300B. The method can display the facial image as the first image (reference number 100) and the intraoral scan as the second image (reference numbers 300A and 300B) on the dentist's computer screen. This would allow the dentist to compare the patient's facial features (e.g. jaw structure, lip shape, etc.) with the condition of their teeth and gums, helping them to make a more accurate diagnosis and develop a treatment plan. In this arrangement, one advantage is that dental professionals can more easily visualize and analyze the patient's teeth and surrounding structures from both the facial image and the intraoral scan. This can improve the accuracy and precision of treatment planning, as well as reduce the risk of errors or complications. It may be further provided that the first image and the second image are displayed side-by-side to create a more comprehensive and detailed view of the patient's teeth. In this arrangement, one advantage is that dental professionals can more easily compare and contrast the two sets of data to identify any issues or conditions that may not have been visible in one image alone. This can lead to more accurate and comprehensive treatment planning and execution. It may be provided that the computer-implemented method 1000 further comprises displaying the first image, the second image, and the overlay image as distinct images from each other. In this arrangement, one advantage is that dental professionals can more easily compare and contrast the different images to identify any issues or conditions that may require attention and can make more informed treatment decisions. It may be further provided that the first image, the second image, and the overlay image are displayed side-by-side or in separate windows on a computer monitor or other display device. In this arrangement, one advantage is that dental professionals can more easily analyze and compare the different images to create a more accurate and comprehensive digital model of the patient's teeth.

Figure 1 further shows the exemplary embodiment where the computer- implemented method 1000 further comprises displaying the intraoral scan of the portion of the upper jaw 300A as the second image and the intraoral scan of the portion of the lower jaw 300B as a third image. In this arrangement, one advantage is that dental professionals can more easily visualize and analyze the patient's upper and lower teeth separately, which can lead to more accurate and comprehensive treatment planning and execution. It may be further provided that the first image, the second image, the third image, and the overlay image are displayed as distinct images from each other, such as side-by-side or in separate windows on a computer monitor or other display device. In this arrangement, one advantage is that dental professionals can more easily compare and contrast the different images to identify any issues or conditions that may require attention and can make more informed treatment decisions.

Preferably, the computer-implemented method 1000 further comprises obtaining 600 at least one Computed Tomography, CT, image of the orthodontic region of the patient. When creating a digital representation of a patient's teeth, the method 1000 may acquire a CT image in addition to facial images and intraoral scans. A CT image can provide more detailed information about the teeth and their structure, which can be useful in creating a comprehensive digital model of the patient's teeth. The CT image is then used to create a third digital representation of the teeth through a process called third segmentation. During this process, the CT image is divided into multiple segments to obtain a digital representation of each segment. Each third digital representation of a tooth includes one or more third parameters, such as a coordinate of the tooth, to provide information about the location and structure of the tooth. Finally, the third digital representation is stored as a separate digital file from the first and second digital files. This allows dental professionals to access and analyze the information from the CT image separately from the other digital representations, providing a more detailed understanding of the patient's teeth and any issues or conditions that may not have been visible in the facial image or intraoral scan. The incorporation of a CT image and third digital representation can provide several advantages for dental professionals. Firstly, it can create a more comprehensive and detailed digital model of the patient's teeth, allowing for more accurate treatment planning. This is because the CT image can provide more detailed information about the structure and alignment of the teeth, which may not be visible in the facial image or intraoral scan. Furthermore, the third digital representation can help identify any issues or conditions that may not have been visible in the facial image or intraoral scan, such as hidden or impacted teeth. This can aid in early detection and treatment of these issues, leading to better outcomes for the patient.

The digital model created through the method 1000 can be further enhanced by generating a root structure. This root structure is created by aligning the first, second, and third digital representations of the teeth. The root structure provides information about the position and alignment of the teeth and any underlying conditions or issues that may affect the treatment plan. During the alignment, the crown shown in the third digital representation is aligned with the crown shown in the first or second digital representation. The root structure in the digital model of the patient’s teeth is then generated by aligning all three digital representations. The root structure of the third digital representation shown in the aligned first, second, and third digital representations is used as the root structure of the patient. This allows for more accurate visualization and analysis of the position and alignment of the teeth and surrounding structures, leading to more accurate and comprehensive treatment planning and execution. Again, the computer-implemented method 1000 maintains the coordinates of the modified tooth in the first digital file, the second digital file, and the third digital file during the step of applying the modification to the overlay image. In this arrangement, one advantage is that dental professionals can ensure that any modifications made to the digital representation of the tooth are accurately applied to all three digital files, which can provide a more comprehensive and accurate digital model of the patient's teeth and surrounding structures. This can improve the accuracy and precision of treatment planning, reduce the risk of errors or complications, and ultimately lead to better patient outcomes.

In addition to displaying the first, second, and third digital representations as images, the CT image can be displayed as a fourth image, see reference number 600. The CT image provides additional information about the bone structure, such as the root structure and density of the teeth, which can be useful in planning treatments such as dental implants or orthognathic surgery. Displaying the CT image as a separate image allows dental professionals to easily visualize and analyze the patient's teeth and surrounding structures from different perspectives and with different levels of detail. It may also be provided that the first, second, optionally third, fourth images, and overlay image are displayed as distinct images from each other, such as side-by-side or in separate windows on a computer monitor or other display device. This allows dental professionals to compare and contrast the different images and identify any issues or conditions that may require attention. This can lead to more informed treatment decisions and better treatment outcomes for the patient.

As mentioned before the method 1000 allows users to modify the digital model of the patients teeth. In such a case it is preferred that the computer-implemented method stores each modification as an independent modification. This means that when the user modifies the digital model, the method stores each modification as an independent modification. This allows dental professionals to track the changes made to the digital model over time and revert to previous versions if necessary. It also allows multiple modifications to be made without affecting the original digital model, providing a more flexible and customizable approach to treatment planning. By storing each modification as an independent modification, the dental professionals can also keep track of the modifications made by different users and track the progress of the treatment plan. This can lead to better communication between dental professionals and more coordinated treatment plans. Preferably, each modification is revertible. This means that each modification made to the digital model is stored in a way that allows dental professionals to undo or revert the modification if necessary. This ensures that errors or complications can be corrected and that the accuracy and precision of treatment planning and execution can be improved. The revertible modifications can be stored in a separate file or database, allowing dental professionals to easily track and analyze the modifications made to the digital model over time. This can help them make informed decisions about the treatment plan and ensure that the modifications made are leading to the desired outcomes for the patient. Storing the revertible modifications in a separate file or database also provides a backup of the digital model, ensuring that the original model can be restored if needed. This can help reduce the risk of errors or complications and provide dental professionals with greater control over the treatment plan.

Preferably, the computer-implemented method further comprises simulating 700 a movement of the patient's jaw using the aligned at least one first digital representation and the at least one second digital representation based on anatomical averages of jaw and tooth moment to position the at least one first digital representation and the at least one second digital representation with respect to each other. This simulation can provide valuable information about the movement of the patient's jaw, which can be used to plan and execute dental treatments such as orthodontics or dental implants. By using anatomical averages of jaw and tooth moment to position the at least one first digital representation and the at least one second digital representation with respect to each other, dental professionals can create a more accurate and precise digital model of the patient's teeth and surrounding structures. This approach can improve the accuracy and precision of dental treatments and reduce the risk of errors or complications. By simulating the movement of the patient's jaw, dental professionals can more accurately predict the impact of various treatments on the overall structure and function of the teeth and surrounding structures, enabling more informed treatment decisions.

Preferably, the computer-implemented method further comprises deriving one or more occlusion parameters based on the simulated movement of the patient's jaw. The occlusion parameters derived from the simulated movement of the patient's jaw can provide valuable information about the alignment and function of the patient's teeth and surrounding structures. By using the simulated movement of the patient's jaw to derive occlusion parameters, dental professionals can more accurately identify any issues or conditions that may require attention and can make more informed treatment decisions. This approach can improve the accuracy and precision of dental treatments and reduce the risk of errors or complications. By deriving occlusion parameters based on the simulated movement of the patient's jaw, dental professionals can more accurately predict the impact of various treatments on the overall structure and function of the teeth and surrounding structures, enabling more informed treatment decisions. Preferably, the computer-implemented method further comprises simulating 700 a movement of the patient's jaw using the aligned at least one first digital representation, the at least one second digital representation, and the at least one third digital representation. The simulation of the movement of the patient's jaw using the aligned at least one first digital representation, the at least one second digital representation, and the at least one third digital representation can provide a more comprehensive and accurate digital model of the patient's teeth and surrounding structures. By incorporating information from the CT image into the simulation, dental professionals can create a more detailed and accurate model of the patient's teeth and surrounding structures, enabling more informed treatment decisions. This approach can improve the accuracy and precision of dental treatments and can reduce the risk of errors or complications. By simulating the movement of the patient's jaw using the aligned at least one first digital representation, the at least one second digital representation, and the at least one third digital representation, dental professionals can more accurately predict the impact of various treatments on the overall structure and function of the teeth and surrounding structures.

Figure 2 shows a flow diagram of an exemplary embodiment for aligning the at least one first digital representation and the at least one second digital representation. As mentioned, after obtaining the first digital file and the second digital file, the next step of the method 1000 is aligning the at least one first digital representation and the at least one second digital representation. This step creates a comprehensive digital model of the patient's teeth that incorporates information from both the facial image and the intraoral scan.

As a first step of aligning 800 the at least one first digital representation and the at least one second digital representation, the alignment process 800 involves determining 811 the center point of each tooth in the first digital representation 801. This is done by analyzing the parameters obtained during the first segmentation process, such as the coordinates of the tooth.

Next, the second digital representation 802 is rotated 821 to match the perspective rotation of the first digital representation 801. This is done to ensure that the two digital representations can be aligned accurately.

After the second digital representation is rotated to match the perspective rotation of the first digital representation, the center point of each tooth in the rotated second digital representation is projected onto a plane that is in line with the first digital representation. This essentially transforms the 3D intraoral scan into a 2D image, allowing for the center points of each tooth to be projected onto it for alignment purposes. This approach leads to a substantially easier computational calculation than aligning the center points of a 3D model to those of a 2D model, enabling faster and more efficient processing of the data.

The final step in the alignment process 800 involves matching the center point of each tooth in the first digital representation to the projected center point of each tooth on the plane. This point matching process ensures that the two digital representations are accurately aligned with each other. By aligning 800 the two digital representations, dental professionals can create a more accurate digital model of the patient's teeth, which is important for treatment planning and execution. Point matching is the process of finding corresponding points between two digital representations to align them accurately. In the context of the method 1000, point matching is used to align the digital representations obtained from the first and second segmentation processes, which are typically a 2D image of the patient's face and a 3D intraoral scan. During the alignment process, the center point of each tooth in the first digital representation is determined, and the center point of each tooth in the rotated second digital representation is projected onto a plane that is in line with the first digital representation. The point matching process involves matching the determined center point of each tooth in the first digital representation to the projected center point of each tooth on the plane of the second digital representation. The point matching process ensures that corresponding points in the two digital representations are aligned with each other, creating a more accurate and comprehensive digital model of the patient's teeth and surrounding structures. By accurately aligning the digital representations, dental professionals can more effectively plan and execute dental treatments, leading to better outcomes for the patient.

Figure 4 is a front view of a display screen or portion thereof with graphical user interface, GUI, showing an ornamental design of a primary user interface display. The term primary user interface display is used to refer to the main screen that users see when they start the application or device. Figure 4 shows three main GUI zone, a title zone, an interaction zone and a tab zone. In the title zone a user has access to controls such as user settings or information such as displaying the title which provides users with information about the current context or task, helping them understand what section or functionality they are viewing or interacting with. In the interaction zone a user is provided with options such as viewing, selecting, searching, filtering, or managing cases. The tab zone may aid the used in searching, filtering, or managing cases and displaying corresponding results in the interaction zone. The tab zone also facilitates collaboration and case sharing between users. Figure 4 may also be seen as a GUI to display a case library.

Figure 5 is a front view of a display screen or portion thereof with graphical user interface, GUI, showing an ornamental design of a secondary user interface display. Figure 5 depicts a front view of a display screen or portion thereof featuring a secondary user interface design specifically for case management related to dental treatment. The secondary user interface display may also be referred to as the virtual treatment room. This design is characterized by its visual arrangement and aesthetic elements that enhance user interaction and provide comprehensive information regarding a patient's dental treatment progress. The design includes sections and panels arranged to display past and future treatment information, for example image files for each respective step in a dental treatment. Each respective step in the treatment may organized and accessed, respectively or viewed as a whole. For example, a time line outlined in the documentation zone may provide a process for final restorations, beginning with the initial documentation on October 17, 2021, followed by initial planning on November 10, 2021. Subsequent steps include preparatory work and Clinical Practice in early 2023, with functional and motivational design elements being addressed. Notably, after orthodontic adjustments on March 16, 2023, and motion recording on March 15, 2023, surgery involving the upper and lower regions, as well as the application of orthodontics and miniplates, was performed. This systematic approach culminated in the final restorations by April 3, 2023. The design may include options to create new galleries or to share galleries with other users.

The design may include a GUI zone that includes a visually distinct interface for filtering images based on file type. The filtering interface is presented as an integrated panel or dropdown menu within the broader GUI, organized to provide an intuitive and visually appealing method for selecting image file types. The interface includes stylized typography for file type labels, using ornamental fonts that are both readable and aesthetically engaging.

The design may include chat interface as a dedicated panel within the GUI, organized to provide an intuitive and visually appealing space for communication.

The design illustrates a GUI title bar that includes visual elements for switching between various sections such as case documentation, projects, ongoing or past activities, and other services. The title bar is presented as a horizontal strip at the top of the GUI, organized to provide an intuitive and visually appealing method for navigating between different sections. The design employs a series of tabs or buttons that are symmetrically arranged to offer easy access to distinct functionalities. Each section option within the title bar is labeled with stylized typography, using ornamental fonts that enhance readability and aesthetic appeal. Accompanying each label are decorative icons representing the nature of the section, such as folders for case documentation or calendars for projects. The images shown in the GUI of figure 5 represent files from the documentation section.

Figure 6 is a front view of a display screen or portion thereof with graphical user interface, GUI, showing a design of a tertiary user interface display. The tertiary user interface display, is designed to provide access to various workflows related to dental treatment, including smile design, blueprint creation, review, and comparison as shown in fig. 7. The tertiary user interface display may include a side panel gallery, specifically designed to provide access to simulated results of dental treatment or intermediary steps of the dental treatment.

Figure 7 is a front view of a display screen or portion thereof with graphical user interface including a drop down panel describing one or more user workflows such as smile design, blueprint creation, review, and comparison. Figure 8A and Figure 8B shows front view of a display screen or portion thereof with graphical user interface illustrating the compare workflow. Figure 8A illustrates a front view of a GUI featuring two upload options designed for the purpose of uploading comparative images. In figure 8A, The interface on the left may be designated for uploading an image that may represent a state before treatment. Conversely, the interface on the right is intended for uploading an image that facilitates comparison with the left image, potentially illustrating a simulated outcome posttreatment. Figure 8C illustrates a comparison functionality within the graphical user interface (GUI) designed to display images depicting states before and after treatment. This functionality enables users to visually assess the differences between the pre-treatment and post-treatment images. The "compare" feature in the GUI facilitates a side-by-side or overlay view of the images, allowing for a detailed examination of changes or improvements resulting from the treatment.

A comparison between two image may also be performed in a single image frame as shown in figure 6. Figure 6 also demonstrates a comparison functionality within the GUI that allows for the analysis of two images within a single image frame. The GUI may incorporate a slider mechanism that facilitates dynamic visual representation of an image, transitioning smoothly between a state before treatment and a state after treatment. This slider allows users to interactively adjust the image display, providing a clear and immediate comparison of changes attributed to the treatment. By manipulating the slider, users can observe a gradual transformation, enhancing the understanding of the treatment's impact.

Figure 9A illustrates the smile design workflow, a process integrated within the GUI that enables a user to upload an initial portrait image or video. This feature is the foundational step in the workflow, allowing the user to input their visual data, which can be used for subsequent design and modification processes related to smile aesthetics.

Figure 9B depicts the simulation of a potential smile candidate superimposed on the user's facial image, which was uploaded in Figure 9A. This feature is part of the smile design workflow, allowing users to visualize how proposed smile modifications would appear on their actual facial portrait. The GUI incorporates a library of potential teeth solutions, conveniently displayed alongside the simulated smile design, as shown in figure 9B. This feature provides users with a selection of various teeth designs, shapes, and configurations that can be previewed and applied to the user's facial image. The library serves as a comprehensive resource for customizing the smile design, allowing users to explore different aesthetic options and select the most suitable teeth solution based on their preferences and facial characteristics.

Figure 9B also illustrates a GUI designed for the manipulation of the image within the smile design workflow. Figure 9C illustrates an enlarged format of the graphical user interface (GUI) designed for the manipulation of the image within the smile design workflow, shown in figure 9B. This GUI provides users with tools and functionalities to adjust and refine the simulated smile and facial features, ensuring a tailored and precise design outcome. The manipulation capabilities may include options for resizing, rotating, adjusting color, and modifying the alignment of the smile elements to achieve an optimal aesthetic result. Rotation allows the user to rotate the facial image or individual components within the image to achieve the desired orientation for optimal smile design presentation. Lip Contour Indication provides tools to define or adjust the contour of the lips, ensuring that the smile design aligns correctly with the user's natural lip shape. Image Parameter Modification enables alterations to basic image parameters such as brightness, contrast, saturation, and sharpness, enhancing the visual clarity and suitability of the image for design purposes. Anatomy Parameter Modification allows modification of anatomical features to simulate realistic changes or enhancements to facial structure, accommodating different smile designs. Visibility Modification offers options to adjust the visibility of specific elements within the image, such as teeth or facial features, to focus on particular aspects of the smile design. Papilla Modification permits adjustments to the papilla, or the soft tissue between the teeth, ensuring that changes to the smile design consider the aesthetic and anatomical coherence of the gum line.

Figures 10A-10K illustrates a front view of GUIs designed for creating a 3D blueprint of dental structures using selected teeth shapes. Shown in figures 9A, 9B and 9C, this interface is characterized by its ornamental design elements that enhance user interaction and visualization capabilities. The blueprint creation interface is organized to provide an intuitive and visually appealing workspace for selecting teeth shapes and positioning them according to patient 3D data. The design incorporates modular panels or sections for shape selection, data integration, and tool activation. Figure 10 presents a GUI with five distinct panels, each dedicated to a specific action within the dental design workflow. These panels facilitate a comprehensive approach to integrating various types of data for enhanced smile and dental design. 2D Image or Video Upload Panel allows users to upload a 2D image or video file of the patient. This serves as the foundational visual data, enabling the initiation of aesthetic evaluations and modifications based on the patient's current facial or dental presentation. 3D Jaw Scan Upload Panel is designed for uploading 3D scans of either the upper or lower jaw. The inclusion of 3D data provides detailed anatomical insights, allowing for precise adjustments and assessments in dental design, particularly in relation to occlusion and jaw alignment. A separate panel for the lower, respectively upper jaw may be included, as shown in figure 10. CBCT Integration Panel facilitates the addition of Cone Beam Computed Tomography, CBCT, data to the dental design. CBCT offers detailed volumetric images, contributing critical information about bone structure and density, which is essential for planning dental treatments and interventions. Motion Integration Panel enables the incorporation of motion data into the dental design process. Motion data can capture dynamic aspects of jaw movement and function, providing valuable insights for designing dental solutions that accommodate natural motion and functional requirements. Other panels may be included, potentially for integrating other relevant data or performing specific actions that complement the dental design workflow.

Figure 10A shows a front view of a basic, uncompleted GUI render intended for dental treatment management, in that none of the panels have been selected. This interface design is characterized by its foundational ornamental elements, which provide the initial aesthetic framework for subsequent development. Figures 11 A and 1 IB represent a GUI line drawing of the render shown in figure 10 A.

Figure 10B shows a detailed view of the graphical user interface (GUI) displayed after the selection of a 2D image. Figure 11C represents a GUI line drawing of the render shown in figure 10B.

Figure 10C shows a GUI render displayed upon selecting the "upper" panel from Figure 10A. This interface design facilitates the selection and compilation of uploaded scans or other documentation within the framework initially outlined in Figure 10A. The actions described in relation to figures 10C and 10D apply to the “lower” from Figure 10A.

Figure 10D shows a GUI render that appears after an "upper" scan has been selected. This interface integrates additional functionality, automatically segmenting and aligning the selected scan with the smile configuration presented in Figure 10A. A result of the segmentation and alignment is shown in figure 10E.

Figure 10F shows a GUI render where the real 2D image is compiled with a simulated dental representation using the "upper" scan imported in the previous step. This interface design showcases the integration of both real and simulated elements, enhancing the visual and functional interaction. Figure 11D represents a GUI line drawing of the render shown in figure 10F.

Figure 10G shows a GUI render that appears upon selecting "structure" in the title panel, enabling user interaction with a 3D model of the scan. This interface design enhances visual engagement and functional interaction through the presentation of three-dimensional elements. The GUI integrates advanced functionality for detecting missing or incorrect teeth within the dental design workflow. This feature leverages image processing and analysis algorithms to identify discrepancies in the dental anatomy, such as absent teeth or structural irregularities. Upon detection, the GUI provides tools that empower users to select, adjust, move, and reshape teeth to achieve the desired aesthetic and functional outcomes.

Figure 10H shows a GUI render that integrates inputs from Figures 10B through 10G, providing a comprehensive platform for designing and reshaping the dental situation of the patient. This GUI design offers a visually engaging and interactive environment for dental customization. This GUI is designed to streamline the design and reshaping of a patient's dental situation, offering an interactive and visually engaging environment tailored for dental customization. It encompasses a set of features designed to enhance the user experience and improve procedural accuracy. Central to its functionality is the Teeth Selection Interface, which provides tools or options for selecting specific teeth targeted for modification or customization. Once selected, practitioners can employ modification tools that offer capabilities for altering the teeth, including resizing, reshaping, or repositioning, thereby accommodating individual patient needs and preferences. Such tools are depicted in the GUI shown in figure 101.

Figure 10J shows a GUI render where the real 2D image is compiled with a simulated dental representation using the "upper" and “lower” scan. This interface design showcases the integration of both real and simulated elements, enhancing the visual and functional interaction. Figure 1 IE represents a GUI line drawing of the render shown in figure 10J.

Figure 10K shows a GUI render where a Cone Beam Computed Tomography image is compiled with a simulated dental representation using the "upper" and “lower” scan. This interface design showcases the integration of both real and simulated elements, enhancing the visual and functional interaction. Figure 11F represents a GUI line drawing of the render shown in figure 10K.

To ensure the modifications are precise and meet the desired outcomes, the interface includes Display Features that visualize the selected teeth along with any applied changes, allowing practitioners to preview adjustments in real-time. This visualization supports informed decision-making and enhances the customization process. Complementing these features are Anatomy Parameter Adjustments, which provide controls for altering anatomical parameters such as dimensions, angles, or alignment, ensuring that the dental structure fits the patient's unique morphology.

Furthermore, the interface supports Color Customization, offering options to adjust the color of the teeth to align with aesthetic preferences, whether those of the patient or the practitioner. Lastly, the interface incorporates Propositions for Teeth, which present suggestions or automated propositions for dental configurations based on predefined criteria or algorithms, assisting practitioners in making optimal design choices. This comprehensive integration of functionalities within a single GUI is intended to elevate efficiency, precision, and user satisfaction in dental practice.

A person of skill in the art would readily recognize that steps of various above-described methods can be performed by programmed computers. Herein, some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machine-executable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of said above-described methods. The program storage devices may be, e.g., digital memories, magnetic storage media such as a magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media. The program storage devices may be resident program storage devices or may be removable program storage devices, such as smart cards. The embodiments are also intended to cover computers programmed to perform said steps of the above-described methods.

The description and drawings merely illustrate the principles of the present invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements that, although not explicitly described or shown herein, embody the principles of the present invention and are included within its scope. Furthermore, all examples recited herein are principally intended expressly to be only for pedagogical purposes to aid the reader in understanding the principles of the present invention and the concepts contributed by the inventor(s) to furthering the art and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the present invention, as well as specific examples thereof, are intended to encompass equivalents thereof.

The functions of the various elements shown in the figures, including any functional blocks labelled as “processors”, may be provided through the use of dedicated hardware as well as hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and/or custom, may also be included. Similarly, any switches shown in the figures are conceptual only. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.

It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the present invention. Similarly, it will be appreciated that any flowcharts, flow diagrams, state transition diagrams, pseudo code, and the like represent various processes which may be substantially represented in computer readable medium and so executed by a computer.

It should be noted that the above-mentioned embodiments illustrate rather than limit the present invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word “comprising” does not exclude the presence of elements or steps not listed in a claim. The word “a” or “an” preceding an element does not exclude the presence of a plurality of such elements. The present invention can be implemented by means of hardware comprising several distinct elements and by means of a suitably programmed computer. In claims enumerating several means, several of these means can be embodied by one and the same item of hardware. The usage of the words “first”, “second”, “third”, etc. does not indicate any ordering or priority. These words are to be interpreted as names used for convenience.

In the present invention, expressions such as “comprise”, “include”, “have”, “may comprise”, “may include”, or “may have” indicate existence of corresponding features but do not exclude existence of additional features. Whilst the principles of the present invention have been set out above in connection with specific embodiments, it is to be understood that this description is merely made by way of example and not as a limitation of the scope of protection which is determined by the appended claims.

Claims

1. A computer-implemented method for digital tooth representation, the method comprising:

- obtaining at least one facial image of a patient, said at least one facial image comprising a head of the patient and a smile comprising a one or more teeth;

- storing the obtained at least one first digital representation of the one or more teeth as a first digital file;

- obtaining at least one intraoral scan of the patient, the at least one intraoral scan comprising at least one of a portion of an upper jaw comprising one or more upper teeth of the patient and a portion of a lower jaw comprising one or more lower teeth of the patient;

2. The computer-implemented method of claim 1, further comprising :

- showing the obtained at least one facial image of the patient as a first image; - showing the obtained at least one intraoral scan of the patient as a second image, wherein the first image and the second image are distinct images from each other.

3. The computer-implemented method according to any one of the previous claims, further comprising showing the overlay image.

4. The computer-implemented method according to the previous claim and claim 2, wherein the first image, the second image and overlay image are distinct images from each other.

5. The computer-implemented method according to any one of the previous claims, wherein the step of obtaining at least one intraoral scan of the patient comprises obtaining an intraoral scan of a portion of an upper jaw comprising one or more upper teeth of the patient, and obtaining an intraoral scan of a portion of a lower jaw comprising one or more lower teeth of the patient; and wherein the step of obtaining at least one second digital representation comprises performing a separate second segmentation on the intraoral scan of the portion of the upper jaw and a separate second segmentation on the intraoral scan of the portion of the lower jaw.

6. The computer-implemented method according to claims 2-5, further comprising:

- showing the intraoral scan of the portion of the upper jaw as the second image, and

- showing the intraoral scan of the portion of the lower jaw as a third image; wherein the first image, the second image, third image and the overlay image are distinct images from each other.

7. The computer-implemented method according to any one of claims 2 - 6, wherein the step of allowing a user to modify at least a dimension of at least one tooth is performed by selecting one of the first image and the second image; selecting at least one tooth shown in the selected first image or the second image; modifying the at least a dimension of the selected at least one tooth.

8. The computer-implemented method according to any one of the previous claims, wherein each modification is stored as an independent modification.

9. The computer-implemented method according to the previous claim, wherein each modification is revertable.

10. The computer-implemented method according to any one of the previous claims, further comprising:

- obtaining at least one Computed Tomography, CT, image of the orthodontic region of the patient;

- obtaining at least one third digital representation of the one or more teeth by performing a third segmentation on the obtained at least one CT image, wherein during the third segmentation each third digital representation of the one or more teeth comprises one or more third parameters including a parameter representative for a coordinate of the tooth;

- storing the obtained at least one third digital representation as a third digital file, wherein the third digital file is distinct from the first digital file and the second digital file.

11. The computer-implemented method according to the previous claim, further comprising

- generating a root structure in the overlay image by

- aligning the at least one first digital representation, the at least one second digital representation and the at least one third digital representation, wherein said aligning comprises aligning a crown shown in the at least one third digital representation with a crown shown in the at least one first digital representation or the at least one second digital representation;

- using the root structure of the at least one third digital representation shown in the aligned at least one first digital representation, the at least one second digital representation and the at least one third digital representation as the root structure of the patient.

12. The computer-implemented method according to the previous claim, wherein the step of applying the modification to the overlay image comprises maintaining the coordinates of the modified tooth in the first digital file, the second digital file and the third digital file.

13. The computer-implemented method according to any one of the previous claims 10-12, optionally in combination with claim 6, further comprising:

- showing the obtained at least one CT image as a fourth image, wherein the first image, the second image, optionally the third image, the fourth and the overlay image are distinct images from each other.

14. The computer-implemented method according to any one of the previous claims, wherein the step of aligning the at least one first digital representation and the at least one second digital representation comprises:

- determining a center point of each tooth in the at least one first digital representation

- rotating the at least one second digital representation to substantially match the perspective rotation of the at least one first digital representation;

- projecting a center point of each tooth in the rotated at least one second digital representation on a plane in line with the at least one first digital representation;

- point matching the determined center point of each tooth in the at least one first digital representation to a projected center point of each tooth on the plane to align the at least one first digital representation and the at least one second digital representation.

15. The computer-implemented method according to any one of the previous claims, further comprising simulating a movement of the patient’s jaw using the aligned at least one first digital representation and the at least one second digital representation based on anatomical averages of jaw and tooth moment to position the at least one first digital representation and the at least one second digital representation with respect to each other.

16. The computer-implemented method according to the previous claim, further comprising deriving one or more occlusion parameters based on the simulated movement of the patient’s jaw.

17. The computer-implemented method according to any one of the previous claims 15-16 and any one of claims 11-13, further comprising simulating a movement of the patient’s jaw using the aligned at least one first digital representation, the at least one second digital representation and the at least one third digital representation

18. A computer program product comprising a computer-executable program of instructions for performing, when executed on a computer, the steps of the method of any one of the previous claims.