WO2025189405A1 - Three-dimensional model usage method based on medical model content library - Google Patents

Abstract

The present application relates to the technical field of computers, and provides a three-dimensional model usage method and apparatus based on a medical model content library, an electronic device, and a storage medium. The method comprises: when performing medical simulation training, invoking a medical virtual content library, wherein the medical virtual content library is obtained by storing three-dimensional data files; and obtaining demand data, searching the medical virtual content library for a corresponding target three-dimensional data file on the basis of the demand data, and invoking a corresponding target three-dimensional model on the basis of a target three-dimensional medical file. The present application solves the problem in the prior art of how to quickly provide medical three-dimensional scenes and matched modes under various scenes.

Description

A method for using three-dimensional models based on medical model content library Technical Field

The present application relates to the field of computer technology. Specifically, the present application relates to a method, device, electronic device and storage medium for using a three-dimensional model based on a medical model content library.

Background Art

The current traditional medical education system, which primarily relies on animal and human specimens and teaching aids, faces challenges in training medical professionals, including insufficient resources and a high risk of harm to patients. Virtual reality (VR) technology uses computers to generate virtual three-dimensional scenes, providing users with a sense of immersion through vision, hearing, and touch. With the recent development of virtual reality and mixed reality (MR) technologies, their application in medical education, training, and assessment has become a new trend. Compared to traditional teaching methods, VR and MR technologies offer significant advantages. By collecting multi-dimensional data from the real human body and constructing digital models of the human body or target tissue through simulation modeling, they enable low-cost, repeatable, and quantifiable digital teaching. This allows students to learn and grow in a repeatable practice environment, providing rich, scientifically standardized simulation materials, and effectively alleviating the challenges of insufficient teaching resources and the difficulty of quantitative assessment. This new simulation-based teaching method can shorten students' clinical practice learning curve while ensuring medical safety during their learning process.

Currently, there are many ways to obtain mixed reality medical three-dimensional models. 3D modeling companies perform professional geometric modeling. This method is costly and time-consuming, and cannot meet the actual medical model requirements in various scenarios. It is difficult to adapt to the large-scale demand for mixed reality medical teaching and application development.

From the above, we can see that how to quickly provide medical 3D scenes and model combinations in various scenarios, suitable for medical teaching, training and assessment, still needs further improvement.

Summary of the Invention

This application provides a method, device, electronic device, and storage medium for using a three-dimensional model based on a medical model content library, which can solve the problem in related technologies of being difficult to quickly provide medical three-dimensional scenes and model combinations in various scenarios. The technical solution is as follows:

According to one aspect of the present application, a method for using a three-dimensional model based on a medical model content library includes the following steps: when conducting medical simulation training, calling a medical virtual content library, wherein the medical virtual content library is obtained by storing three-dimensional data files; wherein medical content data, medical instrument data, and medical operation scene data are obtained, and the medical content data is converted into a three-dimensional medical content data file, the medical instrument data is converted into a three-dimensional medical instrument data file, and the medical operation scene data is converted into a three-dimensional medical operation scene data file; the three-dimensional medical content data file, the three-dimensional medical instrument data file, and the three-dimensional medical operation scene data file are stored to obtain a corresponding medical virtual content library; obtaining demand data, indexing a corresponding target three-dimensional data file in the medical virtual content library based on the demand data, and retrieving a corresponding target three-dimensional model based on the target three-dimensional medical file, wherein the demand data is the attributes corresponding to the model required for simulation training.

According to one aspect of the present application, a three-dimensional model using device based on a medical model content library includes:

A file acquisition module, during medical simulation training, calls a medical virtual content library, which stores three-dimensional data files; wherein the module acquires medical content data, medical instrument data, and medical operation scene data, and the user converts the medical content data into a three-dimensional medical content data file, the medical instrument data into a three-dimensional medical instrument data file, and the medical operation scene data into a three-dimensional medical operation scene data file;

a storage module, configured to store the three-dimensional medical content data file, the three-dimensional medical instrument data file, and the three-dimensional medical operation scene data file, and obtain a corresponding medical virtual content library;

The retrieval module obtains the demand data, indexes the corresponding target three-dimensional data file in the medical virtual content library based on the demand data, and retrieves the corresponding target three-dimensional model based on the target three-dimensional medical file, wherein the demand data is the attributes corresponding to the model required for simulation training.

In an exemplary embodiment, the apparatus further includes but is not limited to:

A medical image acquisition module is used to acquire medical images in different states, wherein the medical images include normal and diseased images;

Classification module, used to classify all medical images based on organs and tissues;

The content determination module segments all classified medical images to obtain corresponding target areas, and is used to determine the target areas as medical content data, wherein the target areas are areas where organs and tissues are located in the medical images.

In an exemplary embodiment, the apparatus further includes but is not limited to:

a label data acquisition module, configured to acquire corresponding label data based on the organs and tissues corresponding to the target area;

a target three-dimensional medical model acquisition module, which performs three-dimensional reconstruction and rendering on the target area to obtain a corresponding target three-dimensional medical model;

The 3D medical content model file determination module is configured to associate the tag data with the target 3D medical model to determine a corresponding 3D medical content data file.

In an exemplary embodiment, the apparatus further includes but is not limited to:

A multi-view image acquisition module, for acquiring a corresponding multi-view image set based on multiple angles, wherein the multi-view image set includes point cloud images of multiple angles;

A feature point extraction module is used to extract feature points based on the multi-view image set and calculate the three-dimensional coordinates corresponding to the feature points, wherein the feature points include corner points and edge points;

A reconstruction module is used to reconstruct a three-dimensional model based on the three-dimensional coordinates of the feature points, wherein the three-dimensional model includes a medical instrument model and a medical operation scene model.

In an exemplary embodiment, the apparatus further includes but is not limited to:

A classification data acquisition module is used to obtain classification data corresponding to the three-dimensional model;

A classification module, which is used to classify each three-dimensional model level by level based on the classification data, wherein when the three-dimensional model is classified level by level, each level corresponds to a level label;

The three-dimensional data file acquisition module associates the three-dimensional model with all corresponding hierarchical labels to obtain the corresponding three-dimensional data file, wherein the three-dimensional data file includes the three-dimensional medical instrument data file and the three-dimensional medical operation scene data file.

In an exemplary embodiment, the apparatus further includes but is not limited to:

A limited index determination module is used to determine the corresponding limited index after the user enters the interface corresponding to the medical virtual content library, wherein the interface includes scene limitation, instrument limitation and part limitation;

a target three-dimensional data file determining module, which searches a corresponding limited index based on the requirement data to determine a corresponding target three-dimensional data file;

A download module, used for downloading the model storage format corresponding to the target three-dimensional data file;

The output module determines the storage format of the downloaded model as the target three-dimensional model and uses it for output.

According to one aspect of the present application, an electronic device includes at least one processor and at least one memory, wherein the memory stores computer-readable instructions; the computer-readable instructions are executed by one or more of the processors, so that the electronic device implements a method for using a three-dimensional model based on a medical model content library as described above.

According to one aspect of the present application, a storage medium stores computer-readable instructions thereon, and the computer-readable instructions are executed by one or more processors to implement the above-mentioned method for using a three-dimensional model based on a medical model content library.

According to one aspect of the present application, a computer program product includes computer-readable instructions, which are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements a three-dimensional model usage method based on a medical model content library as described above.

The beneficial effects of the technical solution provided by this application are:

In the above technical solution, when conducting medical simulation training, a medical virtual content library is called, and the medical virtual content library is obtained by storing three-dimensional data files; wherein, medical content data, medical instrument data and medical operation scene data are obtained, and the medical content data is converted into a three-dimensional medical content data file, the medical instrument data is converted into a three-dimensional medical instrument data file and the medical operation scene data is converted into a three-dimensional medical operation scene data file; the three-dimensional medical content data file, the three-dimensional medical instrument data file and the three-dimensional medical operation scene data file are stored to obtain a corresponding medical virtual content library; demand data is obtained, and the corresponding target three-dimensional data file is indexed in the medical virtual content library based on the demand data, and the target three The corresponding target three-dimensional medical file is retrieved, wherein the demand data is the attributes corresponding to the model required for simulation training; in the process of building a medical virtual content library, by integrating the existing information related to medical training assessment in the early stage, such as disease images corresponding to organs and tissues, reconstruction of medical device models and medical scene models, and finally storing all the corresponding models, when medical training is required, it is only necessary to index in the medical virtual content library according to the needs, and the corresponding required target three-dimensional data file can be searched, so that the requested three-dimensional model can be retrieved, and multiple three-dimensional models can be retrieved according to the demand data, thereby effectively solving the problem of difficulty in quickly providing medical three-dimensional scenes and model matching in multiple scenarios in related technologies.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without inventive efforts.

FIG1 is a schematic diagram of an implementation environment according to the present application;

FIG2 is a flow chart showing a method for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

FIG3 is a flowchart of steps S120 to S140 in a method for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

FIG4 is a flowchart from S1210 to S1212 in a method for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

FIG5 is a flowchart from S1220 to S1222 in a method for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

FIG6 is a flowchart from S1311 to S1313 of a method for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

FIG7 is a schematic diagram of a file naming format for converting medical content data into a three-dimensional medical content data file;

FIG8 is a flowchart from S1321 to S1323 of a method for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

FIG9 is a schematic diagram of a file naming format for converting medical operation scene data into a three-dimensional medical operation scene data file;

FIG10 is a schematic diagram of the file naming format for converting medical device data into a three-dimensional medical device data file;

FIG11 is a flowchart of steps S210 to S240 in a method for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

FIG12 is a schematic diagram of the system interface corresponding to a user entering a medical virtual content library;

FIG13 is a structural block diagram of a device for using a three-dimensional model based on a medical model content library according to an exemplary embodiment;

Fig. 14 is a structural block diagram of an electronic device according to an exemplary embodiment.

DETAILED DESCRIPTION

The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present application.

It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present disclosure refers to the presence of the features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and/or" used herein includes all or any units and all combinations of one or more associated listed items.

In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

FIG1 is a schematic diagram of an implementation environment involved in a method for using a three-dimensional model based on a medical model content library, wherein the implementation environment includes a collection end 110 and a service end 130 .

Specifically, the acquisition end 110 can also be considered as an image acquisition device, which is used to obtain medical images, medical content data, medical instrument data, medical operation scene data, etc. in different states.

Server 130 can be an electronic device such as a desktop computer, laptop computer, or server. It can also be a computer cluster consisting of multiple servers, or even a cloud computing center consisting of multiple servers. Server 130 is used to provide background services, such as, but not limited to, providing 3D model usage services.

A network communication connection is pre-established between the server 130 and the acquisition terminal 110 via a wired or wireless method, and data transmission between the server 130 and the acquisition terminal 110 is achieved through the network communication connection. The transmitted data includes, but is not limited to, medical images, medical content data, medical device data, medical operation scene data, etc.

In one application scenario, with the interaction between the acquisition terminal 110 and the service terminal 130, the use of three-dimensional models based on the medical model content library can be realized, which greatly simplifies user operations and helps to improve user experience.

Please refer to Figure 2. An embodiment of the present application provides a method for using a three-dimensional model based on a medical model content library. The method is applicable to an electronic device, which can be a server in the implementation environment shown in Figure 1.

In the following method embodiments, for ease of description, the execution subject of each step of the method is taken as an electronic device as an example for illustration, but this does not constitute a specific limitation.

As shown in FIG2 , the method may include the following steps:

S100, calling a medical virtual content library during medical simulation training;

Among them, the medical virtual content library is constructed on mobile phones by making medical-related three-dimensional models in advance.

S200 , acquiring demand data, indexing a corresponding target three-dimensional data file in a medical virtual content library based on the demand data, and retrieving a corresponding target three-dimensional model based on the target three-dimensional medical file.

Among them, the demand data is the attributes corresponding to the model required for simulation training, that is, what type of model is needed. It can be the corresponding purpose and function, or it can directly correspond to the name of the model, and the model naming method in the medical virtual content library is changed.

In the process of medical training, it is only necessary to index in the medical virtual content library according to the needs, and the corresponding required target three-dimensional data file can be searched, so that the requested three-dimensional model can be retrieved, and multiple three-dimensional models can be retrieved according to the required data.

It should be noted that the medical virtual content library is obtained by storing three-dimensional data files. Referring to Figure 3, the specific construction process is as follows:

S120, acquiring medical content data, medical device data, and medical operation scenario data;

The method of acquiring data varies depending on the type of medical data. The process of acquiring medical content, referring to FIG. 4 , specifically includes the following steps:

S1210, acquiring medical images in different states;

Among them, medical images include normal and diseased images. For these images, we can use public data sets on the Internet and relevant image data provided by cooperative hospitals.

S1211, classify all medical images based on organs and tissues;

During the classification process, each image is classified according to its corresponding organ and tissue. The images of different organ and tissue types are included in normal conditions and diseased conditions. The images of diseased conditions are further subdivided according to different diseases.

S1212: Segment all classified medical images to obtain corresponding target areas, and determine the target areas as medical content data.

Among them, the target area is the area where the organs and tissues are located in the medical image. In the process of outlining the target area, it needs to be outlined by professional radiologists. At the same time, in order to better outline the image, it can be trained through a deep neural network. After the training converges, the deep neural network user that meets the requirements will outline the image, thereby improving the efficiency of medical image outlining; and the area corresponding to the outlined image is used as the target area, and the target area is the medical content data that needs to be collected.

By executing the above steps S1210 to S1212 , as many medical images as possible can be collected in the early stage, which is more beneficial for filling the data of the medical virtual content library.

As for the acquisition of medical device data and medical operation scene data, refer to Figure 5 , the specific process is as follows:

S1220, acquiring a corresponding multi-view image set based on multiple angles;

Among them, medical devices and operation scenes are reconstructed through multi-view images. The multi-view image set includes images from multiple perspectives, and medical devices and operation scenes are photographed using multiple cameras or from different angles to obtain image data of sufficient quantity and coverage.

S1221: Extract feature points based on the multi-view image set, and calculate the three-dimensional coordinates corresponding to the feature points.

Among them, feature points include corner points and edge points; at the same time, feature matching can be performed between different views to establish the correspondence between feature points.

S1222, reconstructing a three-dimensional model based on the three-dimensional coordinates of the feature points.

Among them, in the reconstruction process, it is necessary to calculate the three-dimensional coordinates of the points based on the results of feature matching and use methods such as triangulation to generate preliminary point cloud data, and optimize the preliminary generated point cloud data, including removing noise points, smoothing, coordinate transformation, etc., to improve the quality and accuracy of the point cloud. Based on the optimized point cloud data, a surface reconstruction algorithm (such as Poisson reconstruction, Delaunay triangulation, etc.) is used to generate a three-dimensional model. It should be pointed out that the three-dimensional model is also a three-dimensional point cloud model, and the three-dimensional model includes a medical device model and a medical operation scene model.

S130 , converting the medical content data into a three-dimensional medical content data file, converting the medical instrument data into a three-dimensional medical instrument data file, and converting the medical operation scene data into a three-dimensional medical operation scene data file.

In the process of converting data into files, referring to FIG6 , the conversion process of medical content data is as follows:

S1311, acquiring corresponding label data based on the organs and tissues corresponding to the target area;

S1312, performing three-dimensional reconstruction on the target area and rendering to obtain a corresponding target three-dimensional medical model;

S1313: Associate the label data with the target three-dimensional medical model to determine a corresponding three-dimensional medical content data file.

The execution steps of S1311 to S1313 are described in detail with reference to FIG7 :

In the process of acquiring medical images, based on the current medical images, it is necessary to determine the basic information corresponding to the current medical images. For example, "the patient's gender is male, the body part is the abdomen, the organ is the liver, and the corresponding disease name is benign liver tumor 10mm" can be used as the label data corresponding to the target area. Then, based on the label data, the developed 3D reconstruction software is used to reconstruct and render the outlined target area to obtain the required target medical 3D model. Finally, the label data is associated with the target 3D medical model to obtain the corresponding 3D medical content data file.

8 , the conversion process of the 3D medical instrument data and the 3D medical operation scene data is as follows:

S1321: Obtain classification data corresponding to the three-dimensional model.

S1322: Classify each three-dimensional model level by level based on the classification data. When classifying the three-dimensional model level by level, each level corresponds to a level label.

S1323, associating the three-dimensional model with all corresponding hierarchical labels to obtain a corresponding three-dimensional data file;

The three-dimensional data file includes a three-dimensional medical instrument data file and the three-dimensional medical operation scene data file.

The execution steps of S1321 to S1323 are described in detail with reference to Figures 9 and 10 based on Figure 10, taking medical instruments as an example: in the process of acquiring three-dimensional medical instrument data, the classification data of the medical instrument is first obtained, "the major category of surgical instruments is scalpel, and the surgical machine model is No. 3", and then it can be classified level by level, and No. 3 is selected in the file name of the scalpel to achieve classification; after the corresponding medical instrument is classified, the corresponding hierarchical label can be associated with the three-dimensional model, and the corresponding three-dimensional data file can be obtained.

S140, storing the 3D medical content data file, the 3D medical instrument data file, and the 3D medical operation scene data file to obtain a corresponding medical virtual content library;

When the medical virtual content library is needed, the demand data is obtained, the corresponding target three-dimensional data file is indexed in the medical virtual content library based on the demand data, and the corresponding target three-dimensional model is retrieved based on the target three-dimensional medical file, wherein the demand data is the attributes corresponding to the model required for simulation training.

When using the Medical Virtual Content Library, we developed client software for 3D reconstruction and display of medical images using the VTK library and Qt. This client connects to the 3D database and includes functionality for automatically selecting 3D content, enabling indexed searches on the client. Referring to Figure 11, the detailed usage process is as follows:

S210, after the user enters the interface corresponding to the medical virtual content library, determining the corresponding limited index;

Among them, referring to the system interface in Figure 12, the interface includes scene limitation, instrument limitation and part limitation.

S220, searching in a corresponding limited index based on the demand data to determine a corresponding target three-dimensional data file;

The required model type is described in the requirement data. For example, if you need to find scalpel No. 3, you can directly select the instrument limitation and then search the instrument model database index to obtain the target three-dimensional data file corresponding to scalpel No. 3.

S230: Download the model storage format corresponding to the target three-dimensional data file.

S240: Determine the storage format of the downloaded model as a target three-dimensional model and output it.

Through S210 to S240, 3D content selection output can be generated to meet the needs of doctors. For example, when developing a mixed reality teaching scenario, select the teaching and research scenario. For teaching the diagnosis of gastric lesions, you can limit the index to the stomach in the site selection, and randomly generate multiple 3D stomach models with different lesions for medical teaching. For anatomy teaching, you can select anatomy teaching in the scene, select anatomical surgical instruments in the instrument selection, and generate one for each site in the site selection, which can meet the needs of our subsequent mixed reality anatomy medical development.

By building a medical virtual content library, the required models can be directly selected and searched on the client side, and then downloaded for use. This realizes the construction of a three-dimensional content library suitable for medical mixed reality, and can automatically provide and select multiple three-dimensional models according to subsequent mixed reality development scenarios, thereby quickly providing medical three-dimensional scenes and model combinations in multiple scenarios, which is suitable for medical teaching, training and assessment.

The following is an embodiment of the device of the present application, which can be used to implement a method for using a three-dimensional model based on a medical model content library involved in this application. For details not disclosed in the embodiment of the device of the present application, please refer to the method embodiment of a method for using a three-dimensional model based on a medical model content library involved in this application.

The present application provides a three-dimensional model using device based on a medical model content library, with reference to FIG13 , including but not limited to:

The file acquisition module 300, during medical simulation training, calls a medical virtual content library, which stores three-dimensional data files; wherein the module acquires medical content data, medical instrument data, and medical operation scene data; and the user converts the medical content data into a three-dimensional medical content data file, the medical instrument data into a three-dimensional medical instrument data file, and the medical operation scene data into a three-dimensional medical operation scene data file;

The storage module 310 is configured to store the 3D medical content data file, the 3D medical instrument data file, and the 3D medical operation scene data file to obtain a corresponding medical virtual content library;

The retrieval module 320 obtains the required data, indexes the corresponding target three-dimensional data file in the medical virtual content library based on the required data, and retrieves the corresponding target three-dimensional model based on the target three-dimensional medical file, wherein the required data is the attributes corresponding to the model required for simulation training.

In an exemplary embodiment, the apparatus further includes, but is not limited to:

The medical image acquisition module 400 is used to acquire medical images in different states, wherein the medical images include normal and diseased images;

Classification module 410, used to classify all medical images based on organs and tissues;

The content determination module 420 segments all classified medical images to obtain corresponding target regions, and is used to determine the target regions as medical content data, wherein the target regions are regions where organs and tissues are located in the medical images.

In an exemplary embodiment, the apparatus further includes, but is not limited to:

A label data acquisition module 500 is used to acquire corresponding label data based on the organs and tissues corresponding to the target area;

A target three-dimensional medical model acquisition module 510 performs three-dimensional reconstruction and rendering on the target area to obtain a corresponding target three-dimensional medical model;

The 3D medical content model file determination module 520 is configured to associate the tag data with the target 3D medical model to determine a corresponding 3D medical content data file.

In an exemplary embodiment, the apparatus further includes, but is not limited to:

A multi-view image acquisition module 600 is used to acquire a corresponding multi-view image set based on multiple angles, wherein the multi-view image set includes point cloud images of multiple angles;

A feature point extraction module 610 is configured to extract feature points based on the multi-view image set and calculate the three-dimensional coordinates corresponding to the feature points, wherein the feature points include corner points and edge points;

The reconstruction module 620 is used to reconstruct a three-dimensional model based on the three-dimensional coordinates of the feature points, wherein the three-dimensional model includes a medical instrument model and a medical operation scene model.

In an exemplary embodiment, the apparatus further includes, but is not limited to:

The classification data acquisition module 700 is used to obtain classification data corresponding to the three-dimensional model;

A classification module 710 is configured to classify each 3D model into different levels based on the classification data, wherein each level has a corresponding level label when classifying the 3D model into different levels.

The 3D data file acquisition module 720 associates the 3D model with all corresponding hierarchical tags to obtain the corresponding 3D data file, wherein the 3D data file includes the 3D medical instrument data file and the 3D medical operation scene data file.

In an exemplary embodiment, the apparatus further includes, but is not limited to:

The limited index determination module 800 is used to determine the corresponding limited index after the user enters the interface corresponding to the medical virtual content library, wherein the interface includes scene limitation, instrument limitation and part limitation;

A target three-dimensional data file determining module 810 searches a corresponding limited index based on the requirement data to determine a corresponding target three-dimensional data file;

A download module 820 is used to download the model storage format corresponding to the target three-dimensional data file;

The output module 830 determines the storage format of the downloaded model as the target three-dimensional model for output.

It should be noted that the above embodiment provides a three-dimensional model usage device based on a medical model content library. When using a three-dimensional model based on a medical model content library, the division of the above-mentioned functional modules is only used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of a three-dimensional model usage device based on a medical model content library will be divided into different functional modules to complete all or part of the functions described above.

In addition, the three-dimensional model usage device based on the medical model content library and the three-dimensional model usage method based on the medical model content library provided in the above embodiment belong to the same concept, and the specific way in which each module performs operations has been described in detail in the method embodiment and will not be repeated here.

Please refer to FIG. 14 . An electronic device 4000 is provided in an embodiment of the present application. The electronic device 4000 may include a desktop computer, a laptop computer, a server, etc.

In FIG. 14 , the electronic device 4000 includes at least one processor 4001 and at least one memory 4003 .

Data exchange between the processor 4001 and the memory 4003 can be achieved through at least one communication bus 4002. The communication bus 4002 may include a path for transmitting data between the processor 4001 and the memory 4003. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in FIG14, but this does not mean that there is only one bus or one type of bus.

Optionally, the electronic device 4000 may further include a transceiver 4004, which may be used for data exchange between the electronic device and other electronic devices, such as data transmission and/or data reception. It should be noted that in actual applications, the number of transceivers 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

Processor 4001 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

The memory 4003 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program instructions or codes in the form of instructions or data structures and can be accessed by the electronic device 400, but is not limited to these.

Computer-readable instructions are stored in the memory 4003 , and the processor 4001 can read the computer-readable instructions stored in the memory 4003 through the communication bus 4002 .

The computer-readable instructions are executed by one or more processors 4001 to implement the method for using a three-dimensional model based on a medical model content library in the above-mentioned embodiments.

In addition, an embodiment of the present application provides a storage medium on which computer-readable instructions are stored. The computer-readable instructions are executed by one or more processors to implement the above-mentioned method for using a three-dimensional model based on a medical model content library.

In an embodiment of the present application, a computer program product is provided, which includes computer-readable instructions. The computer-readable instructions are stored in a storage medium. One or more processors of an electronic device read the computer-readable instructions from the storage medium, load and execute the computer-readable instructions, so that the electronic device implements the three-dimensional model usage method based on the medical model content library as described above.

Compared to related technologies, this technology combines medical images collected from various scenarios, segmenting and reconstructing regions of interest to generate 3D medical models for multiple locations and scenarios. This model then creates a 3D content library based on location and medical descriptions. The rich 3D medical resources provided for mixed reality medical training and instruction can quickly provide different 3D models based on specific needs, simulating the demand for medical 3D models in various scenarios. This significantly improves the efficiency of medical teaching and training using mixed reality, addressing doctors' need for medical 3D models in a variety of different states.

It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

The above description is only part of the implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims (10)

A method for using a three-dimensional model based on a medical model content library, comprising: During medical simulation training, a medical virtual content library is called, wherein the medical virtual content library is obtained by storing three-dimensional data files; wherein medical content data, medical instrument data, and medical operation scene data are obtained, and the medical content data is converted into a three-dimensional medical content data file, the medical instrument data is converted into a three-dimensional medical instrument data file, and the medical operation scene data is converted into a three-dimensional medical operation scene data file; Storing the three-dimensional medical content data file, the three-dimensional medical instrument data file, and the three-dimensional medical operation scene data file to obtain a corresponding medical virtual content library; Obtain demand data, index the corresponding target three-dimensional data file in the medical virtual content library based on the demand data, and retrieve the corresponding target three-dimensional model based on the target three-dimensional medical file, wherein the demand data is the attributes corresponding to the model required for simulation training. The method according to claim 1, wherein, in the process of obtaining the medical content data, the method further comprises: Acquire medical images in different states, wherein the medical images include normal and diseased images; Classify all medical images based on organs and tissues; All classified medical images are segmented to obtain corresponding target areas, and the target areas are determined as medical content data, wherein the target areas are areas where organs and tissues in the medical images are located. The method according to claim 2, wherein, in the method of converting the medical content data into a three-dimensional medical content data file, the method further comprises: Acquire corresponding label data based on the organs and tissues corresponding to the target area; Performing three-dimensional reconstruction on the target area and rendering to obtain a corresponding target three-dimensional medical model; The tag data is associated with the target three-dimensional medical model to determine a corresponding three-dimensional medical content data file. The method according to claim 2, wherein, in the method for collecting medical device data and medical operation scene data, the method further comprises: Acquire a corresponding multi-view image set based on multiple angles, wherein the multi-view image set includes images from multiple angles; Extracting feature points based on the multi-view image set and calculating the three-dimensional coordinates corresponding to the feature points, wherein the feature points include corner points and edge points; A three-dimensional model is reconstructed based on the three-dimensional coordinates of the feature points, wherein the three-dimensional model includes a medical instrument model and a medical operation scene model. The method according to claim 1, wherein after determining the three-dimensional model, the method further comprises: Obtain classification data corresponding to the three-dimensional model; Classifying each three-dimensional model level by level based on the classification data, wherein when the three-dimensional model is classified level by level, each level corresponds to a level label; The three-dimensional model is associated with all corresponding hierarchical labels to obtain a corresponding three-dimensional data file, wherein the three-dimensional data file includes a three-dimensional medical instrument data file and the three-dimensional medical operation scene data file. The method according to claim 1, wherein, in the method of retrieving the corresponding target three-dimensional model, the method further comprises: After the user enters the interface corresponding to the medical virtual content library, wherein the interface includes scene limitation, instrument limitation and part limitation, the corresponding limitation index is determined; Searching the corresponding limited index based on the demand data to determine the corresponding target three-dimensional data file; Downloading the model storage format corresponding to the target three-dimensional data file; The downloaded model storage format is determined as the target 3D model and output. A three-dimensional model using device based on a medical model content library, characterized by comprising: A file acquisition module, during medical simulation training, calls a medical virtual content library, which stores three-dimensional data files; wherein the module acquires medical content data, medical instrument data, and medical operation scene data, and the user converts the medical content data into a three-dimensional medical content data file, the medical instrument data into a three-dimensional medical instrument data file, and the medical operation scene data into a three-dimensional medical operation scene data file; a storage module, configured to store the three-dimensional medical content data file, the three-dimensional medical instrument data file, and the three-dimensional medical operation scene data file, and obtain a corresponding medical virtual content library; The retrieval module obtains the demand data, indexes the corresponding target three-dimensional data file in the medical virtual content library based on the demand data, and retrieves the corresponding target three-dimensional model based on the target three-dimensional medical file, wherein the demand data is the attributes corresponding to the model required for simulation training. The device according to claim 7, characterized in that the device further comprises: A medical image acquisition module is used to acquire medical images in different states, wherein the medical images include normal and diseased images; Classification module, used to classify all medical images based on organs and tissues; The content determination module segments all classified medical images to obtain corresponding target areas, and is used to determine the target areas as medical content data, wherein the target areas are areas where organs and tissues are located in the medical images. An electronic device, comprising: at least one processor and at least one memory, wherein: The memory has computer-readable instructions stored thereon; The computer-readable instructions are executed by one or more of the processors, so that the electronic device implements a method for using a three-dimensional model based on a medical model content library as described in any one of claims 1 to 6. A storage medium having computer-readable instructions stored thereon, characterized in that the computer-readable instructions are executed by one or more processors to implement a method for using a three-dimensional model based on a medical model content library as described in any one of claims 1 to 6.