CN117635795A - Hybrid semitransparent medical visualization method and system applied to surgical navigation - Google Patents
Hybrid semitransparent medical visualization method and system applied to surgical navigation Download PDFInfo
- Publication number
- CN117635795A CN117635795A CN202311762528.8A CN202311762528A CN117635795A CN 117635795 A CN117635795 A CN 117635795A CN 202311762528 A CN202311762528 A CN 202311762528A CN 117635795 A CN117635795 A CN 117635795A
- Authority
- CN
- China
- Prior art keywords
- color
- polygon
- pass
- voxel
- pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/005—General purpose rendering architectures
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T15/00—3D [Three Dimensional] image rendering
- G06T15/06—Ray-tracing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2065—Tracking using image or pattern recognition
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/41—Medical
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2210/00—Indexing scheme for image generation or computer graphics
- G06T2210/52—Parallel processing
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer Graphics (AREA)
- Health & Medical Sciences (AREA)
- Software Systems (AREA)
- Surgery (AREA)
- Life Sciences & Earth Sciences (AREA)
- Public Health (AREA)
- Medical Informatics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Veterinary Medicine (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Animal Behavior & Ethology (AREA)
- Heart & Thoracic Surgery (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Robotics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Geometry (AREA)
- Image Generation (AREA)
Abstract
The invention discloses a mixed semitransparent medical visualization method and a system applied to surgical navigation, belonging to the technical field of visualization, wherein the method comprises the following steps: the speed is ensured while the sequence related semitransparent correctness is ensured by designing a double-pass architecture supporting two different pipelines of the polygon meshes and the voxels; the first pass: dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel; the second pass: firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and the voxel rendering processing of polygon color mixing or ray tracing is carried out for the first segment whether the first segment is in the voxel, and the final pixel color is generated. Through the scheme, only the double-pass structure is used, the parallelism and the calculation density of the renderer are better, the problem that whether the sequence of the tissue structure is correctly displayed before and after the medical image is semitransparent displayed can be effectively solved, and meanwhile real-time and efficient calculation in operation is realized.
Description
Technical Field
The disclosure relates to the technical field of visualization, in particular to a hybrid semitransparent medical visualization method and system applied to surgical navigation.
Background
In medical image analysis and surgical navigation applications, a great advantage of surgical navigation is that occlusion problems can be resolved. For example: when a doctor looks at orthopedics directly, the nerves and blood vessels in the bone can be displayed. This structure is normally not visible in direct view and will be displayed in a semi-transparent manner in the surgical navigation system. In surgical navigation applications, the conventional translucency approach is to mix colors of different objects, but not see the front-to-back relationship of the objects. Although the method is simple and efficient, doctors are easy to mislead, and medical accidents are easy to occur in medical use. In surgical navigation applications, it is critical that the visualization system correctly display the front-to-back positional relationship of objects, which is known in the field of computer visualization as order dependent transparency.
Although order dependent transparent methods have been proposed, there are serious computational efficiency problems. It is difficult to apply in real-time applications such as surgical navigation. The reason is that: the prior art method requires multiple rendering passes and then synthesizing the final image, the rendering time being several times that of the conventional method. The reasons for rendering multiple passes are mainly: 1) Since the geometry in the renderer is committed together and processed in parallel, it is difficult to determine the precedence order. And the sequence of the geometric bodies is different in different areas. If the display effect with the correct semitransparent sequence is required, the renderer can only render a part with fixed sequence or a part with a certain depth at a time, and then finally synthesizes the parts; 2) Medical surgery navigation divides two different data of a visual polygon model (mesh) and a voxel (voxel), and the data formats of voxel rendering and polygon rendering are different, and the rendering modes are different. When rendering, the polygon and the voxel are required to be separately rendered, and finally mixed to synthesize a final image.
Aiming at the problems, the invention provides a mixed semitransparent medical visualization method applied to surgical navigation, which combines more than ten computing functions in two different pipelines of a polygonal grid and a voxel into two functions, namely a double pass (dual pass) architecture, so that the accuracy is ensured and the speed is ensured. Regardless of how complex the scene is, the renderer only needs to render twice (dual pass) to achieve order dependent translucency, and can support both polygonal meshes and voxels. Because the method fully utilizes the parallelism of the GPU, and the computation density is higher, the computation efficiency is higher, and the rendering time is several tenths of that of the traditional method.
Disclosure of Invention
In view of the above, the embodiments of the present disclosure provide a hybrid semitransparent medical visualization method and system applied to surgical navigation, which can effectively solve the problem that whether the sequence of the tissue structure is correctly displayed before and after semitransparent display of the medical image, and simultaneously realize real-time and efficient calculation in surgery.
1) Ensuring the correctness of the order dependent translucency:
when surgical navigation uses a semi-transparent display, conventional semi-transparent methods simply mix the colors of different objects, but do not see the front-to-back relationship of the objects. Although the method is simple and efficient, doctors are easy to mislead, and medical accidents are easy to occur in medical use. In surgical navigation applications, it is critical that the visualization system correctly display the front-to-back positional relationship of objects, which is known in the field of computer visualization as order dependent transparency. By the sequence related transparent structure design method, semitransparent medical visualization effect of correct display of the front and back sequence can be realized.
2) Two different pipelines of polygon meshes (meshes) and voxels (voxels) are supported:
in medical operation navigation, two different data of a visual polygon model (mesh) and a voxel (voxel) exist, the data formats of voxel rendering and polygon rendering are different, and the rendering modes are different. When rendering, the polygon and the voxel are required to be separately rendered, and finally mixed to synthesize a final image. The invention provides a mixed semitransparent medical visualization method applied to surgical navigation, which combines more than ten computing functions in two different pipelines of a polygonal network and voxels into two functions, namely a double pass architecture. Order dependent translucence may be implemented and both polygon meshes and voxels may be supported.
3) Guarantee speed while guaranteeing correctness:
the existing order related transparent technical method needs to render for multiple times and then synthesize a final image, the rendering time is several times that of the conventional method, the serious calculation efficiency problem exists, and the real-time requirement of surgical navigation is difficult to reach. The invention designs a double pass (dual pass) architecture which simultaneously supports the mixing of polygon mesh rendering and voxel rendering, and a renderer only needs to render twice and then synthesize no matter how complex the scene is. The dual-pass architecture adopted by the invention has better parallelism and computation density of the renderer, effectively solves the problem that whether the sequence of the tissue structure is displayed correctly before and after the medical image is displayed in a semitransparent way, simultaneously realizes real-time and efficient computation in operation, ensures the accuracy and simultaneously ensures the speed.
In a first aspect, embodiments of the present disclosure provide a hybrid semi-transparent medical visualization method for surgical navigation, comprising:
in a medical visualization system, a double pass (dual pass) architecture supporting two different pipelines of a polygonal grid and a voxel is designed, so that semitransparent medical visualization effects of correct front-back sequence display are realized, and speed is ensured while sequence related semitransparent correctness is ensured;
the dual pass architecture is to combine more than ten computing functions in two different pipelines of a polygon mesh (mesh) and a voxel (voxel) into two functions, and the renderer only needs to render twice and then synthesize the functions;
the specific implementation steps of the dual pass architecture design structure comprise:
first pass: dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel;
the second pass: firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and the voxel rendering processing of polygon color mixing or ray tracing is carried out for the first segment whether the first segment is in the voxel, and the final pixel color is generated.
According to a specific implementation manner of the embodiment of the present disclosure, in the first pass, all three-dimensional objects are divided into a plurality of triangle meshes, and a polygon semitransparent linked list is established for each pixel, including:
all objects are drawn using a pipeline of polygons (mesh). Voxel data is approximated by boundary polygons, the system being parallel triangle by triangle. The pass renders and does not output color, and establishes a polygon semitransparent chain table for each pixel, which is used for storing transparency, depth and color information of the polygon.
According to a specific implementation manner of the embodiment of the disclosure, in the second pass, firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and for whether the first segment is in the voxel, the voxel rendering process of polygon color mixing or ray tracing is performed, and the final pixel color is generated, including:
using a line trace pipeline, the system is parallel on a pixel-by-pixel basis. Mainly using ray tracing color mixing pipeline, the system calculates polygon chain table of each pixel in parallel. And ordering the first point in the linked list according to the depth order, and judging whether to perform polygon color mixing or ray tracing voxel rendering processing on the ordered vertexes according to the condition between two adjacent points. If the adjacent two points are located in the blank area, performing sequence-related color mixing on the blank area; if the two adjacent points are located in the voxels, ray tracing is performed, color calculation is performed simultaneously until the last node in the linked list is processed, and finally, the color is output.
According to a specific implementation manner of the embodiment of the disclosure, in the second pass, firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and for whether the first segment is in the voxel, the voxel rendering process of polygon color mixing or ray tracing is performed, and the final pixel color is generated, and further including:
and simultaneously carrying out polygon color mixing and ray tracing-based voxel rendering treatment on each sampling point through a dual pass (dual pass) design structure of polygon mesh rendering and voxel rendering mixing to obtain final pixel colors, namely color values synthesized by voxels of each pixel on a fragment shader and a polygon patch, thereby obtaining semitransparent visual effects of correctly displaying the tissue structure in front-back sequence.
According to a specific implementation of an embodiment of the disclosure, the method further includes:
when the background object is covered by the upper object, modeling is performed by two variables, and the color at the time of superposition is obtained by calculation according to the formula (1) of the graphic OVER operation:
formula (1): c (C) f =C 1 +(1-α 1 )C 0
Wherein α represents transparency, C represents color, C 0 Represents background color, C 1 Color of upper object representing background, C f Representation for background color C 0 Is colored C 1 The final color of the partial coverage of the surface.
According to a specific implementation of an embodiment of the disclosure, the method further includes:
when a plurality of layers of operations are performed, mixed superposition is performed layer by layer in the order from back to front, data are recursively applied, and the color at the final superposition is obtained through calculation according to the formula (2):
formula (2): c (C) f =[C n +(1-α n )...[C 2 +(1-α 2 )[C 1 +(1-α 1 )C 0 ]]...]
Wherein alpha is n Indicating the transparency of the nth layer itself, C n Represents the color of the nth layer, C 0 Represents background color, C f Indicating the color of the mixed layers 0 to n.
According to a specific implementation of an embodiment of the disclosure, the method further includes:
when the same object is at different hierarchical positions, the displayed colors are different due to different front-to-back sequences, and the color error calculation process comprises the following steps:
for A, B, C objects present in the medical image, the color thereof is denoted as C 1 、C 2 And C 3 The transparency of which is denoted as alpha 1 、α 2 And alpha 3 ;
When the object C is positioned between the object A and the object B, the rays from the viewpoint sequentially pass through the object A, the object C and the object B, and the color finally displayed by the object C is C f1 :
C f1 =C 3 +(1-α 3 )C 2
When the object C is in front of the object A, the rays from the viewpoint sequentially pass through the object C, the object A and the object B, and the color finally displayed by the object C is C f2 :
C f2 =C 3 +(1-α 3 )[C 1 +(1-α 1 )C 2 ]
The color error of the front-back relation of the same object at different positions is C e :
C e =C f2 -C f1 =(1-α 3 )(C 1 -α 1 C 2 )
In operation, the doctor judges the front-back sequence position of the tissue and organ according to different overlapped colors.
In a second aspect, embodiments of the present disclosure provide a hybrid semi-transparent medical visualization system for surgical navigation, comprising:
a dual pass module for combining more than ten computing functions in two different pipelines of a polygon mesh and a voxel into two functions, wherein a renderer only needs to render twice and then synthesize the functions, and the dual pass module is used for realizing semitransparent medical visualization effects of front-back sequential correct display by designing a dual pass architecture supporting two different pipelines of the polygon mesh and the voxel in a medical visualization system, guaranteeing speed while guaranteeing sequence related semitransparent correctness, and comprises:
the first pass sub-module: using a polygon pipeline, the polygons are parallel one by one. Dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel;
the second pass sub-module: using a line trace pipeline, pixel-by-pixel parallelism is used. And simultaneously performing polygon color mixing and voxel rendering processing based on ray tracing to obtain the final pixel color.
In a third aspect, the presently disclosed embodiments also provide a non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the hybrid semi-transparent medical visualization method of the first aspect or any implementation of the first aspect as applied to surgical navigation.
In a fourth aspect, the presently disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the hybrid semi-transparent medical visualization method of the first aspect or any implementation of the first aspect as applied to surgical navigation.
A hybrid semitransparent medical visualization scheme for surgical navigation in an embodiment of the present disclosure includes: in a medical visualization system, a double pass (dual pass) architecture supporting two different pipelines of a polygonal grid and a voxel is designed, so that semitransparent medical visualization effects of correct front-back sequence display are realized, and speed is ensured while sequence related semitransparent correctness is ensured; the dual pass architecture is to combine more than ten computing functions in two different pipelines of a polygon mesh (mesh) and a voxel (voxel) into two functions, and the renderer only needs to render twice and then synthesize the functions; the first pass: dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel; the second pass: firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and the voxel rendering processing of polygon color mixing or ray tracing is carried out for the first segment whether the first segment is in the voxel, and the final pixel color is generated. Through the processing scheme disclosed by the invention, only the double pass structure is used, so that the problem that whether the sequence of the tissue structure is displayed correctly before and after the medical image is displayed in a semitransparent manner is effectively solved, and meanwhile, real-time and efficient calculation in operation is realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings that are needed in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present disclosure, and other drawings may be obtained according to these drawings without inventive effort to a person of ordinary skill in the art.
Fig. 1 is a schematic flow chart of a hybrid semitransparent medical visualization method for surgical navigation according to an embodiment of the present disclosure;
FIG. 2 is a transparent schematic diagram related to the order provided by an embodiment of the present disclosure;
FIG. 3 is a schematic diagram illustrating a drawing sequence of objects during rendering according to an embodiment of the present disclosure;
FIG. 4 is a schematic diagram of a polygon semitransparent linked list according to an embodiment of the present disclosure;
fig. 5 is a schematic diagram of a hybrid semi-transparent medical visualization volume rendering for surgical navigation provided by an embodiment of the present disclosure.
Detailed Description
Referring to fig. 1, 2, 3, 4 and 5, the present invention discloses a hybrid semitransparent medical visualization method applied to surgical navigation, comprising:
in a medical visualization system, a double pass (dual pass) architecture supporting two different pipelines of a polygonal grid and a voxel is designed, so that semitransparent medical visualization effects of correct front-back sequence display are realized, and speed is ensured while sequence related semitransparent correctness is ensured;
the dual pass architecture is to combine more than ten computing functions in two different pipelines of a polygon mesh (mesh) and a voxel (voxel) into two functions, and the renderer only needs to render twice and then synthesize the functions;
the specific implementation steps of the dual pass architecture design structure comprise:
first pass: dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel;
the second pass: firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and the voxel rendering processing of polygon color mixing or ray tracing is carried out for the first segment whether the first segment is in the voxel, and the final pixel color is generated.
Order dependent transparency
In medical imaging, semi-transparent displays are common, but it is still a major challenge to correctly display the tissue structure in order. For the design of the front-back sequence structure, which is called sequence related transparency, the specific method is as follows:
1. obtaining the color and transparency of various tissue structures in the three-dimensional reconstruction model of the medical navigation image;
2. calculating according to the sequence related transparent mixing formula to obtain the color when in superposition:
1) The two layers are stacked and mixed:
when the background object is covered by the upper object, modeling is mathematically performed by two variables, and the color at the time of superposition is obtained by calculation according to formula (1) of the graphic OVER operation:
formula (1): c (C) f =C 1 +(1-α 1 )C 0
Where α represents transparency (α=1 represents complete coverage of the background, α=0.5 represents coverage of half of the background area), C represents color, C 0 Represents background color, C 1 Color of upper object representing background, C f Representation for background color C 0 Is colored C 1 The final color of the partial coverage of the surface.
2) Multi-level superposition mixing:
when performing multiple hierarchical operations, the data is applied recursively, mixed overlapping layer by layer in order from back to front. According to formula (2), the color at the time of final superposition is calculated:
formula (2): c (C) f =[C n +(1-α n )...[C 2 +(1-α 2 )[C 1 +(1-α 1 )C 0 ]]...]
Wherein alpha is n Indicating the transparency of the nth layer itself, C n Represents the color of the nth layer, C 0 Represents background color, C f Indicating the color of the mixed layers 0 to n.
3. When the same object is in different hierarchical positions, the displayed colors are different due to different front-to-back sequences, and the color error calculation process is as follows:
1) There are A, B, C objects in the medical image, and the color and transparency are shown in the following table:
| object | Color of | Transparency of the film |
| A | C 1 | α 1 |
| B | C 2 | α 2 |
| C | C 3 | α 3 |
2) When the object C is positioned between the object A and the object B, the rays from the viewpoint sequentially pass through the object A, the object C and the object B, and the color finally displayed by the object C is C f1 :
C f1 =C 3 +(1-α 3 )C 2
3) When the object C is in front of the object A, the rays from the viewpoint sequentially pass through the object C, the object A and the object B, and the color finally displayed by the object C is C f2 :
C f2 =C 3 +(1-α 3 )[C 1 +(1-α 1 )C 2 ]
4) The color error of the front-back relation of the same object at different positions is C e :
C e =C f2 -C f1 =(1-α 3 )(C 1 -α 1 C 2 )
5) In the operation, the sequential positions of the tissue and organ are judged according to different overlapped colors.
4. In order to ensure accuracy of order correlation, the drawing order of the respective objects is ensured at the time of rendering. As shown in fig. 3, the drawing is performed in the order from front to back. If two objects intersect, multiple renderings are performed according to the order. The order of the objects cannot be changed at each rendering. The first time, only drawing A in front of B, the second time, only drawing B in front of A … …, is rendered for multiple times, and finally, the correctness of the drawing order of each object is realized.
Double pass structural design
The invention adopts a design structure of double pass (dual pass), and the two-channel real-time and efficient rendering is realized through polygonal grid rendering and voxel rendering, so that the processing efficiency can be improved by more than 70 times, and the semitransparent medical visualization effect can be realized more quickly and more accurately.
1. First pass: using a polygon pipeline, the polygons are parallel one by one. Dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel
Polygonal mesh models are widely used in the fields of computer graphics, virtual reality, computer aided design technology, medical image systems, and the like. One type of polygon is a triangular patch, which is also often used to represent various geometries such as organs and surgical tools in three-dimensional modeling of surgical navigation systems. In the dual pass (dual pass) design structure of the polygon mesh rendering and voxel rendering mixture disclosed by the invention, the first pass performs the following operations:
1) All objects are drawn using a pipeline of polygons. The voxel data are approximated by boundary polygons, and the system divides all three-dimensional objects into a plurality of triangle meshes in parallel by triangles;
2) The rendering process does not output color, and a polygon semitransparent chain table is established for each pixel and used for storing the transparency, depth and color information of the polygons so as to facilitate correct ordering and color mixing during rendering.
2. The second pass: using a line trace pipeline, pixel-by-pixel parallelism is used. Simultaneous polygon color blending and ray tracing based voxel rendering
Voxel rendering is a rendering technology in the graphic processing process, and efficient and real three-dimensional image display is realized by calculating and rendering voxels in a three-dimensional space. In the process of calculating voxel space mapping and rendering voxel colors by ray tracing, the invention adopts a double pass (dual pass) design structure of mixing volume rendering and polygon patch rendering, correctly sequences voxels and polygon patches in real time in a fragment shader of a GPU, and renders the colors by using a sequence related transparent method so as to solve the problem that whether the front and rear sequences of the tissue structures are correctly displayed when semitransparent display is carried out in medical images. In the dual pass (dual pass) design structure of the polygon mesh rendering and voxel rendering mixture disclosed by the invention, the second pass performs the following operations:
1) Using a line trace pipeline, the system is parallel on a pixel-by-pixel basis. Mainly using ray tracing color mixing pipeline, the system calculates polygon chain table of each pixel in parallel. And ordering the first point in the linked list according to the depth order, and judging whether to perform polygon color mixing or ray tracing voxel rendering processing on the ordered vertexes according to the condition between two adjacent points.
2) If the adjacent two points are located in the blank area, performing sequence-related color mixing on the blank area; if the two adjacent points are located in the voxels, ray tracing is performed, color calculation is performed simultaneously until the last node in the linked list is processed, and finally, the color is output.
3) And simultaneously carrying out polygon color mixing and ray tracing-based voxel rendering treatment on each sampling point through a dual pass (dual pass) design structure of polygon mesh rendering and voxel rendering mixing to obtain final pixel colors, namely color values synthesized by voxels of each pixel on a fragment shader and a polygon patch, thereby obtaining semitransparent visual effects of correctly displaying the tissue structure in front-back sequence.
According to the invention, the interactive results of the models are processed in real time, so that high-precision and real-time visual rendering images are provided for users. Regardless of how complex the actual application scene is, the high-efficiency calculation can be performed by the mixed semitransparent visualization method.
According to a specific implementation manner of the embodiment of the present disclosure, in the first pass, all three-dimensional objects are divided into a plurality of triangle meshes, and a polygon semitransparent linked list is built for each pixel, including:
all objects are drawn using a pipeline of polygons. Voxel data is approximated by boundary polygons, the system being parallel triangle by triangle. The pass renders and does not output color, and establishes a polygon semitransparent chain table for each pixel, which is used for storing transparency, depth and color information of the polygon.
According to a specific implementation manner of the embodiment of the disclosure, in the second pass, firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and for whether the first segment is in the voxel, the voxel rendering process of polygon color mixing or ray tracing is performed, and the final pixel color is generated, including:
using a line trace pipeline, the system is parallel on a pixel-by-pixel basis. Mainly using ray tracing color mixing pipeline, the system calculates polygon chain table of each pixel in parallel. And ordering the first point in the linked list according to the depth order, and judging whether to perform polygon color mixing or ray tracing voxel rendering processing on the ordered vertexes according to the condition between two adjacent points. If the adjacent two points are located in the blank area, performing sequence-related color mixing on the blank area; if the two adjacent points are located in the voxels, ray tracing is performed, color calculation is performed simultaneously until the last node in the linked list is processed, and finally, the color is output.
According to a specific implementation manner of the embodiment of the disclosure, in the second pass, firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and for whether the first segment is in the voxel, the voxel rendering process of polygon color mixing or ray tracing is performed, and the final pixel color is generated, and further including:
searching a polygon semitransparent linked list for each acquired pixel while performing volume rendering, and acquiring color and depth value information of all triangular grids corresponding to the pixel;
and simultaneously carrying out polygon color mixing and ray tracing-based voxel rendering treatment on each sampling point through a dual pass (dual pass) design structure of polygon mesh rendering and voxel rendering mixing to obtain final pixel colors, namely color values synthesized by voxels of each pixel on a fragment shader and a polygon patch, thereby obtaining semitransparent visual effects of correctly displaying the tissue structure in front-back sequence.
According to a specific implementation of an embodiment of the disclosure, the method further includes:
when the background object is covered by the upper object, modeling is performed by two variables, and the color at the time of superposition is obtained by calculation according to the formula (1) of the graphic OVER operation:
formula (1): c (C) f =C 1 +(1-α 1 )C 0
Wherein α represents transparency, C represents color, C 0 Represents background color, C 1 Color of upper object representing background, C f Representation for background color C 0 Is colored C 1 The final color of the partial coverage of the surface.
According to a specific implementation of an embodiment of the disclosure, the method further includes:
when a plurality of layers of operations are performed, mixed superposition is performed layer by layer in the order from back to front, data are recursively applied, and the color at the final superposition is obtained through calculation according to the formula (2):
formula (2): c (C) f =[C n +(1-α n )...[C 2 +(1-α 2 )[C 1 +(1-α 1 )C 0 ]]...]
Wherein alpha is n Indicating the transparency of the nth layer itself, C n Represents the color of the nth layer, C 0 Represents background color, C f Indicating the color of the mixed layers 0 to n.
According to a specific implementation of an embodiment of the disclosure, the method further includes:
when the same object is at different hierarchical positions, the displayed colors are different due to different front-to-back sequences, and the color error calculation process comprises the following steps:
for A, B, C objects present in the medical image, the color thereof is denoted as C 1 、C 2 And C 3 The transparency of which is denoted as alpha 1 、α 2 And alpha 3 ;
When the object C is positioned between the object A and the object B, the rays from the viewpoint sequentially pass through the object A, the object C and the object B, and the color finally displayed by the object C is C f1 :
C f1 =C 3 +(1-α 3 )C 2
When the object C is in front of the object A, the rays from the viewpoint sequentially pass through the object C, the object A and the object B, and the color finally displayed by the object C is C f2 :
C f2 =C 3 +(1-α 3 )[C 1 +(1-α 1 )C 2 ]
The color error of the front-back relation of the same object at different positions is C e :
C e =C f2 -C f1 =(1-α 3 )(C 1 -α 1 C 2 )
In operation, the doctor judges the front-back sequence position of the tissue and organ according to different overlapped colors.
Corresponding to the above method embodiments, the present application also discloses a hybrid semitransparent medical visualization system for surgical navigation, comprising:
a dual pass module for combining more than ten computing functions in two different pipelines of a polygon mesh and a voxel into two functions, wherein a renderer only needs to render twice and then synthesize the functions, and the dual pass module is used for realizing semitransparent medical visualization effects of front-back sequential correct display by designing a dual pass architecture supporting two different pipelines of the polygon mesh and the voxel in a medical visualization system, guaranteeing speed while guaranteeing sequence related semitransparent correctness, and comprises:
the first pass sub-module: using a polygon pipeline, the polygons are parallel one by one. Dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel;
the second pass sub-module: using a line trace pipeline, pixel-by-pixel parallelism is used. And simultaneously performing polygon color mixing and voxel rendering processing based on ray tracing to obtain the final pixel color.
The foregoing is merely specific embodiments of the disclosure, but the protection scope of the disclosure is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the disclosure are intended to be covered by the protection scope of the disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.
Claims (8)
1. A hybrid translucency medical visualization method for surgical navigation, comprising:
in a medical visualization system, a double-pass dual pass architecture supporting two different pipelines of a polygonal grid and a voxel is designed, so that semitransparent medical visualization effects of correct front-back sequence display are realized, and speed is ensured while sequence related semitransparent correctness is ensured;
the double pass architecture is to combine more than ten calculation functions in two different pipelines of the polygon mesh and the voxel volume into two functions, and the renderer only needs to render twice and then synthesize;
the implementation steps of the dual pass architecture design structure comprise:
first pass: dividing all three-dimensional objects into a plurality of triangular grids, and establishing a polygonal semitransparent linked list for each pixel;
the second pass: firstly, the polygon linked list is ordered according to the depth, then the depth space is segmented, and the voxel rendering processing of polygon color mixing or ray tracing is carried out for the first segment whether the first segment is in the voxel, and the final pixel color is generated.
2. The method of claim 1, wherein the first pass divides all three-dimensional objects into a plurality of triangular meshes and creates a polygon translucent linked list for each pixel, comprising:
all objects are drawn using a pipeline of polygons. The voxel data is approximated by boundary polygons, the system renders no color output in triangle-by-triangle parallel, and establishes a polygon translucent linked list for each pixel for storing the transparency, depth and color information of the polygon.
3. The method of claim 2, wherein the second pass first orders the linked list of polygons by depth and then segments the depth space, performs voxel rendering processing for polygon color blending or ray tracing for whether the first segment is within a voxel, and generates a final pixel color, comprising:
using a line tracking pipeline, enabling the system to be parallel in a pixel-by-pixel mode, using a color mixing pipeline of the line tracking, enabling the system to calculate a polygon chain table of each pixel in parallel, sequencing a first point in the chain table according to a depth order, judging whether polygon color mixing or voxel rendering processing of the line tracking is carried out according to the situation between two adjacent points for the sequenced vertexes, and carrying out color mixing related to the sequence if the adjacent points are located in a blank area; if the two adjacent points are located in the voxels, ray tracing is performed, color calculation is performed simultaneously until the last node in the linked list is processed, and finally, the color is output.
4. A method according to claim 3, wherein the second pass first orders the linked list of polygons by depth and then segments the depth space, performs voxel rendering processing for polygon color blending or ray tracing for whether the first segment is within a voxel, and generates a final pixel color, further comprising:
and simultaneously carrying out polygon color mixing and ray tracing-based voxel rendering processing on each sampling point through a double-pass dual pass design structure of polygon mesh rendering and voxel rendering mixing to obtain final pixel colors, namely color values synthesized by voxels of each pixel and a polygon patch on a fragment shader, thereby obtaining semitransparent visual effects of correctly displaying the front and rear sequences of the tissue structure.
5. The method according to claim 1, wherein the method further comprises:
when the background object is covered by the upper object, modeling is performed by two variables, and the color at the time of superposition is obtained by calculation according to the formula (1) of the graphic OVER operation:
formula (1): c (C) f =C 1 +(1-α 1 )C 0
Wherein α represents transparency, C represents color, C 0 Represents background color, C 1 Color of upper object representing background, C f Representation for background color C 0 Is colored C 1 The final color of the partial coverage of the surface.
6. The method of claim 5, wherein the method further comprises:
when a plurality of layers of operations are performed, mixed superposition is performed layer by layer in the order from back to front, data are recursively applied, and the color at the final superposition is obtained through calculation according to the formula (2):
formula (2): c (C) f =[C n +(1-α n )...[C 2 +(1-α 2 )[C 1 +(1-α 1 )C 0 ]]...]
Wherein alpha is n Indicating the transparency of the nth layer itself, C n Represents the color of the nth layer, C 0 Represents background color, C f Indicating the color of the mixed layers 0 to n.
7. The method of claim 6, wherein the method further comprises:
when the same object is at different hierarchical positions, the displayed colors are different due to different front-to-back sequences, and the color error calculation process comprises the following steps:
for A, B, C objects present in the medical image, the color thereof is denoted as C 1 、C 2 And C 3 The transparency of which is denoted as alpha 1 、α 2 And alpha 3 ;
When the object C is positioned between the object A and the object B, the rays from the viewpoint sequentially pass through the object A, the object C and the object B, and the color finally displayed by the object C is C f1 :
C f1 =C 3 +(1-α 3 )C 2
When the object C is in front of the object A, the rays from the viewpoint sequentially pass through the object C, the object A and the object B, and the object CThe final color displayed is C f2 :
C f2 =C 3 +(1-α 3 )[C 1 +(1-α 1 )C 2 ]
The color error of the front-back relation of the same object at different positions is C e :
C e =C f2 -C f1 =(1-α 3 )(C 1 -α 1 C 2 )
In operation, the doctor judges the front-back sequence position of the tissue and organ according to different overlapped colors.
8. A hybrid semi-transparent medical visualization system for surgical navigation, comprising:
the utility model provides a two pass dual pass module, merges into two functions with the polygon net with ten more computational functions in voxel two different pipelines, and the renderer only need render twice and then synthesize for in medical visualization system, through the design support the two pass dual pass framework of polygon net and voxel two different pipelines, realize the translucent medical visual effect that order correctly shows from beginning to end, guarantee speed when the relevant translucent correctness of order guarantees, two pass dual pass module includes:
the first pass sub-module: using a polygon assembly line to divide all three-dimensional objects into a plurality of triangle grids in parallel by polygons, and establishing a polygon semitransparent linked list for each pixel;
the second pass sub-module: and using a line tracking pipeline to perform pixel-by-pixel parallelism, and simultaneously performing polygon color mixing and voxel rendering processing based on the line tracking to obtain the final pixel color.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311762528.8A CN117635795A (en) | 2023-12-20 | 2023-12-20 | Hybrid semitransparent medical visualization method and system applied to surgical navigation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311762528.8A CN117635795A (en) | 2023-12-20 | 2023-12-20 | Hybrid semitransparent medical visualization method and system applied to surgical navigation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117635795A true CN117635795A (en) | 2024-03-01 |
Family
ID=90023523
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311762528.8A Pending CN117635795A (en) | 2023-12-20 | 2023-12-20 | Hybrid semitransparent medical visualization method and system applied to surgical navigation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117635795A (en) |
-
2023
- 2023-12-20 CN CN202311762528.8A patent/CN117635795A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP3483929B2 (en) | 3D image generation method | |
| CN104794758A (en) | Three-dimensional image cutting method | |
| US20110082667A1 (en) | System and method for view-dependent anatomic surface visualization | |
| JP2007014706A (en) | Image fusion processing method, image fusion processing program, image fusion treatment device | |
| CN103077557B (en) | The implementation method that a kind of adaptive layered time big data of chest show | |
| CN103106685A (en) | Abdominal viscera three dimension visualization method based on graphic processing unit (GPU) | |
| US7778451B2 (en) | Cylindrical projected picture generation method, program, and cylindrical projected picture generation device | |
| JP4885042B2 (en) | Image processing method, apparatus, and program | |
| JP4018679B2 (en) | Rendering processing method, rendering processing program, and rendering processing apparatus | |
| JP3589654B2 (en) | Volume rendering method and its program | |
| CN111009033A (en) | OpenGL-based focal region visualization method and system | |
| Haubner et al. | Virtual reality in medicine-computer graphics and interaction techniques | |
| EP1775685B1 (en) | Information processing device and program | |
| US7586501B2 (en) | Simultaneous projection of multi-branched vessels and their context on a single image | |
| JP2019207450A (en) | Volume rendering apparatus | |
| Preim et al. | Visualization, visual analytics and virtual reality in medicine: State-of-the-art Techniques and Applications | |
| CN108510580A (en) | A kind of vertebra CT image three-dimensional visualization methods | |
| Tatarchuk et al. | Advanced interactive medical visualization on the GPU | |
| US7893938B2 (en) | Rendering anatomical structures with their nearby surrounding area | |
| Westwood | Anatomic hepatectomy planning through mobile display visualization and interaction | |
| CN117635795A (en) | Hybrid semitransparent medical visualization method and system applied to surgical navigation | |
| CN103745495A (en) | Medical volume data based volume rendering method | |
| Kwon et al. | Reliable subsurface scattering for volume rendering in three-dimensional ultrasound imaging | |
| CN107038747B (en) | Medical image file multi-plane processing method and device | |
| CN118982610B (en) | Three-dimensional visual display method and system for diagnosis and treatment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |