CN117392105A - Detection method of Crohn's disease and intestinal tuberculosis based on full-field digital sectioning - Google Patents

Abstract

This application discloses a detection method for Crohn's disease and intestinal tuberculosis based on full-field digital slices. The method includes acquiring several full-field digital slices, determining several image blocks corresponding to each full-field digital slice, and converting each full-field digital slice into Several image blocks corresponding to the digital slices are input into a trained detection network model, and the initial prediction probability matrix corresponding to each full-field digital slice is determined through the detection network model; based on the initial prediction probability matrix of each full-field digital slice, the Prediction results for the target object. This application determines the patient-level prediction results by integrating the initial probabilities of multiple full-field digital slices, which can determine the patient-level results through multi-level classification results, improve the patient-level prediction accuracy, and thereby improve Crohn's disease and intestinal tuberculosis detection.

Description

Detection method of Crohn's disease and intestinal tuberculosis based on full-field digital sectioning

Technical field

The present application relates to the field of biological technology, and in particular to a detection method for Crohn's disease and intestinal tuberculosis based on full-field digital slices and related devices.

Background technique

Crohn’s disease (CD) is a chronic, nonspecific granulomatous disease whose exact cause remains unclear. Over time, the incidence of Crohn's disease in my country has shown an upward trend. Another disease, Intestinal tuberculosis (ITB), is a specific infectious disease caused by Mycobacterium tuberculosis and is also one of the main types of extrapulmonary tuberculosis. Although the two diseases share many similarities in symptoms, endoscopic observations, and pathology, they are prone to misdiagnosis.

The current method of identifying these two diseases through AI means generally uses pathological image diagnosis based on deep learning to improve the efficiency and accuracy of diagnosis of Crohn's disease and intestinal tuberculosis. However, the current pathological image diagnosis model mainly focuses on the slice level and ignores the patient-level classification that is more closely related to clinical practice, thus affecting the accuracy of identification.

Therefore, the existing technology still needs to be improved and improved.

Contents of the invention

The technical problem to be solved by this application is to provide a detection method and related device for Crohn's disease and intestinal tuberculosis based on full-field digital slices in view of the shortcomings of the existing technology.

In order to solve the above technical problems, the first aspect of the embodiment of the present application provides a detection method for Crohn's disease and intestinal tuberculosis based on full-field digital slices. The method includes:

Obtain a number of full-field digital slices, and determine a number of image blocks corresponding to each full-field digital slice, where the several full-field digital slices are full-field digital slices of the same target object;

For each full-field digital slice, input several image blocks corresponding to the full-field digital slice into a trained detection network model, and determine the initial prediction probability matrix corresponding to each full-field digital slice through the detection network model;

Based on the initial prediction probability matrix of each full-field digital slice, a prediction result of the target object is determined, wherein the prediction result includes a Crohn's disease category, an intestinal tuberculosis category, or a normal tissue category.

According to the above technical means, after obtaining the patient's full field of view digital slices and further determining several image blocks corresponding to each slice, this application uses a trained detection network model to process the image blocks to determine the initial prediction probability of each slice, and then The predicted probability of full-field digital slices is determined based on the initial predicted probability. In this way, by integrating slice-level and patient-level classification results, more accurate multi-level classification can be achieved, effectively reducing misdiagnosis and further improving the accuracy of treatment.

In one implementation, the detection method of Crohn's disease and intestinal tuberculosis based on full-field digital slices, wherein the detection network model includes a feature extraction module and a classification module, and each of the several full-field digital slices is The full-field digital slice is input into a trained detection network model, and the initial prediction probability matrix corresponding to each full-field digital slice is determined through the detection network model, which specifically includes:

For each full-field digital slice, determine several image blocks corresponding to the full-field digital slice;

Input several image blocks into the feature extraction module respectively, and determine the corresponding feature vectors of each image block through the feature extraction module;

The corresponding feature vectors of each image block are input into the classification module, and the initial prediction probability matrix corresponding to the full-view digital slice is determined through the classification module.

According to the above technical means, it can be ensured that each full-field digital slice can be decomposed into several image blocks, and feature extraction is performed on each image block to realize the analysis of the full-field digital slice; at the same time, each full-field digital slice is determined through the feature extraction module The feature vectors of image blocks increase the ability to recognize and identify pathological features and provide information accuracy for diagnosis.

In one implementation, the classification module includes an attention unit and a multi-layer perceptron; the feature vector corresponding to each image block is input into the classification module, and the corresponding feature vector of the full-field digital slice is determined through the classification module The initial prediction probability matrix specifically includes:

Input the corresponding feature vectors of each image block into the attention unit, determine the attention coefficient of each image block through the attention unit, and form an attention feature matrix based on the attention coefficient and the corresponding feature vector of each image block;

The attention feature matrix is input into the multi-layer perceptron, and the initial prediction probability matrix corresponding to the full-view digital slice is determined through the multi-layer perceptron.

According to the above technical means, the attention unit is used to strengthen the focus on key image areas related to the disease, which is helpful for the detection and diagnosis of Crohn's disease and intestinal tuberculosis. Secondly, the attention-based feature weighting method can improve the accuracy of diagnosis and reduce the misdiagnosis rate of clinicians. Furthermore, the introduction of multi-layer perceptron provides learning capabilities for prediction models and can perform pattern recognition in image data.

In one implementation, the method for detecting Crohn's disease and intestinal tuberculosis based on full-field digital slices, wherein the determining several image blocks corresponding to each full-field digital slice specifically includes:

Determine a binary mask of the full-field digital slice, and fill the binary mask to obtain the foreground object outline in the full-field digital slice;

Segment the full-field digital slice based on the foreground object outline to obtain a foreground image;

The foreground image is divided into several image blocks to obtain several image blocks corresponding to the full-field digital slice.

According to the above technical means, image blocks are provided for subsequent detection network model input, ensuring the integrity and accuracy of the image.

In one implementation, the detection method of Crohn's disease and intestinal tuberculosis based on full-field digital slices, wherein the determining the binary mask of the full-field digital slice specifically includes:

Convert full-field digital slices into a label image file format, and convert the converted full-field digital slices from RGB color space to hexagonal pyramid color model color space;

The binary mask of the full-field digital slice is calculated based on the saturation channel thresholding of the full-field digital slice after color space conversion.

According to the above technical means, the conversion of color space can more intuitively analyze and process the color changes and distribution in the image, and the valuable image areas can be expressed more intuitively with the help of binary masks. At the same time, it can ensure that the color distribution between different slices is consistent, and there will be no large color differences in different slices due to staining operations, thereby improving the accuracy of subsequent detection.

In one implementation, the method for detecting Crohn's disease and intestinal tuberculosis based on full-field digital slices, wherein the specific prediction result of the target object is determined based on the initial prediction probability matrix of each full-field digital slice. include:

Weight the initial prediction probability matrix of each full-field digital slice to obtain the target prediction probability matrix;

The prediction result of the target object is determined based on the target prediction probability matrix.

According to the above technical means, the weighted method enables the prediction probability of full-field digital slices to be weighted, representing the condition of the target object, reducing the misdiagnosis rate of clinicians, improving the accuracy of treatment, and in the detection of Crohn's disease and intestinal tuberculosis. Pathological characteristics are identified during the process to improve the efficiency of diagnosis.

In one implementation, the method for detecting Crohn's disease and intestinal tuberculosis based on full-field digital slices, wherein the method further includes:

For each full-field digital slice, obtain the similarity degree of each image block in the full-field digital slice relative to other image blocks to obtain the attention score of each image block;

An attention distribution map is formed based on the attention score, wherein the images corresponding to different attention scores in the attention distribution map have different colors;

The attention distribution map is superimposed on the full-field digital slice to obtain an attention heat map of the full-field digital slice.

According to the above technical means, the above method displays areas that are important or different from other image blocks in the slice to help clinicians locate and identify pathological features; through an attention distribution map formed based on attention scores, it enhances clinicians' understanding of specific areas. Attention; attention heatmaps obtained by superimposing attention maps onto full-field digital slices provide clinicians with tools to improve the accuracy of locating and analyzing regions when detecting Crohn's disease and intestinal tuberculosis.

The second aspect of the embodiment of the present application provides a detection device for Crohn's disease and intestinal tuberculosis based on full-field digital slices. The device includes:

The acquisition module is used to acquire a number of full-field digital slices, and determine a number of image blocks corresponding to each full-field digital slice, where the several full-field digital slices are full-field digital slices of the same target object;

A control module used to control the trained detection network model to determine the initial prediction probability matrix corresponding to each full-field digital slice;

A determining module, configured to determine the prediction result of the target object based on the initial prediction probability matrix of each full-field digital slice, wherein the prediction result includes a Crohn's disease category, an intestinal tuberculosis category, or a normal tissue category.

The third aspect of the embodiments of the present application provides a computer-readable storage medium that stores one or more programs, and the one or more programs can be executed by one or more processors to Implement the steps in the method for detecting Crohn's disease and intestinal tuberculosis based on full-field digital sectioning as described in any one of the above.

The fourth aspect of the embodiment of the present application provides a terminal device, which includes: a processor and a memory;

The memory stores a computer-readable program that can be executed by the processor;

When the processor executes the computer-readable program, the steps in any of the above-described methods for detecting Crohn's disease and intestinal tuberculosis based on full-field digital slices are implemented.

Beneficial effects: Compared with the existing technology, this application provides a detection method for Crohn's disease and intestinal tuberculosis based on full-field digital slices. The method includes acquiring several full-field digital slices and determining each full-field digital slice. Several image blocks corresponding to the digital slices, among which several full-field digital slices are full-field digital slices of the same target object; input several image blocks corresponding to each full-field digital slice into the trained detection network model to determine each full-field digital slice The initial prediction probability matrix corresponding to the digital slice; based on the initial prediction probability matrix of each full-field digital slice, determine the prediction result of the target object. This application determines the patient-level prediction results by integrating the initial probabilities of multiple full-field digital slices, which can determine the patient-level results through multi-level classification results, improve the patient-level prediction accuracy, and thereby improve Crohn's disease and intestinal tuberculosis detection.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without incompatible creative efforts.

Figure 1 is a flow chart of the detection method for Crohn's disease and intestinal tuberculosis based on full-field digital sectioning provided by this application.

Figure 2 is a deep learning pipeline diagram of the detection method for Crohn's disease and intestinal tuberculosis based on full-field digital slices provided by this application.

Figure 3 is a full-field digital slice image provided by this application and its corresponding attention heat map.

Figure 4 is a schematic structural diagram of the detection device for Crohn's disease and intestinal tuberculosis based on full-field digital slices provided by this application.

Figure 5 is a schematic structural diagram of the terminal equipment provided by this application.

Detailed ways

This application provides a detection method and related devices for Crohn's disease and intestinal tuberculosis based on full-field digital slices. In order to make the purpose, technical solutions and effects of this application more clear and definite, the following examples of this application are described with reference to the accompanying drawings. To elaborate further. It should be understood that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

Those skilled in the art will understand that, unless expressly stated otherwise, the singular forms "a", "an", "the" and "the" used herein may also include the plural form. It should be further understood that the word "comprising" used in the description of this application refers to the presence of stated features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, Integers, steps, operations, elements, components and/or groups thereof. It will be understood that when we refer to an element being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Additionally, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. As used herein, the term "and/or" includes all or any unit and all combinations of one or more of the associated listed items.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It should also be understood that terms, such as those defined in general dictionaries, are to be understood to have meanings consistent with their meaning in the context of the prior art, and are not to be used in an idealistic or overly descriptive manner unless specifically defined as here. to explain the formal meaning.

It should be understood that the sequence number and size of each step in this embodiment does not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

The inventor discovered through research that Crohn’s disease (CD) is a chronic non-specific granulomatous disease, and its exact cause is still unclear. Over time, the incidence of Crohn's disease in my country has shown an upward trend. Another disease, Intestinal tuberculosis (ITB), is a specific infectious disease caused by Mycobacterium tuberculosis and is also one of the main types of extrapulmonary tuberculosis. Although the two diseases share many similarities in symptoms, endoscopic observations, and pathology, they are prone to misdiagnosis.

The current method of identifying these two diseases through AI is to improve the efficiency and accuracy of diagnosis of Crohn's disease and intestinal tuberculosis by using pathological image diagnosis based on deep learning. However, the current pathological image diagnosis model mainly focuses on the slice level and ignores the patient-level classification that is more closely related to clinical practice, thus affecting the accuracy of identification.

In order to solve the above problem, in the embodiment of the present application, several full-field digital slices are obtained, and several image blocks corresponding to each full-field digital slice are determined, where the several full-field digital slices are full-field digital slices of the same target object; Input several image blocks corresponding to each full-field digital slice into the trained detection network model to determine the initial prediction probability matrix corresponding to each full-field digital slice; based on the initial prediction probability matrix of each full-field digital slice, determine the target The predicted result of the object. This application determines the patient-level prediction results by integrating the initial probabilities of multiple full-field digital slices, which can determine the patient-level results through multi-level classification results, improve the patient-level prediction accuracy, and thereby improve Crohn's disease and intestinal tuberculosis detection.

The content of the application will be further explained below by describing the embodiments in conjunction with the accompanying drawings.

This embodiment provides a detection method for Crohn's disease and intestinal tuberculosis based on full-field digital slices, as shown in Figure 1. The method includes:

S10. Obtain a number of full-field digital slices, and determine a number of image blocks corresponding to each full-field digital slice, where the several full-field digital slices are full-field digital slices of the same target object.

Specifically, full-field digital slices are high-resolution images used for medical image analysis. High-resolution images are usually scanned from biological samples (for example, thin sections prepared from tissues or cells, etc.). Full-field digital slices are often used in medical research fields such as pathological diagnosis, disease research, and drug effect evaluation, and several full-field digital slices are multiple continuous or random slices of the target object. For example, a target subject's Crohn's disease tissue sample will be cut into dozens to hundreds of continuous thin slices and scanned separately to obtain multiple full-field digital slices. Continuous full-field digital slices can provide researchers with continuous observation of the target object, allowing more accurate identification and localization of abnormal areas.

Each full-field digital slice is large in size and is usually divided into several smaller image blocks for easier analysis and processing. For example, a full-field digital slice of 10,000x10,000 pixels can be divided into several image blocks of 224*224 pixels or 384*384 pixels. Each image patch remains the same size and resolution for batch analysis and processing.

In one implementation, determining several image blocks corresponding to each full-field digital slice specifically includes:

S11. Determine the binary mask of the full-field digital slice, and fill the binary mask to obtain the foreground object outline in the full-field digital slice;

S12. Segment the full-field digital slice based on the contour of the foreground object to obtain a foreground image;

S13. Divide the foreground image into several image blocks to obtain several image blocks corresponding to the full-field digital slice.

Specifically, in step S11, the full-field digital slices are derived from microscopic images in the biomedical field, such as histology or cytology, which contain target cells, tissues and pathological areas. In the solution of this application, the full-field digital slices Pathological tissue samples from target subjects. In order to accurately locate and analyze the target pathological area, a binary mask needs to be constructed first to represent the foreground and background parts in the image. A binary mask is an image composed of 0 and 1, where 1 represents the foreground (such as target cells, tissue, and case areas) and 0 represents the background. Binary masks can be obtained through thresholding methods, image segmentation techniques, or deep learning models (such as U-Net, Yolo, etc.). For example, when the brightness of a cell in the image is higher than a preset threshold, that cell location is marked as 1 in the mask, otherwise it is 0, where, in a full-field digital slice, each pixel has an associated The brightness or intensity value usually ranges from 0 to 255 in a grayscale image (8-bit depth). Therefore, the preset threshold ranges from 0 to 255, such as 150.

However, due to the presence of noise or other interference factors in full-field digital slices, the obtained binary mask is imperfect (missing or broken). In order to obtain a continuous foreground object outline, the preliminary binary mask needs to be filled. Filling Use mathematical morphological operations such as erosion and dilation, or utilize image processing techniques such as median filtering, opening and closing operations, etc., to eliminate noise and refine the outline of foreground objects.

In one implementation of this application, when there is a full-field digital slice showing intestinal tuberculosis pathological cells, a preliminary binary mask can be obtained through the U-Net model, and then the closing operation of mathematical morphology is used to fill the binary mask. , thereby obtaining a continuous foreground profile of intestinal tuberculosis pathological cells, which allows researchers or clinicians to accurately analyze the morphology and number of cells based on the continuous foreground profile of intestinal tuberculosis pathological cells, providing important information for further research or diagnosis.

In the implementation of this application, determining the binary mask of the full-field digital slice specifically includes:

S111. Convert the full-field digital slice to TIF (Tag ImageFile Format) format, and convert the converted full-field digital slice from RGB color space to HSV (Hue, Saturation, Value) color space;

S112. Calculate the binary mask of the full-field digital slice based on the saturation channel thresholding of the full-field digital slice after color space conversion.

Specifically, in step S111, after obtaining the full-field digital slice, in order to ensure compatibility with a variety of image processing tools and software, and to improve image storage and transmission efficiency, the full-field digital slice needs to be converted into TIF format. TIF is a widely used image format that supports lossless compression and multi-level storage, and is suitable for medical and scientific research applications that require high quality and high resolution.

Based on this, full-field digital slices are first converted into formats using preset image processing software or custom-written tools. For example, use an open source image processing library (such as OpenCV or image processing toolkit (Python ImageLibrary, PIL)) to perform format conversion. During the conversion process, image compression or quality adjustment is used to optimize storage size and preserve image details. In the implementation of this application, KFBioConverter is used to convert TIF format.

Furthermore, in order to perform more detailed and professional analysis of the color information in the image, the OpenCV tool library is usually used to convert the converted full-field digital slice from the RGB color space to the HSV color space. Among them, RGB color space represents color based on the combination of three channels: red, green and blue, while HSV describes color based on hue (Hue), saturation (Saturation) and brightness (Value). In the HSV color space, hue describes the type of color (such as red, green, blue, etc.), saturation describes the purity of the color, and brightness describes the brightness of the color.

Among them, the advantage of converting from RGB color space to HSV color space is that it can more intuitively analyze and process the color changes and distribution in the image, especially in practical applications, such as cell staining or tissue analysis, where subtle differences in color are related to certain related to biological or pathological characteristics. In addition, each channel in the HSV color space can be processed independently to facilitate more professional color adjustment and analysis.

In one implementation, when analyzing a specific cell color, you are only interested in the color of a specific hue. You can directly use the OpenCV tool library to set a threshold on the hue channel of the HSV color space, which can effectively separate out of the target color area.

In the implementation of this application, the OpenSlide tool is first used to process full-field digital slices. OpenSlide is an open source library that provides support for high-resolution images, especially biomedical slide images. Considering the large size of the full-field digital slices, directly loading the entire image into memory will be very resource-intensive, so we chose to use OpenSlide to downsample the full-field digital slices. This means that what is read in is a scaled-down version of the full-field digital slice, with lower resolution but still retaining most of the key structural and color information. Through downsampling operations, preliminary image processing and analysis can be performed more efficiently. After the downsampled full-field digital slice is read into the memory, the OpenCV tool library is used to convert the full-field digital slice from the RGB color space to the HSV color space. This step is implemented through the cvtColor function in the OpenCV library, which accepts the original RGB image and the conversion type (cv2.COLOR_RGB2HSV in this case) as input parameters, and returns the converted HSV image. In this way, more professional analysis and processing of color changes and distribution in full-field digital slices is achieved, so that full-field digital slices can be accurately processed and analyzed in subsequent steps.

Further, in step S112, after obtaining the full-field digital slice after color space conversion, in order to further process and analyze the specific content and structure in the full-field digital slice, its saturation channel is used for thresholding processing to obtain the corresponding binary mask. Among them, the saturation channel describes the purity of the color in the HSV color space. Therefore, the saturation channel is usually used to separate areas with higher or lower saturation in the image.

Specifically, an appropriate threshold is selected based on the content and application requirements in the full-field digital slice. For example, extract areas with higher saturation from a full-field digital slice, such as a specific stain or marker, and set a preset threshold such as 150 (on a scale of 0-255) so that only saturation is higher than the preset threshold pixels are recognized. On the other hand, if the purpose is to identify less saturated areas, such as background or unstained parts, the preset threshold can be set lower, such as 50. Based on experimental data or prior knowledge, preset thresholds can be fine-tuned to ensure effectiveness and accuracy for specific applications. In addition, for best results, the best preset threshold can be automatically determined in combination with histogram analysis or Otsu's method.

By applying a thresholding operation, pixels in the saturation channel of a full-field digital slice are converted to binary values, either "0" or "1". Among them, pixels higher than the preset threshold will be marked as "1" (or white), while pixels lower than or equal to the preset threshold will be marked as "0" (or black). In turn, a binary mask is obtained in which white areas represent the more saturated parts of interest and black areas represent other parts.

In addition, in order to enhance the accuracy of the mask and remove potential noise, image processing techniques are used to post-process the binary mask, such as using morphological operations such as opening or closing operations to remove small noise points or fill small holes.

Specifically, in step S12, based on the foreground object outline obtained in step S11, the entire full-field digital slice can be accurately segmented, thereby extracting the image area related to the foreground object, which is called the foreground image. Among them, the full field of view digital slice contains a variety of elements, such as background, noise and foreground objects of interest (such as target cells, tissues or specific pathological areas). In order to focus only on the foreground objects, the obtained foreground object contours are used to segment the original full field of view. Digital slicing. Segmentation can be accomplished by a simple dot-multiplication operation, i.e. dot-multiply the foreground object outline (consisting of 1 and 0) with the original full-field digital slice, thereby retaining only the foreground area and setting the background area to 0 or other specified background value .

In practical applications, in order to further improve the clarity and readability of the foreground image, post-processing techniques, such as smoothing, sharpening, etc., are used to enhance the details of the foreground image. In addition, the foreground image can be further adjusted in color or grayscale, to meet specific application or analytical requirements.

In the implementation of this application, full-field digital slices are derived from related research on "Discrimination of Crohn's and Intestinal Tuberculosis" and are more concerned with the morphology and structure of disease characteristics. After obtaining the foreground object contours, the contours are applied to the original full-field digital slices, resulting in Crohn's and intestinal tuberculosis-related foreground images. In order to better observe the structure and characteristics of the disease, image enhancement operations are performed on the foreground image to make certain details of the relevant areas more obvious.

Specifically, in step S13, based on the foreground image obtained in step S12, in order to further refine the analysis or perform specific foreground image processing operations, the foreground image is divided into a plurality of small image blocks. Partitioning operations allow for more detailed study of local features, structures, or textures in foreground images, or for processing individual image patches in parallel to increase computational efficiency. Specifically, the foreground image can be equally divided or divided into multiple image blocks according to a predetermined scale and step size. The image blocks can be square, rectangular or other specified shapes. The size and number of the image blocks are based on actual needs and application scenarios. to make sure.

In one implementation, the resolution of the foreground image is very high, and in order to avoid a large amount of calculations, it can be divided into larger image blocks; conversely, if the microscopic features in the foreground image need to be studied in detail, the foreground image can be divided into For smaller image blocks, in the implementation of this application, OpenCV is used to divide the tissue into several image blocks of 384x384 pixels at 40X resolution.

Secondly, in one implementation, the sliding window method is used to divide the foreground image, and partial overlap between image blocks can be obtained, thereby obtaining better continuity and consistency in subsequent processing. The overlapping sliding window method It is useful when edge information is needed or when the boundary of an image patch intersects the target structure of interest. It can ensure that the target structure is completely included in a certain image patch instead of being cut off.

However, in storage or parallel processing scenarios, it is more desirable to generate non-overlapping image blocks. In this case, the foreground image can be evenly divided directly according to fixed image block sizes and intervals. For example, if the selected image block size is 64x64 pixels, then divide it every 64 pixels to ensure that each image block is independent and does not have any overlapping parts to avoid any problems that will not occur during the division of non-overlapping image blocks. Information loss or error, ensure that the selected image patch size matches the size of the target structure or feature in the foreground image.

In addition, if the size of the foreground image is not an integral multiple of the image block size, the image can be padded or cropped appropriately to ensure complete division. In addition, in order to ensure that each image patch has enough information or feature points, a preset evaluation mechanism is used, such as the number of feature points within the image patch, contrast or texture complexity, etc., to decide whether to accept or reject an image patch. . For example, when the full-field digital slice is the foreground image of Crohn's pathology tissue, it is divided into image blocks of size 256x256, and for each image block, specific image analysis (such as cell counting, tissue structure analysis, or Color statistics), after all image patches have been processed, the image patches are reassembled to obtain a complete, processed full-field digital slice.

In the implementation of this application, the OpenCV library is used to complete the preprocessing operation of full-field digital slices. First, full-field digital slices were read using OpenSlide. Considering that full-field digital slices contain a variety of elements, such as background, noise, and foreground objects of real interest (pathological areas of Crohn's disease and intestinal tuberculosis), the full-field digital slices need to be segmented first to ensure that only the Foreground object.

In order to remove irrelevant blank areas and improve processing efficiency, OpenCV was used to segment tissue contours on full-field digital slices at low resolution. After obtaining the foreground object outline, directly perform a dot multiplication operation on the foreground object outline and the original full-field digital slice. The dot product operation can be implemented simply by dot multiplying the foreground object outline (a binary image consisting of 1s and 0s) with the original full-field digital slice. Furthermore, the image area associated with the foreground object is retained, while other areas are set to 0 or other specified background values.

After completing the foreground contour segmentation, for further refinement and analysis, the foreground object is divided into a number of non-overlapping image patches (patches) of 384×384 size at 40X resolution. Then, the local features, structures or textures within each image patch are studied in more detail. Moreover, processing small image patches is more computationally efficient than processing an entire large image.

Each image patch is saved with its upper left corner pixel coordinates recorded in the original full-field digital slice. In addition, in subsequent processing and analysis, if it is necessary to reconstruct the entire foreground object or consider the relationship between image blocks, the pixel coordinate information will play a positioning role.

In addition, each image patch can undergo further image enhancement and processing depending on the needs of the analysis. For example, for a specific cell staining technology, you are more concerned about the cell morphology and structure after staining. In this case, you can apply image enhancement algorithms, such as sharpening, contrast enhancement, etc., to make the details inside the cells more prominent.

After all image tiles have been processed, the image tiles can be recombined to generate a processed full-field digital slice, providing richer information for subsequent analysis.

S20. For each full-field digital slice, input several image blocks corresponding to the full-field digital slice into a trained detection network model, and determine the initial prediction probability matrix corresponding to each full-field digital slice through the detection network model.

Specifically, after obtaining a full-field digital slice and dividing it into several image blocks, in order to analyze and identify specific structures or contents in the image blocks, a trained detection network model is used to process the image blocks. Among them, the detection network model is usually a deep learning model, which is often used for the detection and recognition of targets in images.

Second, each image patch is individually input into the trained detection network model. This model usually consists of multiple convolutional layers, pooling layers, and fully connected layers, aiming to extract meaningful features from input image blocks and detect related targets. In this way, the model can generate a predicted probability matrix for each image patch, representing the probability that each pixel in the image patch belongs to a certain target class.

For example, if the actual task is to identify a specific pathological tissue, then each value in the prediction probability matrix will represent the probability of whether the corresponding pixel belongs to this pathological tissue. A high probability value indicates that the pixel belongs to the target pathological tissue.

For each full-field digital slice, its corresponding image blocks will be input into the detection network model in turn, and the prediction probability matrix of each image block will be obtained. Among them, the prediction probability matrices can be combined or merged to obtain the initial prediction probability matrix of the entire full-field digital slice.

In addition, detection network models usually require a large amount of annotated data for training. Annotation data is annotated image blocks, and the target structure or content of the image block has been accurately marked. Through a large amount of training data, the model can learn how to accurately identify and detect targets, thereby achieving high accuracy and robustness in practical applications.

In one implementation, as shown in Figure 2, the detection network model includes a feature extraction module and a classification module.

Specifically, the main task of the feature extraction module is to extract meaningful features from the input image patches, thereby capturing important information in the image, such as edges, texture, shape, and color. To achieve this purpose, the feature extraction module usually adopts a structure of multiple convolutional layers, activation layers and pooling layers. For example, the convolutional layer slides the image through different convolution kernels to capture various spatial features; the pooling layer reduces the spatial dimension of the features while retaining the most important information.

In actual implementation, the feature extraction module can use a pre-trained network structure, such as VGG16, ResNet50 or MobileNet, etc. The pre-trained model is pre-trained on a large amount of image data and can effectively capture the basic features of the image, thereby accelerating and improving the learning efficiency of new tasks.

In the implementation of this application, the feature extraction module uses a Vision Transformer (ViT) model pre-trained based on the ImageNet data set to ensure that each image block can obtain high-quality, low-dimensional feature representation. In order to further optimize and enhance the performance and generalization ability of the model, each image patch undergoes a series of data enhancement operations before being fed into the ViT model. First, each image patch is read based on the saved coordinate information, and then one or more of the following data augmentation methods are randomly applied:

1. Perform random rotation, such as a 90° rotation angle, or perform a horizontal flip.

2. Realize scaling and rotation of image blocks through affine transformation. Specifically, the scaling range is between 80% and 120% of the original image, while the rotation angle ranges from -30° to +30°.

3. Introduce Gaussian noise or apply Gaussian blur to the image block to make it more robust.

4. Adjust the brightness and contrast of image blocks to further increase the fault tolerance of the model.

5. Adjust the hue and saturation of the image blocks to increase the model's adaptability to various real environmental conditions.

After the data augmentation step, each image patch is fed into the ViT model for processing and converted into a feature vector with 768 dimensions. In this way, each full-field digital slice can generate thousands of such feature vectors, providing a rich information basis for subsequent image analysis.

Specifically, after the feature extraction module extracts meaningful features from the image, the task of the classification module is to classify and predict image blocks based on the meaningful features. Usually, the process of classification and prediction involves several fully connected neural network layers. The feature vector is converted into the final prediction result through the fully connected neural network layer. In the implementation of this application, it is finally converted into a probability matrix.

To prevent overfitting and improve the generalization ability of the model, regularization techniques (such as random dropout or L2 regularization) are included in the classification module.

In addition, the classification module also includes activation functions (such as linear rectification functions or flexible maximum functions), which are used to increase the nonlinear capabilities of the model and convert the output into probability values.

To sum up, when the image blocks of full-field digital slices are input to the detection network model, the image blocks first pass through the feature extraction module to obtain a set of feature vectors. Secondly, the feature vector is passed to the classification module to obtain the final probability matrix. This method of combining feature extraction and classification not only improves the accuracy of the model, but also ensures that the model has good robustness when facing different types of image data.

In another implementation, each of the plurality of full-field digital slices is input into a trained detection network model, and the initial prediction probability matrix corresponding to each full-field digital slice is determined through the detection network model. Specifically include:

S221. For each full-field digital slice, determine several image blocks corresponding to the full-field digital slice;

S222. Input several image blocks into the feature extraction module respectively, and determine the corresponding feature vectors of each image block through the feature extraction module;

S223. Input the corresponding feature vectors of each image block into the classification module, and determine the initial prediction probability matrix corresponding to the full-view digital slice through the classification module.

Specifically, after obtaining the feature vectors corresponding to each image block, the feature vectors are further processed using a classification module to determine the initial prediction probability matrix corresponding to the full-view digital slice.

In actual implementation, the above classification module is usually designed based on a deep neural network (for example, using multiple layers of fully connected neural network layers) to ensure that useful and decisive information for classification is extracted from the feature vector. At the end of the classification module, a flexible maximum activation function is used so that the module can output a prediction probability matrix for each image block.

The prediction probability matrix reflects the model's confidence in the category to which each image patch belongs. Specifically, when the actual task is a binary classification task, each element of the prediction probability matrix represents the probability that the corresponding image block is judged to be a preset category. For multi-classification tasks, each row of the prediction probability matrix shows the model's prediction probability that the image patch belongs to all categories.

In order to prevent overfitting and improve the generalization ability of the model, some regularization techniques such as random deactivation are also included in the classification module.

In addition, in order to improve the training efficiency and convergence speed of the model, a batch normalization (Batch Normalization) layer can also be added to the module.

After the classification module is processed, all image blocks will obtain a corresponding prediction probability matrix. Combining the above probability matrices together forms the initial prediction probability matrix corresponding to the full-field digital slice, which provides key data for subsequent analysis and decision-making.

In one implementation, the classification module includes an attention unit and a multi-layer perceptron.

Specifically, after obtaining the feature vector, the feature vector is further passed to the classification module to ensure that accurate classification judgments can be made based on the feature vector, and an initial prediction probability matrix corresponding to the full-field digital slice is generated.

In the implementation of this application, the classification module first uses one or more attention units to process the input feature vector. The attention mechanism has proven to be a powerful method in the field of deep learning, which can effectively provide the model with the ability to distinguish different parts of features, thereby ensuring that the model pays more attention to the feature parts that are more critical for classification. The attention unit can be implemented through self-attention or other variants (such as sparse attention Sparse-Attention). The self-attention mechanism can assign a weight to each input feature, and the weight is calculated based on the relationship between the input features. Therefore, higher weights are given to features that are potentially highly relevant to the current task.

After being processed by the attention unit, each part of the feature vector has been weighted, thereby providing a richer and more discriminative feature representation for subsequent classification.

Subsequently, the processed features are passed to the multilayer perceptron for further processing. Multi-layer perceptron is composed of multiple fully connected neural network layers, which can learn non-linear representation of input features and ensure that accurate classification results can be generated based on non-linear representation.

At the end of the multi-layer perceptron, a flexible maximum activation function is usually used to output the predicted probability, ensuring that each input image block can obtain a clear, probabilistic classification result.

In addition, in order to increase the robustness of the model and prevent overfitting, the classification module also adds random deactivation and other regularization methods between or within the attention unit and the multi-layer perceptron.

In one implementation, inputting the corresponding feature vectors of each image block into the classification module, and determining the initial prediction probability matrix corresponding to the full-field digital slice through the classification module specifically includes:

S231. Input the corresponding feature vectors of each image block into the attention unit, determine the attention coefficient of each image block through the attention unit, and form an attention feature matrix based on the attention coefficient and the corresponding feature vector of each image block. ;

S232. Input the attention feature matrix into the multi-layer perceptron, and determine the initial prediction probability matrix corresponding to the full-view digital slice through the multi-layer perceptron.

Specifically, in step S231, after obtaining the feature vector, and in order to capture and strengthen the correlation and importance between specific areas, the feature vector is passed to the attention unit to determine the attention coefficient of each image block. The attention coefficient provides a mechanism for the model to focus on those feature areas that contain key information.

In the implementation of this application, each full-field digital slice is tiled into thousands of image blocks, and a visual transformer network model is used to extract features from the image blocks. This means that for each slice of thousands of image patches, corresponding feature vectors are extracted, which represent the content and structural properties of the respective image patch.

In order to calculate the attention score, the feature vector is first linearly transformed to obtain three new sets of vectors: query, key and value. For each query vector, a weight coefficient is obtained by performing a dot product with all key vectors and then passing it through the flexible maximum function.

The weight coefficient obtained above is the attention score, which represents the correlation strength between the current image block and all other image blocks. The attention score is multiplied by the corresponding value vector to obtain a weighted feature representation, also known as the attention output. In this way, for each image patch, a feature vector weighted based on the contextual information of all other image patches can be obtained.

Using the weighted feature vector, a new, weighted feature matrix is formed, which is called the "attention feature matrix". The attention feature matrix provides a richer and more discriminative feature representation, allowing the model to pay more attention to those areas that have a decisive impact on classification decisions in subsequent processing.

Further, in step S232, after obtaining the attention feature matrix, in order to further determine the prediction probability of the full-field digital slice, the attention feature matrix is input into the multi-layer perceptron.

Among them, the multi-layer perceptron mainly consists of an input layer, several hidden layers and an output layer. Each layer contains multiple neurons, and each neuron is connected to all neurons in the previous layer.

The input layer receives feature vectors generated by the attention mechanism. Specifically, if there is a d-dimensional feature vector, then the input layer will have d neurons.

Hidden layers are composed of multiple neurons and are used to extract and learn complex patterns in the input data. The calculation formula of neurons in the hidden layer is:

h _i =f(W _i ·h _i-1 +b _i )

Among them, h _i is the output of the i-th hidden layer, hi-1 is the output of the i-1 hidden layer or input layer, Wi and _{b i are} the weight and bias of the i-th hidden layer respectively, and f is a nonlinear activation function, such as a linear rectifier function, a logistic function, or a hyperbolic tangent function.

The output layer produces the model’s predictions. In classification tasks, the number of neurons in the output layer is usually the same as the number of categories. The calculation formula of the output layer can be expressed as:

o=softmax(W _i ·h _last +b _i )

Among them, o is the output of the output layer, that is, the predicted probability of each category; h _last is the output of the last hidden layer; Wi _{and bi} are the weight and bias of the output layer respectively; and the Softmax function is to convert the original output of the model Function converted to probability.

In the implementation of this application, each feature vector in the attention feature matrix is weighted according to the attention score calculated by the attention mechanism in step S231. This step can be understood as multiplying the feature vector of each image patch by its importance in the entire slice, thereby obtaining a feature matrix that combines spatial relationship and content information.

Secondly, the attention score of the image patch obtained by the multi-layer perceptron is multiplied by the corresponding feature matrix to obtain the feature matrix of the slice based on the weight change of the image patch.

Then, the obtained weighted feature matrix is fed into the multilayer perceptron. The multi-layer perception opportunity further transforms the feature matrix based on the image block weight changes through the hidden layer, and finally gives the predicted probability of a single full-field digital slice in the output layer.

After obtaining the predicted probability of each full-field digital slice, in order to obtain the prediction result at the overall patient level, the predicted probabilities of all slices are averaged. By comprehensively considering the information from each slice, a more comprehensive and robust prediction is obtained.

S30. Based on the initial prediction probability matrix of each full-field digital slice, determine the prediction result of the target object, where the prediction result includes a Crohn's disease category, an intestinal tuberculosis category, or a normal tissue category.

Specifically, after obtaining the initial prediction probability matrix of each full-field digital slice, the initial prediction probability matrix is further analyzed to determine the specific prediction result of the target object.

In the specific implementation of this application, each initial prediction probability matrix provides probability distributions for three main categories: Crohn's disease, intestinal tuberculosis, and normal tissue. For each full-field digital slice, the probabilities of the three categories are compared, and the category with the highest probability is determined as the prediction for that slice.

To ensure the accuracy and robustness of predictions, a voting mechanism or ensemble strategy is adopted. For example, if a full-field digital slice is divided into multiple image blocks, and the prediction results of most of the image blocks point to "Crohn's disease", then the final prediction result of the full-field digital slice is determined to be "Crohn's disease" ".

In addition, in order to further improve the accuracy of classification, a preset probability threshold mechanism can also be added. For example, if the predicted probability of Crohn's disease exceeds a preset threshold, such as 90%, this category will be identified with greater confidence. On the other hand, if the probabilities are low for all categories, the slice is marked as "to be reviewed" or "uncertain" and an expert is recommended for further review.

In order to avoid false positive and false negative predictions, additional post-processing steps can be introduced, such as spatial continuity analysis or structural analysis, to ensure that the prediction results are consistent and consistent both globally and locally.

After obtaining the prediction results of each full-field digital slice, a visual interface is further provided for doctors or related personnel to display the prediction results, associated probability values, and abnormal areas to assist them in making final diagnostic judgments.

In one implementation, determining the prediction result of the target object based on the initial prediction probability matrix of each full-field digital slice specifically includes:

S31. Weight the initial prediction probability matrix of each full-field digital slice to obtain the target prediction probability matrix;

S32. Determine the prediction result of the target object based on the target prediction probability matrix.

Specifically, in step S31, after obtaining the initial prediction probability matrix of each full-field digital slice, in order to further optimize and improve the prediction performance of the model, a weighting strategy can be adopted to obtain a more accurate target prediction probability matrix.

For example, weighting strategies are based on several considerations:

Data quality weighting: Not all full-field digital slices are of equal quality. Some full-field digital slices have lower image quality due to differences in sampling, equipment, or other factors. Therefore, each full-field digital slice can be assigned a preset weight based on its quality (for example, slices with high image clarity receive a higher weight of 0.8, with a weight range of 0-1).

Contextual information weighting: If certain full-field digital slices are similar in content or features to their surrounding slices, their prediction results are considered more reliable. Therefore, one might consider assigning a higher weight to a specific full-field slice.

Model confidence weight: For each prediction, the model itself will have an internal confidence score. Predictions for which the model is very confident are given higher weights, while predictions for which the model is less certain are given lower weights.

Combined with the above weights, the initial prediction probability matrix of each full-field digital slice is weighted. Specifically, the above weight can be used as a weighting factor and multiplied by the initial prediction probability matrix to obtain a weighted prediction probability matrix.

Further, in step S32, after obtaining the target prediction probability matrix, the system further performs processing to determine the final target object prediction result.

In a practical implementation, the target prediction probability matrix provides a probability value for each image block or region, indicating the likelihood that the target object is contained in the block or region. This approach provides a granular, per-area-level probability map, helping to locate and identify targets more precisely.

In order to obtain clear prediction results from the probability matrix, a preset threshold is usually set. When the probability of a region exceeds this threshold, the region is judged to contain the target object. On the contrary, if the probability is lower than the threshold, the region is judged as not containing the target object. Choosing an appropriate threshold is key and is usually determined based on cross-validation techniques to ensure the accuracy and robustness of predictions.

Specifically, the choice of threshold will be affected by the application scenario and data set characteristics. For example, in medical image analysis tasks, considering patient safety and the importance of avoiding missed diagnoses, there is a tendency to choose a lower threshold to ensure that all diseased areas can be detected, even if this will bring higher false positive rate. On the contrary, in other cases, such as when performing object detection, a higher threshold is chosen in order to avoid a large number of false positives and improve prediction accuracy.

In order to determine the optimal threshold, a common method is to draw the ROC (Receiver Operating Characteristic) curve, and then select a point that makes the trade-off between the true positive rate and the false positive rate optimal as the threshold.

Another approach is to use the F1 score, which is the harmonic mean of precision and recall, and choose a threshold that maximizes the F1 score. For example, in a deep learning model for intestinal nodule detection, the trained model gives each prediction area a probability value between 0 and 1, indicating the probability that the area contains a nodule. To determine the optimal threshold, use a cross-validation dataset and calculate the F1 score for different thresholds. Assume that four thresholds of 0.3, 0.5, 0.7 and 0.9 are tried and it is found that the F1 score is the highest when the threshold is 0.5, then 0.5 can be selected as the threshold to determine the presence or absence of nodules.

In the implementation of this application, the training set part of the Crohn's disease and intestinal tuberculosis data sets is first input into the previously described model to finely adjust the parameters of the model, thereby obtaining a model that can better identify Crohn's disease. Deep learning models of encephalitis, intestinal tuberculosis and normal tissues. At the same time, each patient has a clinically given disease diagnosis as the patient's label.

Considering that there are two levels of classification tasks, the data used not only utilize the diagnosis results of each patient as patient-level labels, but also perform clinical evaluation on each full-field digital slice to obtain full-field digital slice-level labels.

Corresponding feature vectors are extracted from thousands of image patches for each full-field digital slice. In order to obtain slice-level predictions, the weight of each image block relative to other image blocks is first obtained through the attention coefficient, thereby obtaining the final feature matrix. The feature matrix is processed by a multi-layer perceptron to obtain the predicted probability of a single full-field digital slice. Next, predictions from multiple full-field digital slices are averaged to obtain patient-level predictions.

To ensure the performance and generalization ability of the model, five-fold cross-validation was used to evaluate the overall prediction performance in the training cohort based on the probability of each sample. Subsequently, the model was further validated in an independent testing cohort.

During the training process of the model, this application uses the Adam (Adaptive Moment Estimation optimizer) optimizer to update the weight of the network, and adopts parameter settings with an initial learning rate of 0.0001, a weight attenuation of 0.0001, and a number of iterations of 100. Considering the data imbalance problem existing in the dataset, in order to ensure that the model can pay equal attention to each category, lower frequency categories are deliberately assigned higher weights, and LDAM loss is used to calculate the loss. This method ensures that the model can obtain stable and accurate prediction results even in the face of imbalanced data.

In the embodiment of the present application, the detection method of Crohn's disease and intestinal tuberculosis based on full-field digital slices also includes:

S41. For each full-field digital slice, obtain the similarity degree of each image block in the full-field digital slice relative to other image blocks to obtain the attention score of each image block;

S42. Form an attention distribution map based on the attention score, wherein the images corresponding to different attention scores in the attention distribution map have different colors;

S43. Superimpose the attention distribution map onto the full-field digital slice to obtain an attention heat map of the full-field digital slice.

Specifically, in step S41, when processing the full-field digital slice, in order to ensure that the model can fully understand the relative importance of each area within the full-field digital slice for the overall analysis, the similarity of each image block to other image blocks is key. . This similarity measures how the internal details of an image patch relate to the internal details of other image patches, thus providing an attention score for each image patch. The attention score represents the degree of attention the model pays to this image patch, helping to analyze pathological information more accurately.

In order to explain the relative importance of the model to different regions at the slice level, a specific quantification method is adopted in the implementation of this application. First, the full-field digital slices are cropped into multiple 16x16 pixel image blocks. This specific size selection is based on the size and distribution of pathological cells to ensure that each image block can capture sufficient pathological information. Each image block contains information about pathological cells. In addition, this cropping method is performed in the area of pathological cells and is therefore fair and free of any bias or bias.

Second, for each image patch, its degree of similarity is quantified by calculating the similarity of its feature vectors with other image patches. Specifically, the similarity is calculated by calculating the cosine similarity score between feature vectors. Based on the cosine similarity score similarity score, an attention score is extracted for each image patch, which represents the importance of this image patch relative to other image patches.

Specifically, extracting attention scores for each image patch based on feature vector similarity scores between image patches is a method to enhance key information and reduce the impact of unimportant information on model analysis. Usually this process can be achieved through the following steps:

S411. Calculate the similarity score: For a given image block A, calculate the similarity score through the feature vectors of all other image blocks. A commonly used similarity measure is cosine similarity, which measures the cosine angle between two vectors. The formula is:

Among them, A and B are the feature vectors of the two image blocks respectively.

S412. Normalized similarity scores: In order to ensure that the scores are within a reasonable range, all similarity scores are normalized. This can be achieved via a soft maxima function:

Among them, the attention score (Attention-Score) represents the importance of image block A relative to other image blocks.

S413. Weighted average: Obtain the normalized similarity score, and calculate a weighted average feature vector for each image block, where the weight is the normalized similarity score between the selected image block and other image blocks. The specific formula is:

Weighted-Feature-Vector(A)＝∑Attention-Score(B)×Feature-Vector(B)

S414. Attention score extraction: The weighted average feature vector captures the importance of image block A relative to other image blocks. This difference is further strengthened by comparison with the original feature vector to extract a more accurate attention score.

Specifically, in step S42, when training and inferring the deep learning model, the attention mechanism plays a very important role in explaining the decision-making process of the model. The attention score clearly shows which parts of the input data the model pays more attention to, that is, which parts the model considers more important for decision-making.

In the implementation of this application, as shown in Figure 3, an attention distribution map is formed based on the attention score. The attention scores are converted into RGB colors using a divergent color map and displayed at their respective spatial locations, allowing the observer to visually identify different regions. Specifically, different attention scores in the attention distribution map correspond to different image colors, which intuitively shows which parts of the input image the model pays higher attention to.

In the implementation of this application, to further determine the specific nature of the areas of high concern, they were evaluated by a gastrointestinal pathologist and classified into 10 groups, including giant cells/large granulomas/caseating necrosis , mucosa/gland, inflammatory infiltrate, loose or dense fibrous connective tissue, thickened muscle layer, submucosal/subserosa adipose tissue, hyperplastic myenteric plexus, blood vessels/red blood cells, extracellular mucus and lymph fluid. Detailed evaluation of the above areas provides a more in-depth diagnostic analysis for each full-field digital slice.

Compared with traditional methods, the above-mentioned interpretable deep learning method based on full-field digital slices provides more refined results. Second, attention-based learning is used to automatically identify subregions of high diagnostic value and accurately classify full-field digital slices without relying on multivariate analysis or colonoscopy images as in previous studies in this application.

It is worth noting that the above-mentioned interpretable deep learning method based on full-field digital slices does not rely on the information or annotations of any specific area in the full-field digital slices, because the entire diagnosis is performed based on the entire full-field digital slices. This also demonstrates that the model can provide refined heatmaps for interpretation without using class activation maps or pixel-level annotations. Although not specifically trained on normal tissue during the training process, the method is still able to effectively identify the regions most relevant to predictions and show clinicians how each tissue region of each full-field digital slice contributes to the model predictions. Relative contribution and importance.

Specifically, in step S43, after determining the initial prediction probability matrix, the distribution map generated by the attention mechanism can be further combined with the full-field digital slice to intuitively display the area that the model focuses on when classifying, thereby generating attention heat. picture.

First, obtain the attention distribution map generated by the above attention unit. The attention distribution map is a matrix with the same size as the input feature vector. The value of the matrix represents the model's attention to each area. The value of the matrix ranges from 0 to 1, with higher values indicating that the model pays more attention to this part when making decisions.

To map values to the colors of the visualization, a color coding strategy is employed. A common strategy is to use a color gradient from cool colors (e.g., blue, representing low attention) to warm colors (e.g., red, representing high attention) to represent the intensity of attention.

Next, the color values are overlaid with the original full-field digital slice, using a transparency blending technique that takes both the pixels of the original slice and the corresponding color values into account, creating an overlay effect visually.

The result of the overlay is an "attention heat map," in which shades of color and temperature changes visually demonstrate the areas the model focuses on when classifying. For example, if a certain area of a full-field digital slice appears bright red in the heat map, it means that the model pays special attention to this part and this part plays a key role in the model's decision-making.

In addition, attention heat maps not only provide researchers and doctors with the opportunity to gain insight into the model's decision-making process, but also provide doctors with intuitive clues about the location and nature of disease in medical image diagnosis.

This embodiment uses a detection network model trained based on full-field digital slices of Crohn's disease and full-field digital slices of intestinal tuberculosis to identify the disease type of the target object, and can quickly identify the pathological images of the target patient.

To sum up, this embodiment provides a method for detecting Crohn's disease and intestinal tuberculosis based on full-field digital slices. The method includes acquiring several full-field digital slices and determining the corresponding value of each full-field digital slice. Several image blocks, wherein several full-field digital slices are full-field digital slices of the same target object; for each full-field digital slice, input several image blocks corresponding to the full-field digital slice into a trained detection network model, and pass The detection network model determines the initial prediction probability matrix corresponding to each full-field digital slice; based on the initial prediction probability matrix of each full-field digital slice, determines the prediction result of the target object, wherein the prediction result includes Crohn's Disease category, intestinal tuberculosis category or normal tissue category. According to the above technical means, after obtaining the patient's full field of view digital slices and further determining several image blocks corresponding to each slice, this application uses a trained detection network model to process the image blocks to determine the initial prediction probability of each slice, and then The predicted probability of full-field digital slices is determined based on the initial predicted probability. In this way, by integrating slice-level and patient-level classification results, more accurate multi-level classification can be achieved, effectively reducing misdiagnosis and further improving the accuracy of treatment.

Based on the above-mentioned detection method of Crohn's disease and intestinal tuberculosis based on full-field digital slices, this embodiment provides a detection device for Crohn's disease and intestinal tuberculosis based on full-field digital slices, as shown in Figure 4 , the device includes:

The acquisition module 100 is used to acquire a number of full-field digital slices, and determine a number of image blocks corresponding to each full-field digital slice, where the several full-field digital slices are full-field digital slices of the same target object;

The control module 200 is used to control the trained detection network model to determine the initial prediction probability matrix corresponding to each full-field digital slice;

The determination module 300 is configured to determine the prediction result of the target object based on the initial prediction probability matrix of each full-field digital slice, where the prediction result includes a Crohn's disease category, an intestinal tuberculosis category, or a normal tissue category.

Based on the above detection method of Crohn's disease and intestinal tuberculosis based on full-field digital slices, this embodiment provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one Or multiple programs may be executed by one or more processors to implement the steps in the method for detecting Crohn's disease and intestinal tuberculosis based on full-field digital slices as described in the above embodiment.

Based on the above detection method of Crohn's disease and intestinal tuberculosis based on full-field digital slices, this application also provides a terminal device, as shown in Figure 5, which includes at least one processor (processor) 20; display screen 21; As well as a memory (memory) 22, it may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can complete communication with each other through the bus 24. The display screen 21 is configured to display a user guidance interface preset in the initial setting mode. Communication interface 23 can transmit information. The processor 20 can call logical instructions in the memory 22 to execute the methods in the above embodiments.

In addition, the above-mentioned logical instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

As a computer-readable storage medium, the memory 22 can be configured to store software programs, computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes software programs, instructions or modules stored in the memory 22 to execute functional applications and data processing, that is, to implement the methods in the above embodiments.

The memory 22 may include a program storage area and a data storage area, where the program storage area may store an operating system and at least one application program required for a function; the storage data area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory. For example, there are many media that can store program code, such as U disk, mobile hard disk, read-only memory (ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, or they can also be temporary state storage media.

In addition, the specific process of loading and executing the multiple instruction processors in the above storage medium and terminal device has been described in detail in the above method, and will not be described one by one here.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in the embodiments of the present application.

Claims (10)

1. A method for detecting crohn's disease and intestinal tuberculosis based on full-field digital slicing, the method comprising:

acquiring a plurality of full-view digital slices, and determining a plurality of image blocks corresponding to each full-view digital slice, wherein the plurality of full-view digital slices are full-view digital slices of the same target object;

for each full-view digital slice, inputting a plurality of image blocks corresponding to the full-view digital slice into a trained detection network model, and determining an initial prediction probability matrix corresponding to each full-view digital slice through the detection network model;

a prediction result of the target object is determined based on an initial prediction probability matrix for each full-field digital slice, wherein the prediction result comprises a crohn's disease category, an intestinal tuberculosis category, or a normal tissue category.

2. The method for detecting crohn's disease and intestinal tuberculosis based on full-field digital slicing according to claim 1, wherein the detecting network model includes a feature extraction module and a classification module, the inputting each full-field digital slicing of the plurality of full-field digital slicing into the trained detecting network model, and determining the initial prediction probability matrix corresponding to each full-field digital slicing by the detecting network model specifically includes:

For each full-view digital slice, determining a plurality of image blocks corresponding to the full-view digital slice;

respectively inputting a plurality of image blocks into the feature extraction module, and determining the feature vectors corresponding to the image blocks through the feature extraction module;

and inputting the feature vectors corresponding to the image blocks into the classification module, and determining an initial prediction probability matrix corresponding to the full-view digital slice through the classification module.

3. The method for detecting crohn's disease and intestinal tuberculosis based on full-field digital slicing according to claim 2, wherein the classification module includes an attention unit and a multi-layer perceptron; the step of inputting the feature vectors corresponding to the image blocks into the classification module, and the step of determining the initial prediction probability matrix corresponding to the full-view digital slice through the classification module specifically comprises the following steps:

inputting the feature vectors corresponding to the image blocks into an attention unit, determining the attention coefficient of each image block through the attention unit, and forming an attention feature matrix based on the attention coefficient and the feature vectors corresponding to the image blocks;

and inputting the attention characteristic matrix into the multi-layer perceptron, and determining an initial prediction probability matrix corresponding to the full-view digital slice through the multi-layer perceptron.

4. The method for detecting crohn's disease and intestinal tuberculosis based on full-field digital slices according to claim 1, wherein the determining a plurality of image blocks corresponding to each full-field digital slice specifically includes:

determining a binary mask of the full-view digital slice, and filling the binary mask to obtain a foreground object contour in the full-view digital slice;

dividing the full-view digital slice based on the foreground object contour to obtain a foreground image;

and dividing the foreground image into a plurality of image blocks to obtain a plurality of image blocks corresponding to the full-view digital slice.

5. The method for detecting crohn's disease and intestinal tuberculosis based on the full-field digital slice according to claim 4, characterized in that the determining the binary mask of the full-field digital slice specifically comprises:

converting the full-view digital slice into a TIF format, and converting the converted full-view digital slice from an RGB color space to an HSV color space;

and (3) calculating the binary mask of the full-field digital slice according to the saturation channel thresholding of the full-field digital slice after the color space conversion.

6. The method for detecting crohn's disease and intestinal tuberculosis based on full-field digital slices according to claim 1, wherein the determining the prediction result of the target object based on the initial prediction probability matrix of each full-field digital slice specifically includes:

Weighting the initial prediction probability matrix of each full-view digital slice to obtain a target prediction probability matrix;

and determining a prediction result of the target object based on the target prediction probability matrix.

7. The method for detecting crohn's disease and intestinal tuberculosis based on full-field digital slicing according to claim 1, further comprising:

for each full-view digital slice, obtaining the similarity degree of each image block in the full-view digital slice relative to other image blocks to obtain the attention score of each image block;

forming an attention distribution map based on the attention scores, wherein the images corresponding to different attention scores in the attention distribution map are different in color;

the attention profile is superimposed on the full-view digital slice to obtain an attention heat map of the full-view digital slice.

8. A full-field digital slice-based device for detecting crohn's disease and intestinal tuberculosis, the device comprising:

the acquisition module is used for acquiring a plurality of full-view digital slices and determining a plurality of image blocks corresponding to each full-view digital slice, wherein the plurality of full-view digital slices are full-view digital slices of the same target object;

The control module is used for controlling the trained detection network model to determine an initial prediction probability matrix corresponding to each full-field digital slice;

and the determining module is used for determining a prediction result of the target object based on the initial prediction probability matrix of each full-field digital slice, wherein the prediction result comprises a Crohn disease category, an intestinal tuberculosis category or a normal tissue category.

9. A computer readable storage medium storing one or more programs executable by one or more processors to perform the steps in the method of detecting crohn's disease and tuberculosis based on full field digital slicing as in any one of claims 1-7.

10. A terminal device, comprising: a processor and a memory;

the memory has stored thereon a computer readable program executable by the processor;

the processor, when executing the computer readable program, implements the steps in the method for detecting crohn's disease and tuberculosis based on full-field digital slicing as described in any one of claims 1-7.