RU2828554C1 - System and method for diagnosing sinusitis by x-ray images - Google Patents

Abstract

FIELD: image processing.

SUBSTANCE: group of inventions relates to image processing and more specifically to support of medical decision making based on analysis of medical images. Disclosed is a system for determining pathologies of paranasal sinuses by X-ray images for implementing the method, which comprises a unit for obtaining X-ray images of the paranasal sinuses; pathology diagnostic unit, which, using a segmentation-based neural network, detects on the obtained image areas of reduced pneumatisation and areas with fluid in the paranasal sinuses; an anatomical structure segmentation unit, which determines on the obtained image areas with high pneumatisation and external contours of each paranasal sinus using a neural network based on segmentation; pathology localization unit, which determines, by combining the output data from the pathology diagnostic unit and the output data from the anatomical structures segmentation unit, the pathologies belonging to a particular sinus; and a unit for transmitting a report to a device requesting diagnostics, wherein when training neural networks, a focal loss function and pixel weight are used, which are weights for positive pixels of the true class mask and negative pixels of the true class mask, based on the frequency of pixels belonging to a specific class in the entire sample.

EFFECT: group of inventions provides higher diagnostic accuracy and higher automation of processing.

6 cl, 2 dwg

Description

Field of technology to which the invention relates

The present invention relates to the field of image processing, and more particularly to supporting medical decision making based on the analysis of medical images.

State of the art

Sinusitis is an inflammation of the membranes lining the paranasal sinuses. Sinusitis is one of the most common diseases in clinical practice, affecting more than 15% of the population annually. Its diagnosis is usually based on a combination of clinical and radiographic examinations. Although computed tomography (CT) is the preferred imaging modality for sinusitis because it provides the greatest gross anatomical detail of the paranasal sinuses, the use of radiographic images such as Waters and Caldwell images has been the traditional method of evaluating the paranasal sinuses and remains widely used as an imaging modality when sinusitis is suspected due to the availability, simplicity, and relatively low cost of the examination, as well as the low radiation dose. Radiographic examinations are aimed at qualitatively assessing the condition of the paranasal sinus mucosa and assessing the amount of fluid in the paranasal sinus, which, in combination with clinical features, allows for a diagnosis of acute or chronic sinusitis.

In recent years, with the widespread development of information technology in medicine, digital data processing methods have been actively introduced, including artificial intelligence (AI) technologies to support medical decision-making. For example, in the known methods of modern AI diagnostics, a neural network that has been pre-trained on a labeled set of images receives a digitized X-ray image of the area being examined, analyzes it and predicts whether the patient has a pathology, thereby helping the doctor make an optimal, balanced decision.

Known methods for diagnosing sinusitis using AI are based on a combination of a neural network detector with a neural network classifier for two-dimensional images. This limits the accuracy of the systems used and requires a large amount of data for their training.

There are also examples of networks that combine segmentation of the target region of the paranasal sinuses as a single class with subsequent transfer of the target region and its classification for the presence of pathology and regression that identifies the level of reduction in pneumatization of the sinuses. Such methods of automatic diagnosis of paranasal sinus pathologies require a significant number of images to train the neural network of the classifier and regressor to achieve target metrics of pathology detection accuracy and assessment of quantitative characteristics of the pathology.

The essence of the invention

In order to eliminate at least some of the above-mentioned disadvantages of the prior art, the present invention is directed to improving the efficiency of systems and methods for diagnosing sinusitis using artificial intelligence.

In particular, the present invention effectively overcomes the shortcomings of known solutions by using segmentation-based classification and segmentation-based detection for such pathological features as decreased pneumatization of the paranasal sinuses, in combination with segmentation of normal anatomical structures and analysis of quantitative characteristics of each sinus based on segmentation masks, as well as by eliminating false-positive results based on an analysis of the intersection of masks of normal anatomy and pathology and eliminating false-negative results by using the calculated quantitative characteristics of decreased pneumatization of the paranasal sinuses.

According to a first aspect of the present invention, a system for diagnosing sinusitis using X-ray images is provided, comprising:

an imaging unit configured to obtain a two-dimensional X-ray image of the paranasal sinus region;

a pathology diagnostic unit, configured to detect, using at least one processor and memory, using a neural network based on segmentation, areas of reduced pneumatization and areas with fluid in the paranasal sinuses on the obtained image;

a segmentation unit for anatomical structures, configured to, using at least one processor and memory, determine on the obtained image areas with high pneumatization and the external contours of each paranasal sinus using a neural network based on segmentation;

a pathology localization unit configured to, using at least one processor and memory, determine by combining output data from the pathology diagnostic unit and output data from the anatomical structure segmentation unit, the belonging of pathologies to a specific paranasal sinus; and

a report transmission unit configured to generate a report containing information on the presence or absence of paranasal sinus pathologies in the received image based on output data from at least one of the pathology diagnostic unit, the anatomical structure segmentation unit, and the pathology localization unit,

wherein each neural network is pre-trained to determine a mask for each class corresponding to it, which contains the probabilities of belonging of each pixel to this class, wherein during training a loss function is used based on the Jaccard coefficient and the class weights of the positive and negative class labels, which are determined based on the inverse frequency of positive and negative class labels for all images in the sample for each class, wherein if for a given image in the true mask of a given class there are positive elements, then the entire instance is assigned a positive class label, if absent - a negative class label, wherein the loss function based on the Jaccard coefficient is calculated for each class mask after determining the class mask for each class, and if the true class label corresponding to the image is positive, then the error of the corresponding class is multiplied by the class weight of the positive instance of the class, and if negative - by the class weight of the negative instance of the class, respectively.

In one embodiment, the report transmission unit is additionally configured to transmit a copy of the input image with contours applied to it, limiting the area of the pathology, text information about the localization of the pathology and numerical values determining the degree of reduction in pneumatization of the paranasal sinuses.

In one embodiment, the pathology localization unit is additionally configured to check image areas marked by the pathology diagnostics unit as containing pathology for intersection with image areas identified by the anatomical structure segmentation unit as corresponding to a particular paranasal sinus, and to localize pathologies based on this.

In one embodiment, the pathology localization unit is additionally configured to indicate a decrease in pneumatization for a sinus in which the level of pneumatization reduction, measured based on the analysis of normal anatomical structures identified by the anatomical structure segmentation unit, exceeds a critical threshold.

In one embodiment, each neural network is pre-trained to determine a mask for each class corresponding to it, which contains the probabilities of each pixel belonging to this class, wherein during training a focal loss function and pixel weights are used, which represent weights for positive pixels of the true class mask and negative pixels of the true class mask, based on the frequency of pixels belonging to a specific class in the entire sample, wherein the focal loss function is calculated for each pixel of the mask after determining the class mask, and if the corresponding pixel of the true class mask is positive, the error obtained on this pixel of the class mask is multiplied by the weight of the pixel of a positive example of this class, and if the corresponding pixel of the true class mask is negative, the error obtained on this pixel of the class mask is multiplied by the weight of the pixel of a negative example of this class.

According to a second aspect of the present invention, a method for diagnosing sinusitis using X-ray images is provided, comprising the steps of:

Using an imaging unit, a two-dimensional X-ray image is obtained containing the paranasal sinus area;

using a pathology diagnostic block containing a neural network for segmentation, areas of reduced pneumatization and areas with fluid in the paranasal sinuses are determined on the resulting image;

using a block for segmentation of anatomical structures containing a neural network for segmentation, areas with high pneumatization and the external contours of each paranasal sinus are determined on the resulting image;

using the pathology localization unit, based on the output data from the pathology diagnostics unit and the output data from the anatomical structure segmentation unit, by determining the intersection between areas where there is a decrease in pneumatization and which correspond to a specific paranasal sinus, the localization of the pathology is determined; and

using a report transmission unit, a report is generated and transmitted containing information about the presence or absence of paranasal sinus pathologies in the received image based on output data from at least one of the pathology diagnostic unit, the anatomical structure segmentation unit, and the pathology localization unit, to the device that requested the diagnostics,

wherein each neural network is pre-trained to determine a mask for each class corresponding to it, which contains the probabilities of each pixel belonging to this class, wherein during training a loss function is used based on the Jaccard coefficient and the class weights of the positive and negative class labels, which are determined based on the inverse frequency of positive and negative class labels for all images in the sample for each class, wherein if for a given image in the true mask of a given class there are positive elements, then the entire instance is assigned a positive class label, if absent - a negative class label, wherein the loss function based on the Jaccard coefficient is calculated for each class mask after determining the class mask for each class, and if the true class label corresponding to the image is positive, then the error of the corresponding class is multiplied by the class weight of the positive instance of the class, and if negative - by the class weight of the negative instance of the class, respectively.

Technical result

The present invention makes it possible to increase the efficiency of systems and methods for diagnosing sinusitis in the paranasal sinuses using two-dimensional X-ray images. This ensures:

- increasing the accuracy of diagnostics;

- increasing the speed of processing two-dimensional images;

- simplification of interpretation and description of images;

- the possibility of performing early diagnostics of paranasal sinus diseases in screening mode;

- freeing medical personnel from routine processes of analyzing images of patients who have no symptoms and previously identified pathologies in the paranasal sinuses;

- reducing the influence of the human factor (attentiveness, fatigue, responsibility).

- increasing the ratio between the number of images used for training and the obtained classification and detection metrics.

These and other advantages of the present invention will become apparent from reading the following detailed description with reference to the accompanying drawings.

Brief description of the drawings

Fig. 1 shows an example of X-ray images of the paranasal sinuses in a direct projection with markings applied.

Fig. 2 shows a block diagram of a sinusitis diagnostic system according to the present invention.

It should be understood that the figures may be diagrammatic and not to scale and are intended primarily to facilitate an understanding of the present invention.

Detailed description

Diseases of the paranasal sinuses are usually diagnosed by physicians in practice by manual analysis of images. The present invention provides a method and system that automate the diagnostic process, providing the physician with useful data such as the presence of thickening of the walls of the paranasal sinuses, the presence of fluid levels, the degree of reduction of pneumatization, and the localization of pathology.

The present inventors collected an anonymous database of patients with and without sinusitis. Each patient case consists of a AP image of the paranasal sinuses. On each X-ray image, an experienced radiologist manually outlined the outer contour of each sinus, each sinus in which decreased pneumatization was detected, and the fluid detected in the sinus. An example of AP X-ray images of the paranasal sinuses with the applied markings is shown in Fig. 1. As will be shown below, such a database will be used to train machine learning models.

General description of the system

As shown in the block diagram in Fig. 2, the sinusitis diagnostic system 100 according to the present invention comprises an image acquisition unit 110, an anatomical structure segmentation unit 120, a pathology diagnostic unit 130, a pathology localization unit 140, and a report transmission unit 150.

In one embodiment, the image acquisition unit may be a network card or other data receiving means. In another embodiment, the image acquisition unit may be the X-ray apparatus itself.

The pathology diagnostics unit, the pathology localization unit, and the anatomical structure segmentation unit are implemented on the basis of a processing device containing a computing module (e.g., a central processor), a memory module (e.g., RAM and ROM), and, if necessary, an additional computing module (e.g., a graphic processor or a neural processor). Thus, the pathology diagnostics unit, the pathology localization unit, and the anatomical structure segmentation unit can be implemented on the basis of a computer or a server.

The report transmission unit may be a network card, a hardware data/video signal transmission interface, or other data exchange means.

The method of diagnosing sinusitis is performed by the sinusitis diagnostic system and is as follows.

First, a two-dimensional image of the paranasal sinuses is obtained using an imaging unit.

As indicated above, in one embodiment, if the image acquisition unit is an X-ray machine, it can acquire an image independently (or under the control of an operator). In another embodiment, the image acquisition unit, being part of a pathology diagnostic service, can receive from outside (e.g., from a medical organization, from a doctor, from a regional medical data storage system, etc.) a diagnostic request or input data containing one or a plurality (e.g., a batch) of images, and extract the desired image from the received input data or request.

If necessary, the resulting image is pre-processed in a pre-processing block (not shown in Fig. 2) so that it is suitable for processing by further blocks. For example, if the original image has insufficient or excessive resolution in any direction, then the image can be transformed or reconstructed to the required resolution. Brightness, contrast, and other image parameters can also be adjusted.

The image is then fed to the anatomical structure segmentation unit and the pathology diagnostic unit.

In the anatomical structures segmentation block, the probability of class presence and binary segmentation masks are determined for each of the following classes using a pre-trained neural network: “outer contour - right maxillary sinus”, “outer contour - left maxillary sinus”, “outer contour - right frontal sinus”, “outer contour - left frontal sinus”, “area of high pneumatization - right maxillary sinus”, “area of high pneumatization - left maxillary sinus”, “area of high pneumatization - right frontal sinus”, “area of high pneumatization - left frontal sinus”. After that, the degree of pneumatization reduction is calculated for each paranasal sinus using the following formula:

X% = (S _sinus – S _{internal void} ) / S _sinus ,

where S is the number of pixels belonging to the corresponding classes. The output of the anatomical structure segmentation block is a binary segmentation mask and a set of values from 0 to 1 corresponding to the degree of reduction in pneumatization of each paranasal sinus.

In the pathology diagnostics block, binary pathology masks and probabilities of pathology class presence are generated for each pathology class using a pre-trained neural network. The pathology classes are the following classes: “reduced pneumatization of the paranasal sinus”, “horizontal fluid level in the sinus”. In the case of the “reduced pneumatization of the paranasal sinus” class, the entire sinus with reduced pneumatization is matched with the desired class during data labeling. In the case of the “horizontal fluid level in the sinus” class, the area presumably occupied by fluid in the doctor’s opinion is matched with the desired class during labeling. The output of the pathology diagnostics block is binary pathology masks and a set of values from 0 to 1 for each pathology, interpreted as the probability of its presence.

The output data from the anatomical structure segmentation and pathology diagnostics units are sent to the pathology localization unit. In the pathology localization unit, image areas marked as containing pathology are checked for intersection with image areas marked as corresponding to a particular maxillary or frontal sinus, thereby localizing the pathology and reducing or even eliminating false positive results.

In one implementation, for a sinus in which the level of pneumatization reduction measured by the analysis of masks of normal anatomical structures exceeds a critical threshold, a “pneumatization reduction” mask is assigned. This ensures a reduction or even elimination of false negative results.

In fact, both the anatomical structure segmentation block and the pathology diagnostics block first perform segmentation, and then, based on this segmentation, form a binary mask and calculate probabilities. The difference is that in the anatomical structure segmentation block, segmentation is intended to classify normal anatomical structures and anatomical structures with pathology, while in the pathology diagnostics block, segmentation is intended to detect pathologies in general. That is, both of these blocks search for pathologies, but they approach this from different points of view. Then, the data from these blocks is combined in the pathology localization block to improve the accuracy of diagnostics and cut off false positives that could occur if only one of the two approaches using segmentation were used, rather than the proposed combination. This compares favorably with known solutions from the state of the art, which either do not use segmentation at all, or use segmentation only to detect anatomical structures.

Next, the output from the pathology localization unit is transmitted to the report transmission unit. In one embodiment, a report on the state of the paranasal sinuses can be generated for each image. In another embodiment, a diagnostic request or diagnostic system settings can provide for the need to generate a report only for cases in which pathology is detected. For example, if the system has detected one case of pathology in a received batch of 10 images, it can generate a report for the corresponding image (or patient) containing the calculated clinical indicators. The report can also contain a copy of the input image generated by the report transmission unit with pathology contours applied to it.

Next, the report on the health status of the paranasal sinuses is displayed on the monitor screen connected to the server, or transmitted using the report transmission unit to the device that requested the diagnostics - for example, to a remotely located automated workstation (AWS) of the doctor.

Using the image with the contours applied to it and other data contained in the report (the degree of reduction in pneumatization of each sinus, an indication of the presence/absence of pathology - sinusitis), the doctor makes a decision on the patient's health condition.

Detection and classification based on segmentation masks

Below, the process performed in the blocks of segmentation of anatomical structures and diagnostics of pathologies will be described in detail.

For each mask of each class, the output of the neural network is a class mask (respectively, the class of the anatomical structure or the class of pathology) with probabilities of pixel belonging to the class in the range of 0-1. For each class mask, image areas are determined - candidates for belonging to the class. Determination of candidates means simple segmentation by a threshold calculated on the basis of maximization of the pixel-by-pixel F1-score, hereinafter referred to as the segmentation threshold. The result of segmentation is a binary class mask, hereinafter referred to as an object mask. Next, using the opencv library, a search for contours for segments contained in the object mask is performed. Next, for each contour on the object mask from the contour, a mask of the area of interest is generated - this means that all elements of the mask lying inside this contour are considered equal to one, and outside this contour - equal to zero. The mask of the area of interest is multiplied element by element by the original class mask. Next, all non-zero pixels of the region of interest mask (N) and the sum of probabilities for all pixels of the mask (M) are counted. The probability of belonging of the considered area (region of interest) to the class is defined as P=M/N. P is compared with the detection threshold. The detection threshold is the threshold calculated based on maximization of average F1 for the object mask. If P is greater than the detection threshold, the area is considered to belong to the desired class. If P is less than the detection threshold, the candidate area is considered to be an area with a false positive response, and the object mask in the area of the corresponding area is reset to zero. For each isolated area of the object mask, the obtained average probability for the area is saved. The maximum probability for all isolated areas is considered to be the probability of the presence of the class as a whole.

The neural network performing the segmentation may be based on various suitable architectures known to the person skilled in the art. As non-limiting examples, the neural network may be based on the UNet or FPN architecture.

UNet as a source of basic segmentation

The UNet architecture was developed for image segmentation and is currently widely used in medical imaging due to its versatility and high-accuracy segmentation results. It consists of a downsampling encoder path that extracts image features at different scales and complexities, and an upsampling decoder path that generates segmentation results using the extracted features. The basic concept of UNet is to use skip connections that run from the encoder to the decoder, going from the early layers of the encoder path, where fine image details are still preserved, to the later layers of the decoder path, which contain the coarse feature information needed for object recognition and segmentation. The encoder path consists of four blocks, each consisting of two 3×3×3 convolutional layers followed by a ReLU activation layer and a 2×2×2 max-pooling layer. The decoder path consists of four blocks, each consisting of an upsampling layer, a 2×2×2 transpose convolution layer, and two 3×3×3 convolution layers, each followed by a ReLU layer.

UNet can be trained using the Adam optimizer with a learning rate of 0.0002 and a weight decay of 0.0001.

FPN as a source of basic segmentation

The FPN architecture with RPN is based on the principle of efficient multi-scale feature extraction, which allows combining semantically robust features with low resolution and semantically weak features with high resolution. This approach is effective for the task of object detection. The FPN component extracts features from the input image, and during this process, it has some similarities with UNet. The decoder part creates higher-resolution feature maps by upsampling the results of previous layers, and each block in the decoder path generates a separate output for training the network. Trailing connections consist of 1×1 convolution layers applied to encoder blocks, which are then added to the corresponding decoder blocks. As a result, FPN aims to generate variable-resolution features that can describe the target object. In a specific example, FPN can be trained using the Adam optimizer with a learning rate of 0.0002 and a weight decay of 0.0001.

Training neural networks

In neural network training, it is proposed to use the mean value of the focal loss function as a loss function - this is a variation of crossentropy designed to solve the problem of class imbalance, and the loss function based on the Jaccard coefficient (Jaccard loss), the feature of which is the independence of the error value from the size of objects in the image.

Pixel-weight combined with focal loss to improve segmentation accuracy

In one embodiment, the focal loss weights may be designed to correct for systematic bias caused by the different number of pixels that belong, on average across the sample, to each class and background.

For this purpose, the focal loss can use weights for positive elements of the true class mask and negative elements of the class mask based on the frequency of pixels belonging to a specific class in the entire sample (hereinafter pixel-weight, pixel weights). After obtaining the segmentation mask, the focal loss is calculated for each mask element. If the corresponding element of the true segmentation mask is positive, the error obtained on this segmentation mask element is multiplied by the pixel-weight of the positive example of this class. If the segmentation mask element is negative, the error obtained on this segmentation mask element is multiplied by the pixel-weight of the negative example of this class. After multiplying by the weights, the error is averaged over all elements of the weighted error. Thus, the segmentation accuracy increases.

Class-weight combined with Jaccard loss to improve segmentation accuracy

In one embodiment, the Jaccard loss weights may be designed to correct for the systematic imbalance caused by the fact that not all images contain pathological classes, as opposed to normal anatomy classes.

For this purpose, for each error in Jaccard loss, it is proposed to use the proper weights of each class. The pathology classes in the present invention are not opposed to each other. The normal anatomy classes in the present invention are not opposed to each other.

If for some image in the true mask of some class there are positive elements, then the whole instance is assigned a positive class label, if absent - a negative class label. Based on the frequency of positive and negative class labels for all images in the sample for each class, based on the inverse frequency of class labels, class weights (class-weight) of the positive and negative class labels are determined. After obtaining the segmentation masks for each class, Jaccard loss is calculated for each mask. If the true class label corresponding to the image is positive, then the error of the corresponding class is multiplied by the class-weight for the positive instance of the class, if negative - by the class-weight for the negative instance of the class, respectively. The resulting error is averaged. Thus, the accuracy of segmentation increases.

Example of implementation

The proposed system was tested on a database containing 1123 paranasal sinus X-ray images (specifically 523 without pathology and 600 with pathology). From this set, 75% of the images were used for training and 25% for testing.

The age of the patients was not taken into account. The most common diagnosis was chronic maxillary sinusitis. The second most common pathology was acute maxillary sinusitis.

The parameters of individual components of the system were determined as follows.

The network was trained in the multilabel-segmentation mode, sigmoid activation was applied to the network output using the Jaccard loss and focal loss error functions, followed by calculating their average.

The learning rate for the models was chosen to be 0.00001. The framework was trained on a server equipped with a Tesla V100-SXM2-80GB graphics processing unit (GPU), an 80-core Intel(R) Xeon(R) E5-2698 v4 central processing unit (CPU) @ 2.20 GHz, and 503GB of RAM.

The effectiveness of the system was assessed by assessing the accuracy of the IOU indicator for segmentation masks and class labels calculated based on segmentation, according to the method described earlier, using such classification metrics as recall, precision, F1-score. The results were compared with other systems and with human assessments, and it was found that the proposed system can successfully quantitatively determine paranasal sinus pathologies, demonstrating increased accuracy and speed of diagnosis for almost all indicators. With the presence of an external correction mechanism, it is possible to further improve the accuracy of diagnosis, while the help of an external specialist is required only in cases where there is a possibility of an incorrect automatic diagnosis.

Application

The systems and methods according to the present invention can be used to process diagnostic medical images containing the paranasal sinus region in order to detect signs of sinusitis therein.

Additional implementation features

Although this document may indicate that data is transmitted/sent or received/received by a person (e.g., a medical professional, a doctor, an expert), one skilled in the art should understand that such indication is used solely for the purpose of simplifying the description, whereas in fact it is meant that the data is transmitted/sent or received/received by the corresponding device used and/or controlled by such person.

One or more of the transmission units or devices (transmitters) and one or more reception units or devices (receivers) described herein may be physically implemented in the same transmission/reception unit or device or in different units or devices.

In this document, a device or unit for transmitting may be referred to as a device or unit having functions of not only transmitting but also receiving data, information and/or signals for ease of description. Similarly, a device or unit for receiving may also include functions of transmitting data, information and/or signals.

The various illustrative blocks and modules described in connection with the disclosure herein may be implemented or executed by a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete logic element or transistor logic, discrete hardware components, or any combination of the foregoing designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (e.g., a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration).

Some units individually or together may represent, for example, a computer, and include a processor that is configured to call and execute computer programs from the memory to perform the steps of the method or the functions of the units according to the embodiments of the present invention. According to the embodiments, the device may further include a memory. The processor may call and execute computer programs from the memory to perform the method. The memory may be a separate device independent of the processor, or may be integrated into the processor. The memory may store code, instructions, commands and/or data for execution on a set of one or more processors of the described device. The codes, instructions, commands may cause the processor to perform the steps of the method or the functions of the device.

The functions described in this document may be implemented in hardware, software executed by one or more processors, firmware, or any combination thereof. The hardware and software implementing the functions may also be physically located in different locations, including in a distribution such that parts of the functions are implemented in different physical locations, i.e., distributed processing or distributed computing may be performed.

If necessary (for example, if the volume of data and/or calculations that need to be performed on this data is large), multi-threaded data processing can be performed, which in a simple representation can be expressed in the fact that the entire set of data to be processed is divided into a set of subsets, and each processor core performs processing on the subset of data assigned to it.

The above-mentioned memory may be volatile or nonvolatile memory or may include both volatile and nonvolatile memory. It should also be clear to a person skilled in the art that when talking about memory and storing data, programs, codes, instructions, commands, etc., it is implied that there is a machine-readable (or computer-readable, processor-readable) storage medium. Machine-readable storage media include both non-transitory computer storage media and communication media, including any transmission medium that facilitates the movement of a computer program or a portion thereof from one place to another. A non-transitory machine-readable storage medium may be any available medium that can be used to carry or store the desired program code means in the form of instructions or data structures and that can be accessed by a computer, processor or other general-purpose or special-purpose processing device.

By way of example, and not limitation, computer-readable media may comprise read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electronically erasable programmable read-only memory (EEPROM), flash memory, random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), increased speed synchronous dynamic random access memory (ESDRAM), synchronous line DRAM (SLDRAM), and direct access bus random access memory (DR RAM), register, cache memory, semiconductor memories, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROMs and digital versatile discs (DVD), as well as any other data carriers known in the prior art.

The information and signals described herein may be represented by any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, and signal elements, which may be exemplified in the above description, may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof, as applicable to the present invention.

At least one of the steps in the method or units in the device may use an artificial intelligence (AI) model to perform the corresponding operations. The function associated with AI may be performed via a processor and non-volatile and/or volatile memory.

The processor may include one or more processors. At the same time, the one or more processors may be a general-purpose processor, such as a central processing unit (CPU), an application processor (AP), etc., a graphics-only processing unit, such as a graphics processing unit (GPU), a visual processing unit (VPU), and/or a specialized AI processor, such as a neural processing unit (NPU).

One or more processors control the processing of input data in accordance with a predetermined operating rule or an artificial intelligence (AI) model stored in non-volatile memory and/or volatile memory. The predetermined operating rule or an AI model may be obtained by training. In this case, the processor may perform a pre-processing operation of the data to transform it into a form suitable for use as input data for the AI model.

"Obtained by training" means that by applying a training algorithm to a trainable artificial intelligence model using a plurality of training data, a predetermined operating rule or AI model with a desired characteristic is created. The training may be performed on the device itself, in which the AI is executed according to the embodiment, and/or may be implemented through a separate server/system.

An artificial intelligence model may include multiple neural network layers. Each of the multiple neural network layers includes multiple weight values (coefficients) and performs a work operation for that layer by calculating using the multiple weight values of that layer with respect to the input data or the calculation result of the previous layer.

Examples of neural networks include, but are not limited to, convolutional neural network (CNN), deep neural network (DNN), recurrent neural network (RNN), restricted Boltzmann machine (RBM), deep belief network (DBN), bidirectional recurrent deep neural network (BRDNN), generative adversarial networks (GANs), and deep Q-networks.

A learning algorithm is a method for training a predetermined target device (e.g., a GPU-based neural network or NPU) using a set of training data to cause, enable, or control the target device to make a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

It should be understood that although terms such as "first", "second", "third", etc. may be used herein to describe various blocks, modules, networks, elements, components, regions, layers and/or sections, these blocks, modules, networks, elements, components, regions, layers and/or sections should not be limited by these terms. These terms are used only to distinguish one block, module, network, element, component, region, layer or section from another block, module, network, element, component, region, layer or section. Thus, a first block, module, network, element, component, region, layer or section may be referred to as a second block, module, network, element, component, region, layer or section without departing from the scope of the present invention. As used herein, the term "and/or" includes any and all combinations of one or more of the corresponding listed items. Elements mentioned in the singular do not exclude a plurality of elements unless otherwise specifically indicated.

The functionality of an element indicated in the description or claims as a single element may be implemented in practice by means of several components of the device, and vice versa, the functionality of elements indicated in the description or claims as several separate elements may be implemented in practice by means of a single component.

In one embodiment, some or all elements/blocks/modules of the proposed device are in a common housing, can be placed on a single frame/structure/printed circuit board/crystal and are structurally connected to each other by means of assembly operations and functionally by means of communication lines. The said communication lines or channels, unless otherwise specified, are typical communication lines known to specialists, the material implementation of which does not require creative efforts. A communication line can be a wire, a set of wires, a bus, a track, a wireless communication line (inductive, radio frequency, infrared, ultrasonic, etc.). Communication protocols over communication lines are known to specialists and are not disclosed separately.

The functional connection of elements should be understood as a connection that ensures the correct interaction of these elements with each other and the implementation of one or another functionality of the elements. Particular examples of a functional connection may be a connection with the ability to exchange information, a connection with the ability to transmit electric current, a connection with the ability to transmit mechanical movement, a connection with the ability to transmit light, sound, electromagnetic or mechanical vibrations, etc. The specific type of functional connection is determined by the nature of the interaction of the mentioned elements, and, unless otherwise specified, is ensured by widely known means, using widely known principles in technology.

Although exemplary embodiments have been described in detail and shown in the accompanying drawings, it should be understood that such embodiments are illustrative only and are not intended to limit the present invention, and that the present invention should not be limited to the specific arrangements and structures shown and described, since various other modifications and embodiments of the invention may be apparent to one skilled in the art from the information set forth in the description and knowledge of the prior art without departing from the spirit and scope of the present invention.

Claims (18)

1. A system for diagnosing sinusitis using X-ray images, containing: an imaging unit configured to obtain a two-dimensional X-ray image of the paranasal sinus region; a pathology diagnostic unit, configured to detect, using at least one processor and memory, using a neural network based on segmentation, areas of reduced pneumatization and areas with fluid in the paranasal sinuses on the obtained image; a segmentation unit for anatomical structures, configured to, using at least one processor and memory, determine on the obtained image areas with high pneumatization and the external contours of each paranasal sinus using a neural network based on segmentation; a pathology localization unit configured to, using at least one processor and memory, determine by combining output data from the pathology diagnostic unit and output data from the anatomical structure segmentation unit, the belonging of pathologies to a specific paranasal sinus; and a report transmission unit configured to generate a report containing information on the presence or absence of paranasal sinus pathologies in the received image based on output data from at least one of the pathology diagnostic unit, the anatomical structure segmentation unit, and the pathology localization unit, wherein each neural network is pre-trained to determine a mask for each class corresponding to it, which contains the probabilities of belonging of each pixel to this class, wherein during training a loss function is used based on the Jaccard coefficient and the class weights of the positive and negative class labels, which are determined based on the inverse frequency of positive and negative class labels for all images in the sample for each class, wherein if for a given image in the true mask of a given class there are positive elements, then the entire instance is assigned a positive class label, if absent - a negative class label, wherein the loss function based on the Jaccard coefficient is calculated for each class mask after determining the class mask for each class, and if the true class label corresponding to the image is positive, then the error of the corresponding class is multiplied by the class weight of the positive instance of the class, and if negative - by the class weight of the negative instance of the class, respectively. 2. The system according to claim 1, in which the report transmission unit is additionally configured to transmit a copy of the input image with contours applied to it, limiting the area of pathology, text information about the localization of the pathology and numerical values determining the degree of reduction in pneumatization of the paranasal sinuses. 3. The system according to claim 1, in which the pathology localization unit is additionally configured with the ability to check image areas marked by the pathology diagnostic unit as containing pathology for intersection with image areas selected by the anatomical structure segmentation unit as corresponding to a particular paranasal sinus, and to localize pathologies on this basis. 4. The system according to claim 1, in which the pathology localization unit is additionally configured to indicate a decrease in pneumatization for a sinus in which the level of pneumatization reduction, measured based on the analysis of normal anatomical structures identified by the anatomical structure segmentation unit, exceeds a critical threshold. 5. The system according to claim 1, in which each neural network is pre-trained to determine a mask for each class corresponding to it, which contains the probabilities of each pixel belonging to the given class, wherein during training a focal loss function and pixel weights are used, which represent weights for the pixels of a positive example of the true class mask and the pixels of a negative example of the true class mask, based on the frequency of pixels belonging to a specific class in the entire sample, wherein the focal loss function is calculated for each pixel of the mask after determining the class mask, and if the corresponding pixel of the true class mask belongs to a positive example, the error obtained on this pixel of the class mask is multiplied by the weight of the pixel of the positive example of the given class, and if the corresponding pixel of the true class mask belongs to a negative example, the error obtained on this pixel of the class mask is multiplied by the weight of the pixel of the negative example of the given class. 6. A method for diagnosing sinusitis using X-ray images, comprising the following steps: Using an imaging unit, a two-dimensional X-ray image is obtained containing the paranasal sinus area; using a pathology diagnostic block containing a neural network for segmentation, areas of reduced pneumatization and areas with fluid in the paranasal sinuses are determined on the resulting image; using a block for segmentation of anatomical structures containing a neural network for segmentation, areas with high pneumatization and the external contours of each paranasal sinus are determined on the resulting image; using the pathology localization unit, based on the output data from the pathology diagnostics unit and the output data from the anatomical structure segmentation unit, by determining the intersection between areas where there is a decrease in pneumatization and which correspond to a specific paranasal sinus, the localization of the pathology is determined; and using the report transmission unit, a report is generated and transmitted, containing information about the presence or absence of paranasal sinus pathologies in the received image based on output data from at least one of the pathology diagnostic unit, the anatomical structure segmentation unit and the pathology localization unit, to the device that requested the diagnostics, wherein each neural network is pre-trained to determine a mask for each class corresponding to it, which contains the probabilities of each pixel belonging to this class, wherein during training a loss function is used based on the Jaccard coefficient and the class weights of the positive and negative class labels, which are determined based on the inverse frequency of positive and negative class labels for all images in the sample for each class, wherein if for a given image in the true mask of a given class there are positive elements, then the entire instance is assigned a positive class label, if absent - a negative class label, wherein the loss function based on the Jaccard coefficient is calculated for each class mask after determining the class mask for each class, and if the true class label corresponding to the image is positive, then the error of the corresponding class is multiplied by the class weight of the positive instance of the class, and if negative - by the class weight of the negative instance of the class, respectively.