CN105913075A - Endoscopic image focus identification method based on pulse coupling nerve network - Google Patents

Abstract

The invention relates to an endoscopic image focus identification method based on the pulse coupling nerve network. The method comprises steps of video framing, image preprocessing, visual-perception-oriented color space conversion, suspected focus area positioning, characteristic vector construction, mode identification through employing the pulse coupling nerve network, area transfer, accomplishment of all-suspected-focus identification, focus image classification extraction and repetition of all the steps till accomplishment of identification of endoscopic images of a whole video file. Through the method, on the basis of focus area positioning of the human visual attention mechanism and the pulse coupling nerve network, mode identification is carried out, focus mode identification accuracy is improved, focus mode endoscopic image extraction accuracy is further improved, and workload of clinical doctors is reduced.

Description

A method for identifying lesions in endoscopic images based on pulse-coupled neural network

technical field

The invention relates to the field of medical image pattern recognition and analysis, in particular to a lesion image pattern recognition method of a human gastrointestinal endoscopy detection system.

Background technique

Gastrointestinal diseases are increasingly threatening human health. At the same time, many other diseases may be directly or indirectly caused by diseases of the gastrointestinal system. The examination and diagnosis of gastrointestinal diseases are of great significance to human health. . The best way to detect gastrointestinal diseases is to directly observe the gastrointestinal tract, so endoscopy is a more direct and effective method. However, traditional insertion endoscopes, such as colonoscopes and gastroscopes, cannot go deep into the intestinal tract due to mechanical insertion, making the small intestine a blind spot for detection. There is a risk of intestinal perforation. With the development of semiconductor technology, sensor technology, LED lighting technology, wireless communication and micro-control technology, the foundation has been laid for the emergence and popularization of wireless capsule endoscope. The wireless capsule endoscope is composed of a miniature image sensor, an illumination module, a wireless transmission module, and a power management module. After the patient swallows it, the capsule endoscope moves down the digestive tract under the peristalsis of the human gastrointestinal tract. During the movement, the glass cover at the front of the capsule stretches the intestinal tract and clings to the intestinal wall, the lighting module illuminates the intestinal wall in the field of view, and the image sensor obtains the image of the intestinal wall through the short focal length lens, and transmits the image data out of the body. The capsule endoscope continuously transmits images of the gastrointestinal tract out of the body until it is naturally excreted from the human body through the anus. The whole process does not require manual intervention, will not bring any pain and inconvenience to the patient, and there is no detection blind spot, which realizes painless and non-invasive detection of the whole digestive tract. It is precisely because of these advantages that capsule endoscopy, as a new type of digestive tract detection technology, is increasingly used in clinical practice.

The working time of the capsule endoscope in the human body is about 8 hours, and the metabolism time of people with gastrointestinal diseases will be longer, so a detection will generate at least 2×3600×8=57600 frame images. Finding lesions or pathological features in such a huge number of video images is a very time-consuming and labor-intensive task, even for experienced experts, it will take at least 2 hours. This not only wastes time, but also leads to missed inspections due to visual fatigue. Therefore, it is an inevitable trend to use image processing and pattern recognition technology to realize computer intelligent bleeding image recognition. Since the endoscopic image is an image of the human digestive tract, the situation is very complicated, and the characteristics of the lesions are also changeable. It is difficult to deal with complex endoscopic images and changeable lesions by using commonly used digital image processing and pattern recognition algorithms. The pulse-coupled neural network comes from the working mechanism of the mammalian visual nerve. Compared with the traditional neural network models such as BP and RBF, this model has inherent advantages in the field of image processing, and has shown outstanding performance in some applications. It has great application potential in the intelligent identification of lesions.

Existing endoscopic detection technologies for the human gastrointestinal tract are all focused on the detection body of the capsule endoscope, while the research on lesion image processing and pattern recognition is relatively lagging behind, which has become the bottleneck of the capsule endoscopic detection system. Moreover, image lesion recognition technologies all focus on the pattern recognition of specific lesions, but due to the variability of lesions, even the same type of lesions have very variable characteristics. Moreover, conventional digital image processing and pattern recognition algorithms are also difficult to deal with complex endoscopic images, resulting in low specificity and sensitivity of the recognition method.

Contents of the invention

In order to overcome the difficulty of identifying and extracting existing lesion patterns, the present invention provides a positioning method based on pulse-coupled neural network and visual attention mechanism, which does not distinguish the specific type of lesion, and provides a pulse-coupled neural network-based lesion detection method in endoscopic images. The identification method can accurately classify the endoscopic image into normal mode and lesion mode, and mark the identified lesion area of the endoscopic image for clinicians to make further judgments, reducing the workload of clinicians.

The technical scheme that provides in order to solve the above-mentioned technical problem is:

A method for identifying a lesion in an endoscopic image based on a pulse-coupled neural network, the identification method comprising the steps of:

a. Video framing

Input the endoscopic inspection video file, and obtain a single endoscopic image in bitmap format by framing;

b. Image preprocessing

The bitmap image obtained in step a is processed according to the field of view parameter of the endoscope to smooth the edge of the image to obtain an endoscopic image with clear boundaries, and then use a high-pass filter (such as a Butterworth high-pass filter) to filter and denoise , and then use the median filter to filter and enhance, remove the noise in the image area to be processed and retain the high frequency part of the image;

c. Color space conversion for visual perception

The bitmap image obtained in step b is a device-oriented RGB color space, which is converted to a visual perception-oriented Luv color space;

d. Localization of suspected lesions

Using the u and v components of the Luv color space image obtained in step c as input, calculate the color feature saliency map uv(c,s), use the L component as input to calculate the brightness feature saliency map L(c,s), and then use the Lapp The Lass transform algorithm and the virtual connection method are used to obtain the edge area of the salient content in the image, calculate the contour feature saliency map O(c,s) and the texture feature map T(c,s), and convert the obtained color feature saliency map uv (c, s), luminance feature saliency map L(c, s), contour feature saliency map O(c, s) and texture feature saliency map T(c, s) are respectively regularized and fused at multiple scales, Obtain the saliency map S of the image, and then use the etching algorithm to filter out the salient areas with small areas, and then arrange the saliency degrees in the order of the area size, that is, the suspected lesion area;

e. Construct eigenvectors

Taking the saliency map S obtained in step d as input, construct the pixel color feature vector V(uv) and brightness feature vector V(L) in the suspected lesion area, calculate and construct the area contour feature vector V(O) and the texture feature vector V (T);

f. Pattern recognition of lesion area

Using the feature vector constructed in step e as input, using a pulse-coupled neural network to perform pattern recognition to obtain the lesion pattern of the suspected area to be identified, that is, the normal pattern and the lesion pattern;

g. Regional transfer

Recognize according to the order of the size of the suspected lesion areas in the image. If there are other suspected lesion areas, repeat steps e and f for pattern recognition until the identification of all suspected lesion areas is completed;

h. Lesion image classification and extraction

Perform an OR operation on the mode results of all suspected lesion areas in the endoscopic image to obtain the classification mode of the endoscopic image, that is, the image normal mode and the image lesion mode, if the lesion mode marks the lesion area; i. Repeat steps b and c , d, e, f, g, h, until the endoscopic image recognition of the entire video file ends.

Further, in the step e, in the salient region of the endoscopic image saliency map S, construct the pixel color feature vector V'(uv) and the brightness feature vector V'(L), and then map to the high-dimensional space through the Sigmoid kernel function , use the principal component analysis (PCA) method to extract the core principal component features of the feature data, and obtain the dimensionality-reduced color feature vector V(uv) and brightness feature vector V(L). At the same time, calculate and construct the area contour feature in the area The vector V(O) texture feature vector V(T) is used to construct a feature matrix for pattern recognition of lesions.

Further, in the step f, the pulse-coupled neural network is composed of the neuron layer of the input layer, the intercortical model (ICM), and the competition output layer;

The input layer consists of four input channels including color feature input, brightness feature input, contour feature input, and texture feature input. The neuron layer of the cross-cortical model adopts four ICM neurons, and the competition output layer consists of the competition neuron weight matrix LW and the competition function C;

The color feature input and brightness feature input channels of the input layer are connected to the No. 1 ICM neuron input of the ICM neuron layer, and the contour feature input and texture feature input channels are connected to the No. 2 ICM neuron input of the ICM neuron layer. No. 3 and No. 2 ICM neurons are respectively connected to the input of No. 3 and No. 4 ICM neurons, No. 3 and No. 4 ICM neurons are connected to the input of the competition neuron weight matrix LW of the competition output layer, and the weight matrix LW of the competition neuron The output is connected to the input of the competition function C of the competition layer neurons;

Furthermore, the ICM neuron includes two coupled oscillators, and the connection weight coefficient matrix is set to W.

The competition neuron weight matrix LW is a 2-dimensional vector, and at the same time, only one element is 1, and the others are 0, and the competition function of the competition layer neurons uses a Gaussian function to output a 2-dimensional vector, and the one with the highest similarity probability The elements corresponding to one mode class are set to 1, and the others are 0, and the position where 1 appears indicates the recognized class of the input feature matrix, that is, normal mode or lesion mode.

Compared with prior art, the beneficial effect of the present invention is:

1. The lesion pattern recognition method in the endoscopic image of the present invention adopts the positioning of the suspected lesion area based on the human visual attention mechanism, which will greatly reduce the calculation amount of the artificial neural network in the endoscopic image, and further improve the accuracy of the lesion in the endoscopic image. Accuracy of pattern recognition.

2. The image processing method of the present invention utilizes the color information of the endoscopic image to the greatest extent in the color Luv space based on visual perception, and the color information is important information for the diagnosis of the lesion area, which improves the accuracy and specificity of the lesion area determination. Spend.

3. The pattern recognition method of the present invention does not distinguish the specific types of lesions, and classifies all areas into lesion patterns and normal patterns, and uses pulse-coupled neural networks, which will greatly improve the accuracy and practicability of lesion recognition in endoscopic images, Reduce clinician workload.

Description of drawings

Fig. 1 is a flow chart of the lesion identification method based on the pulse coupled neural network of the present invention.

Figure 2 is a structural diagram of a pulse-coupled neural network.

Figure 3 is a structural diagram of ICM neurons.

detailed description

Embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings.

Referring to Figures 1 to 3, a pulse-coupled neural network-based method for identifying lesions in endoscopic images includes the following steps:

a. Video framing

Using the detection video file of Given’s capsule endoscope detection system, input the endoscope detection video, and obtain a single endoscopic image in bitmap format by framing;

b. Image preprocessing

The bitmap image obtained in step a is smoothed according to the field of view parameters of the endoscope to obtain an endoscopic image with clear boundaries, and then filtered by a Butterworth high-pass filter and then by a median filter , remove the noise in the image area to be processed and retain the high frequency part of the image;

c. Color space conversion for visual perception

The bitmap image obtained in step b is a device-oriented RGB color space, which is converted to a visual perception-oriented Luv color space;

d. Localization of suspected lesions

Using the u and v components of the Luv color space image obtained in step c as input, calculate the color feature saliency map uv(c,s), use the L component as input to calculate the brightness feature saliency map L(c,s), and then use the Lapp The Lass transform algorithm and the virtual connection method are used to obtain the edge area of the salient content in the image, calculate the contour feature saliency map O(c,s) and the texture feature map T(c,s), and convert the obtained color feature saliency map uv (c, s), luminance feature saliency map L(c, s), contour feature saliency map O(c, s) and texture feature saliency map T(c, s) are respectively regularized and fused at multiple scales, Obtain the saliency map S of the image, and then use the etching algorithm to filter out the salient areas with small areas, and then arrange the saliency degrees in the order of the area size, that is, the suspected lesion area;

e. Construct eigenvectors

Taking the saliency map S obtained in step d as input, construct the pixel color feature vector V(uv) and brightness feature vector V(L) in the suspected lesion area, calculate and construct the area contour feature vector V(O) and the texture feature vector V (T);

f. Pattern recognition of lesion area

Using the feature vector constructed in step e as input, using a pulse-coupled neural network to perform pattern recognition to obtain the lesion pattern of the suspected area to be identified, that is, the normal pattern and the lesion pattern;

g. Regional transfer

Recognize according to the order of the size of the suspected lesion areas in the image. If there are other suspected lesion areas, repeat steps e and f for pattern recognition until the identification of all suspected lesion areas is completed;

h. Lesion image classification and extraction

Perform an OR operation on the mode results of all suspected lesion areas in the endoscopic image to obtain the classification mode of the endoscopic image, that is, the image normal mode and the image lesion mode, if the lesion mode marks the lesion area; i. Repeat steps b and c , d, e, f, g, h, until the endoscopic image recognition of the entire video file ends.

Further, in the step e, in the salient region of the endoscopic image saliency map S, construct the pixel color feature vector V'(uv) and the brightness feature vector V'(L), and then map to the high-dimensional space through the Sigmoid kernel function , use the principal component analysis (PCA) method to extract the core principal component features of the feature data, and obtain the dimensionality-reduced color feature vector V(uv) and brightness feature vector V(L). At the same time, calculate and construct the area contour feature in the area The vector V(O) texture feature vector V(T) is used to construct a feature matrix for pattern recognition of lesions.

Further, in the step f, the pulse-coupled neural network is composed of an input layer, a neuron layer of an intercortical model (ICM), and a competitive output layer;

The input layer consists of four input channels including color feature input, brightness feature input, outline feature input, and texture feature input. The ICM neuron layer uses four ICM neurons. The competition output layer consists of the competition neuron weight matrix LW and the competition function C composition;

The color feature input and brightness feature input channels of the input layer are connected to the No. 1 ICM neuron input of the ICM neuron layer, and the contour feature input and texture feature input channels are connected to the No. 2 ICM neuron input of the ICM neuron layer. No. 3 and No. 2 ICM neurons are respectively connected to the input of No. 3 and No. 4 ICM neurons, No. 3 and No. 4 ICM neurons are connected to the input of the competition neuron weight matrix LW of the competition output layer, and the weight matrix LW of the competition neuron The output is connected to the input of the competition function C of the competition layer neurons;

Still further, the ICM neuron includes two coupled oscillators, and the connection weight coefficient matrix is set to W.

The competition neuron weight matrix LW is a 2-dimensional vector, and at the same time, only one element is 1, and the others are 0, and the competition function of the competition layer neurons uses a Gaussian function to output a 2-dimensional vector, and the one with the highest similarity probability The elements corresponding to one mode class are set to 1, and the others are 0, and the position where 1 appears indicates the recognized class of the input feature matrix, that is, normal mode or lesion mode.

Finally, it should also be noted that what is listed above is only a specific embodiment of the present invention. Obviously, the present invention is not limited to the above embodiments, and many variations are possible. All deformations that can be directly derived or associated by those skilled in the art from the content disclosed in the present invention should be considered as the protection scope of the present invention.

Claims (5)

1. focus recognition methods in an endoscopic picture based on Pulse Coupled Neural Network, it is characterised in that: described Recognition methods comprises the steps:

A. video framing

Peeping detection video file input by interior, framing obtains the single width endoscopic picture of bitmap format；

B. Image semantic classification

By the bitmap images obtained by step a by the visual field parameter of endoscope by smooth for the edge black surround of image place Reason, obtains sharply marginated endoscopic picture, then uses high pass filter filters denoising, then use medium filtering Device is filtered strengthening, and removes the noise of pending image-region and retains image HFS；

C. towards the color space conversion of visually-perceptible

The bitmap images obtained in step b is device oriented RGB color, converts it to towards vision The Luv color space of perception；

D. suspected abnormality zone location

U, v component of the Luv color space image obtained using step c, as input, calculates color characteristic notable (c, s), (c s), then uses Laplce to figure uv significantly to scheme L using L * component as input calculating brightness Mapping algorithm and virtually connect method, obtains the border area of notable content in image, and calculating contour feature is significantly schemed (c, s) (c, s), obtained color characteristic is significantly schemed uv, and (c, s), brightness is special with the notable T of textural characteristics figure for O Levy and significantly scheme L (c, s), contour feature significantly schemes O, and (c s) significantly schemes T with textural characteristics (c, s) respectively multiple dimensioned Under carry out regularization computing and merge, obtain the saliency map S of image, then use etching algorithm to filter out face Long-pending less marking area, then according to order arrangement the significance degree, i.e. suspected abnormality of region area size Region；

E. structural feature vector

With the notable figure S obtained by step d for input, construct in suspected abnormality region pixel color feature to Amount V (uv) and brightness vector V (L), calculate and structure realm Outline Feature Vector V (O) texture feature vector V(T)；

F. focal area carries out pattern recognition

Using the characteristic vector constructed by step e as input, Pulse Coupled Neural Network is used to carry out pattern recognition, Obtain the focus pattern of suspicious region to be identified, i.e. normal mode and focus pattern；

G. zone-transfer

It is identified respectively according to the order of suspected abnormality area size in image, if also having other suspected abnormality Region, repetition step e, f carry out pattern recognition, until all suspected abnormality region recognition terminate；

H. lesion image classification is extracted

The model results in suspected abnormality regions all in endoscopic picture is carried out or computing, obtains this width endoscopic picture Classification mode, i.e. image normal mode and image focus pattern, if focus mode flag focal area； I. step b, c, d, e, f, g, h are repeated, until the endoscopic picture end of identification of whole video file.

2. focus recognition methods in endoscopic picture based on Pulse Coupled Neural Network as claimed in claim 1, its It is characterised by: in described step e, in endoscopic picture significantly schemes the marking area of S, constructs preliminary pixel face Color characteristic vector V ' (uv) and preliminary brightness vector V ' (L), be then mapped to height by Sigmoid kernel function Dimension space, uses the method for principal component analysis to extract the core principle component feature of characteristic, obtains the color of dimensionality reduction Characteristic vector V (uv) and brightness vector V (L), calculate and structure realm contour feature in region meanwhile Vector V (O) texture feature vector V (T), sets up eigenmatrix, carries out the pattern recognition of focus.

3. focus recognition methods in endoscopic picture based on Pulse Coupled Neural Network as claimed in claim 1 or 2, It is characterized in that: in described step f, Pulse Coupled Neural Network is by input layer, the cortex model neuron that intersects Layer and competition output layer composition；

Described input layer is inputted by color characteristic, brightness inputs, contour feature inputs, textural characteristics input Totally four input channels, intersect cortex model neuronal layers use four ICM neurons, competition output layer by Competition neurons weight matrix LW and competitive function C composition；

Color characteristic input, brightness input channel and No. 1 ICM nerve of neuronal layers of described input layer Unit's input connects, the input of described contour feature, textural characteristics input channel and No. 2 ICM nerves of neuronal layers Unit's input connects, and No. 1 and No. 2 ICM neurons input be connected with each other with No. 3 and No. 4 ICM neurons respectively, No. 3, No. 4 ICM neurons is connected with the competition neurons weight matrix LW input of competition output layer, compete The output of neuron weight matrix LW is connected with the input of competition layer neuron competitive function C.

4. focus recognition methods in endoscopic picture based on Pulse Coupled Neural Network as claimed in claim 3, its It is characterised by: described ICM neuron includes two coupled oscillators, connects weighting coefficient matrix and be set to W.

5. focus recognition methods in endoscopic picture based on Pulse Coupled Neural Network as claimed in claim 3, its It is characterised by: described competition neurons weight matrix LW is 2 n dimensional vector ns, and only one of which element is 1 simultaneously, Other is all 0, and, competition layer neuron competitive function C uses Gaussian function, export one 2 dimension to Amount, the element that wherein that pattern class of likelihood probability maximum is corresponding is arranged to 1, and other is all 0,1 institute The position occurred would indicate that the classification that input feature vector matrix is identified, i.e. normal mode or focus pattern.