CN117036792B - Method for identifying and early warning end state of salvaging ROV umbilical cable - Google Patents
Method for identifying and early warning end state of salvaging ROV umbilical cableInfo
- Publication number
- CN117036792B CN117036792B CN202310935781.2A CN202310935781A CN117036792B CN 117036792 B CN117036792 B CN 117036792B CN 202310935781 A CN202310935781 A CN 202310935781A CN 117036792 B CN117036792 B CN 117036792B
- Authority
- CN
- China
- Prior art keywords
- umbilical cable
- rov
- image
- umbilical
- end state
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
- G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
- G06V10/806—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of extracted features
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Image Analysis (AREA)
Abstract
The method for identifying and early warning the end state of the salvaging ROV umbilical cable comprises the steps of collecting images of the end of the umbilical cable in each state, carrying out rough segmentation on the images of the end of the umbilical cable by using a segmentation positioning model, carrying out fine classification on the images of the end state of the umbilical cable by using an umbilical cable end state identification model, carrying out classification identification and early warning on the images of the end of the salvaging ROV umbilical cable read in real time, and firstly positioning the end region of the umbilical cable by using a recognition algorithm from rough to fine detected after segmentation.
Description
Technical Field
The invention relates to the field of salvaging underwater robots, in particular to a method for identifying and early warning the tail end state of an umbilical cable of a salvaging ROV, which can be applied to the field of operation-type and observation-level ROVs.
Background
In recent years, salvage underwater robots (Remote Operated Vehicle, ROV) have been widely used for salvage recovery of submerged objects in deep sea/offshore/test sites and the like. Salvage ROVs are typically comprised of an underwater ROV body, an umbilical that enables underwater/surface power and data transmission, and a surface monitoring system, which is an important component of the overall system.
However, the umbilical cable is extremely easy to fail due to the complexity and unknowing of the underwater environment in the operation process, particularly, the tail end position is more easy to generate various types of faults in the salvage process, the ROV is lost, the water surface power distribution is abnormal and other problems can be caused when the faults are serious, the life safety and the property safety are greatly threatened, and therefore necessary measures are needed to monitor the tail end position of the umbilical cable:
(1) The umbilical cable image shot by the ROV is observed in real time through the underwater camera with the ROV, and the state of the umbilical cable image is judged by human, however, the measure has low automation level, consumes a great deal of manpower, reduces the salvage operation efficiency, and most of salvage ROVs adopt the mode at present.
(2) In the process of salvage operation, an independent observation level ROV is utilized to observe and monitor the umbilical cable, but the measures need to operate by the independent ROV, so that the cost is increased, and the difficulty of cooperative operation of a plurality of ROVs is increased.
(3) The umbilical cable is monitored, the strain information of the umbilical cable is monitored in real time through designing a strain detection system and the like, and then the umbilical cable state is analyzed, but the cost of the umbilical cable is higher, and the system integration is more complex.
Disclosure of Invention
Based on the method, the invention provides a method for identifying and early warning the state of the tail end of the salvaging ROV umbilical cable, which can automatically identify the states of normal, stretching, bending, interference, twisting, winding, knotting, breakage, fracture and the like of the tail end of the umbilical cable, and realize automatic early warning according to the identification result, thereby solving the defects of manual observation and discrimination, and simultaneously, the system only needs to newly add an operation terminal on the original ROV, thereby improving the feasibility.
In order to achieve the aim of the application, the application adopts the following technical scheme:
The invention discloses a method for identifying and early warning the end state of an ROV salvaged umbilical cable, which comprises the following steps:
(one), collect the end image of the umbilical cable in each state
(A) Shooting video images of the end of the ROV umbilical cable by using a subsurface camera, storing the video images, and turning over, rotating and changing the color of each state image of the acquired video images so as to increase the number of the acquired video images;
Secondly, roughly dividing the video image of the tail end of the umbilical cable by using a division positioning model
(B) Pre-segmenting the image by using a SAM model to obtain an initial segmentation result irrelevant to the category, and then manually optimizing and labeling the segmentation result, wherein the labels are binarization graphs of umbilical cable foreground and other backgrounds to obtain an image to be trained;
(c) Training the segmentation positioning model by using the images
(III) finely classifying the images by using the umbilical cord end-state identification model
(D) Marking the images in the step (b) by using a post-processing module to obtain a sub-image block at the tail end of the umbilical cable, manually marking the sub-image block as nine types of normal, stretching, bending, interference, twisting, winding, knotting, breakage and fracture in sequence, and training an umbilical cable tail end state identification model as input data to obtain 1-9 classification results;
fourth, the real-time read salvaged ROV umbilical cable end images are classified, identified and early-warned
The method comprises the steps of reading a real-time umbilical cable end image, sequentially classifying the real-time umbilical cable end image by using the trained segmentation positioning model, the post-processing module and the trained umbilical cable end state identification model, and according to the classification result, performing a first-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 2-3, performing a second-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 4-5, performing a third-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 4-6, and performing a fourth-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 7-9;
Training the segmentation positioning model by using an image to be trained, and when the loss function of the segmentation positioning model converges, completing the training of the segmentation positioning model, wherein the loss function L is calculated by the following formula (1):
Wherein N represents the total number of predicted pixels, C represents the number of classes, C=0 represents the underwater background, C=1 represents the umbilical cable, 1 {. Cndot } is an indication function, 1 is a condition that is satisfied, 0 is not satisfied, y i represents the real class label of the ith pixel, y i∈{0,1,...,C-1},pij represents the probability that the ith pixel is predicted as class j, t is a threshold value, 0.7 is taken, as p ij is smaller, the pixel i is harder to classify, the pixel point with p ij smaller than 0.7 is regarded as a difficult case, so that the model pays more attention to learning of the difficult case, and θ j represents class weight of the jth class and is represented as formula (2):
Wherein hist j represents the proportion of the number of the pixels of the j-th class to the total number of the pixels of all classes, the smaller the proportion is, the larger the weight theta j is, and gamma is used for controlling the weight range, when gamma=1.10, and theta j is epsilon [1,10.5].
The invention discloses a method for identifying and early warning the end state of an ROV salvaged umbilical cable, wherein the post-processing module processes the images and comprises the following steps:
(I) Binarizing the segmented result to obtain a foreground image and a background image;
(II) performing morphological closing operation on the processed foreground image, and communicating umbilical cable fracture subareas caused by segmentation errors;
(III) carrying out contour searching on the foreground image after the closing operation, calculating the area, perimeter and image moment characteristics of each contour, and removing small-area useless contours formed by noise;
(IV) finding out the horizontal bounding rectangle of each contour;
Finding out the horizontal external rectangle of each outline, combining all the external rectangles in the maximum range, and expanding the combined rectangles by 1.5-2 times to obtain a sub-image block for retaining the umbilical cable and surrounding environment information.
The method for identifying and early warning the end state of the salvaged ROV umbilical cable comprises the steps that the umbilical cable is normally fixed to an ROV frame through a bearing head, the ROV is drawn to be too fast submerged, the umbilical cable is bent to be too long, the umbilical cable is interfered with a hanging cable or the ROV frame, the twisting state is expressed as twisting force generated by water flow of the umbilical cable end, the umbilical cable is twisted with the ROV frame, a salvaged object or an underwater object, the knotting is expressed as knotting of the umbilical cable and the ROV frame, the breakage is expressed as surface breakage of the umbilical cable, and the breakage is expressed as breakage of the umbilical cable.
The invention discloses a method for identifying and early warning the end state of an ROV salvaged umbilical cable, wherein a segmentation positioning model comprises a local feature extraction module, a context feature extraction module, a feature fusion module and a segmentation module which are sequentially connected, wherein the local feature extraction module consists of at least three residual units which are connected in series, the output of the local feature extraction module and the output of the context feature extraction module serve as the input of the feature fusion module, multi-scale fusion is realized step by step, and the segmentation module inputs fusion features output by the feature fusion module into a preset FCN-based semantic segmentation model to finish segmentation of the umbilical cable prospect and other backgrounds.
The invention discloses a method for identifying and early warning the end state of an ROV salvaged umbilical cable, wherein a first residual error unit changes a first convolution kernel with an original step length of 2 and a width of 7 in an initial stage into two groups of second convolution kernels with step lengths of 1 and 3 and a group of third convolution kernels with step lengths of 2 and 3, and the method is used for expanding an image characteristic receptive field and reducing the quantity of parameters.
The invention discloses a method for identifying and early warning a salvaging ROV umbilical end state, which comprises a context feature extraction module, wherein the context feature extraction module comprises a global feature extraction unit and a cross feature extraction unit, the global feature extraction unit carries out global pooling processing on an input feature image, carries out fourth convolution kernel processing with the width of 1, upsamples a convolution result to the size of the input feature image to obtain a global feature image for obtaining maximum receptive field information of an image, the cross feature extraction unit carries out transverse convolution and vertical convolution on the input feature image to obtain a cross enhancement feature image for enhancing umbilical edge features, and then carries out pixel-by-pixel accumulation on the global feature image and the cross enhancement feature image for enhancing the context features.
The invention discloses a method for identifying and early warning the end state of an ROV salvaged umbilical cable, wherein:
The invention discloses a method for identifying and early warning the end state of an umbilical cable of a salvaging ROV, wherein the end state identification model of the umbilical cable is a MobileNetV multi-category image classification model.
The invention discloses a method for identifying and early warning the end state of an umbilical cable of a salvaging ROV, wherein when a first-stage early warning is carried out, an operator carries out water surface display warning information or a water surface umbilical cable winch rapidly reel and unreel the cable, when a second-stage early warning is carried out, the operator carries out water surface display warning information or ROV pause operation and immediately fine-tunes the posture and the position, when a third-stage early warning is carried out, the operator carries out sound-light warning or ROV pause operation and immediately retrieves, and when a fourth-stage early warning is carried out, the operator carries out underwater high-voltage power-off or stops salvaging operation.
The invention designs a method for identifying and early warning the end state of an umbilical cable of a salvaging ROV aiming at the problems of stretching, bending, interference, twisting, winding, knotting, breakage, fracture and the like possibly occurring in the operation process of the salvaging ROV. Firstly, collecting video images of an ROV umbilical cable monitoring underwater camera, marking an umbilical cable area and an image state in the images, training an umbilical cable end segmentation positioning model by using a marked training set for realizing pixel-level identification of the umbilical cable end, positioning the image area, and training an umbilical cable end state identification model by using images of different marked states for identifying the segmented area and confirming a specific state. And secondly, threat grading is carried out according to the identified end state of the umbilical cable, and corresponding early warning measures are executed according to the grade, so that accidents are effectively prevented. Finally, the method can be directly applied to the existing ROV salvaging system, and only the data of the umbilical cable monitoring underwater camera needs to be transmitted to the newly-added operation terminal.
Compared with the prior art, the method has the beneficial effects that:
(1) According to the invention, an artificial intelligence technology is used, automatic identification and early warning of the tail end state of the umbilical cable can be realized without participation of operators, so that the artificial discrimination cost in the salvage operation process is greatly reduced, and the intelligence level is improved;
(2) The invention designs a recognition algorithm of 'from thick to thin' which is detected after segmentation, positions the tail end region of the umbilical cable firstly, and then realizes fine recognition, compared with the method for recognizing the whole image directly, the method can reduce the influence of the underwater complex environment, improve the recognition accuracy, and the segmentation algorithm can better segment the tail end position of the umbilical cable because the underwater image can present a homogeneous region with a larger area;
(3) The early warning module provided by the invention divides the first early warning level, the second early warning level, the third early warning level and the fourth early warning level according to the threat degrees of stretching, bending, interference, twisting, winding, knotting, breakage and fracture, and designs different early warning measures for each level, so that the safety of the salvaging operation is effectively ensured.
(4) According to the method, only one operation terminal is needed to be added in the existing ROV salvaging system, and an expensive GPU server is not needed, so that the existing ROV salvaging system is convenient to integrate in a matched mode, and the application cost is greatly reduced.
Drawings
FIG. 1 is a flow chart of a method for identifying and pre-warning the end state of an ROV umbilical cable
FIG. 2 is a flow chart of a post-processing module according to the present invention
FIG. 3 is a structural view of an umbilical cable end division positioning model according to the present invention;
fig. 4 is a diagram showing the structure of the initial stage of the improved local feature extraction module according to the present invention.
Detailed Description
The objects, technical solutions and advantages of the present application will become more apparent by the following detailed description of the present application with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the application. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present application. Moreover, it should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
As shown in FIG. 1, the steps (I) to (III) are training model stages, and the step (IV) is reasoning stage, and the method for identifying and early warning the end state of the salvaged ROV umbilical cable of the invention comprises the following steps:
(one), collect the end image of the umbilical cable in each state
(A) Shooting video images of the tail end of the ROV umbilical cable by using a submerged camera, storing the video images, turning over, rotating and changing the color of each state image of the acquired video images to increase the number of the acquired video images, and unifying the acquired images to 1920×1080;
Secondly, roughly dividing the video image of the tail end of the umbilical cable by using a division positioning model
(B) Pre-segmenting the image by using a SAM model to obtain an initial segmentation result irrelevant to the category, and then manually optimizing and labeling the segmentation result, wherein the labels are binarization graphs of umbilical cable foreground and other backgrounds to obtain an image to be trained;
(c) Training the segmentation positioning model by using the images
(III) finely classifying the images by using the umbilical cord end-state identification model
(D) Processing the image in the step (b) by using an image cutting tool such as a post-processing module to obtain sub-image blocks, unifying the sizes of the sub-image blocks into 224 multiplied by 224, giving state labels to the sub-image blocks such as nine types of normal, stretching, bending, interference, twisting, winding, knotting, breakage and fracture, and training an umbilical cable end state identification model which is MobileNetV < 2 > multi-category image classification model as input data;
the normal appearance is that the umbilical cable is normally fixed to the ROV frame through the bearing head, and the score is 1;
stretching is expressed as the ROV submerging too fast umbilical cable being stressed and stretched, and the score is 2;
bending is shown as overlong umbilical cable laying, and the score is 3;
The interference is expressed by the fact that the umbilical cable is interfered with the suspension cable or the ROV frame, and the score is 4;
the twisted state is represented by a twisting force of the umbilical cord end due to water flow, and is classified as 5;
the winding is shown by the umbilical cable being wound with the ROV frame, the salvage object or the underwater object, and the score is 6;
the knot was expressed as an umbilical cord tied to the ROV frame, scoring 7;
breakage was expressed as umbilical surface breakage, with a score of 8;
the break was expressed as an umbilical break, score 9;
fourth, the real-time read salvaged ROV umbilical cable end images are classified, identified and early-warned
The method comprises the steps of reading a real-time umbilical cable end image, sequentially classifying the real-time umbilical cable end image by using the trained segmentation positioning model, the post-processing module and the trained umbilical cable end state identification model, and according to the classification result, performing a first-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 2-3, performing a second-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 4-5, performing a third-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 6-7, and performing a fourth-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 8-9;
when the first-stage early warning is carried out, an operator carries out water surface display warning information or a water surface umbilical winch rapidly reel and unreel the cable, when the second-stage early warning is carried out, the operator carries out water surface display warning information or ROV pause operation and fine-adjusts the posture and the position immediately, when the third-stage early warning is carried out, the operator carries out sound and light warning or ROV pause operation and recovery immediately, and when the fourth-stage early warning is carried out, the operator carries out underwater high-voltage power-off or stops salvaging operation.
As shown in fig. 2, in the step (iii) and the step (iv) of the method, the post-processing module processes the image, and includes the following steps:
(I) Binarizing the segmented result to obtain a foreground image and a background image;
(II) performing morphological closing operation on the processed foreground image, and communicating umbilical cable fracture subareas caused by segmentation errors;
(III) carrying out contour searching on the foreground image after the closing operation, calculating the area, perimeter and image moment characteristics of each contour, and removing small-area useless contours formed by noise;
(IV) finding out the horizontal bounding rectangle of each contour;
Finding out the horizontal external rectangle of each outline, combining all the external rectangles in the maximum range, and expanding the combined rectangles by 1.5-2 times to obtain a sub-image block for retaining the umbilical cable and surrounding environment information.
As shown in fig. 3, the segmentation positioning model comprises a local feature extraction module 101, a context feature extraction module 102, a feature fusion module 103 and a segmentation module 104 which are sequentially connected, wherein the local feature extraction module consists of four residual units ResNet which are connected in series, the outputs of the local feature extraction module and the context feature extraction module serve as the inputs of the feature fusion module, multi-scale fusion is realized step by step, and the segmentation module completes segmentation of the umbilical cable prospect and other backgrounds according to the fusion features output by the feature fusion module.
As shown in fig. 4, the local feature extraction module 101 modifies the original step size of 7x7 convolution of 2 in the initial stage into two sets of step size of 3x3 convolutions of 1 and one set of step size of 3x3 convolutions of 2, and then extracts 4 feature maps [ C1, C2, C3, C4] of 1/32, 1/16, 1/8, 1/4 of the resolution of the original image.
The context feature extraction module of the segmentation positioning model comprises a global feature extraction unit and a cross feature extraction unit, wherein the global feature extraction unit carries out global pooling processing on an input feature image, carries out fourth convolution kernel processing with the width of 1 on the input feature image, upsamples a convolution result to the size of the input feature image to obtain a global feature image for obtaining maximum feeling field information of an image, carries out transverse convolution and vertical convolution on the input feature image to obtain a cross enhancement feature image for enhancing the edge feature of an umbilical cable, and then carries out pixel-by-pixel accumulation on the global feature image and the cross enhancement feature image for enhancing the context feature.
The contextual feature extraction module 102 includes a global feature extraction unit and a cross feature extraction unit, and the specific extraction steps are as follows:
a1, carrying out two-dimensional global pooling on an input feature map C1 to enable the size of the output feature map to be changed into 1x1 pixels so as to obtain maximum image receptive field information;
a2, carrying out 1x1 convolution on the output feature map, including convolution, batch normalization and activation layers, so as to obtain a global feature map;
a3, up-sampling the global feature map to the size of the input feature map;
A4, respectively carrying out transverse convolution and vertical convolution on the input characteristic diagram C1 to obtain an umbilical cable cross enhancement characteristic diagram;
A5, accumulating the up-sampled global feature map and the cross enhancement feature map pixel by pixel to obtain a context feature P1.
Further, in the step A4, the lateral convolution and the vertical convolution are 1xn, mx1, respectively, for enhancing the edge feature of the umbilical in each direction, in this embodiment, m=n=7.
Specifically, the feature fusion module 103 is composed of a lateral connection unit and an up-sampling unit, up-samples the feature map [ P1, C2, C3, C4] to the size of the C4 feature map, then performs a connection Cat operation on the feature map, and finally completes feature fusion by using 3x3 convolution to obtain a feature map P.
Specifically, the segmentation module 104 reduces the number of channels of the feature map P to 2 using a3×3 convolution, and then completes the class prediction using Softmax.
Training the segmentation positioning model by using an image to be trained, and when the loss function of the segmentation positioning model converges, completing the training of the segmentation positioning model, wherein the loss function L is calculated by the following formula (1):
Wherein N represents the total number of predicted pixels, C represents the number of classes, C=0 represents the underwater background, C=1 represents the umbilical cable, 1 {. Cndot } is an indication function, 1 is a condition that is satisfied, 0 is not satisfied, y i represents the real class label of the ith pixel, y i∈{0,1,...,C-1},pij represents the probability that the ith pixel is predicted as class j, t is a threshold value, 0.7 is taken, as p ij is smaller, the pixel i is harder to classify, the pixel point with p ij smaller than 0.7 is regarded as a difficult case, so that the model pays more attention to learning of the difficult case, and θ j represents class weight of the jth class and is represented as formula (2):
Wherein hist j represents the proportion of the number of the pixels of the j-th class to the total number of the pixels of all classes, the smaller the proportion is, the larger the weight theta j is, and gamma is used for controlling the weight range, when gamma=1.10, and theta j is epsilon [1,10.5].
Training a segmentation model by using a random gradient descent optimization algorithm and the loss function, specifically, adopting an Adam optimizer and combining a counter propagation training model, wherein the initial learning rate is 4e-4, the batch size is 8, the weight attenuation coefficient is 0.0001, and the maximum iteration number is 8000. And gradually reducing the learning rate by using a cosine learning rate attenuation strategy, so that the model is better converged.
The foregoing has outlined rather broadly the more detailed description of the invention in order that the detailed description of the principles of the invention may be better understood. It should be noted that it will be apparent to those skilled in the art that the present invention may be modified and adapted without departing from the principles of the present invention, and such modifications and adaptations are intended to be within the scope of the appended claims.
Claims (8)
1. A method for identifying and early warning the end state of an ROV umbilical cable is characterized by comprising the following steps:
(one), collect the end image of the umbilical cable in each state
(A) Shooting video images of the end of the ROV umbilical cable by using a subsurface camera, storing the video images, and turning over, rotating and changing the color of each state image of the acquired video images so as to increase the number of the acquired video images;
Secondly, roughly dividing the video image of the tail end of the umbilical cable by using a division positioning model
(B) Pre-segmenting the image by using a SAM model to obtain an initial segmentation result irrelevant to the category, and then manually optimizing and labeling the segmentation result, wherein the labels are binarization graphs of umbilical cable foreground and other backgrounds to obtain an image to be trained;
(c) Training the segmentation positioning model by using the images
(III) finely classifying the images by using the umbilical cord end-state identification model
(D) Marking the images in the step (b) by using a post-processing module to obtain a sub-image block at the tail end of the umbilical cable, manually marking the sub-image block as nine types of normal, stretching, bending, interference, twisting, winding, knotting, breakage and fracture in sequence, and training an umbilical cable tail end state identification model as input data to obtain 1-9 classification results;
fourth, the real-time read salvaged ROV umbilical cable end images are classified, identified and early-warned
The method comprises the steps of reading a real-time umbilical cable end image, sequentially classifying the real-time umbilical cable end image by using the trained segmentation positioning model, the post-processing module and the trained umbilical cable end state identification model, and according to the classification result, performing a first-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 2-3, performing a second-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 4-5, performing a third-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 6-7, and performing a fourth-stage alarm on the umbilical cable end when the output of the umbilical cable end state identification model is 8-9;
The loss function L used to train the segmentation localization model is calculated with the following equation (1):
Wherein N represents the total number of predicted pixels, C represents the number of classes, C=0 represents the underwater background, C=1 represents the umbilical cable, 1 {. Cndot } is an indication function, 1 is a condition that is satisfied, 0 is not satisfied, y i represents the real class label of the ith pixel, y i∈{0,1,...,C-1},pij represents the probability that the ith pixel is predicted as class j, t is a threshold value, 0.7 is taken, as p ij is smaller, the pixel i is harder to classify, the pixel point with p ij smaller than 0.7 is regarded as a difficult case, so that the model pays more attention to learning of the difficult case, and θ j represents class weight of the jth class and is represented as formula (2):
Wherein hist j represents the proportion of the number of the pixels of the j-th class to the total number of the pixels of all classes, the smaller the proportion is, the larger the weight theta j is, and gamma is used for controlling the weight range, when gamma=1.10, and theta j is epsilon [1,10.5].
2. The method for identifying and pre-warning the end state of an ROV umbilical of claim 1, wherein the post-processing module processes the image comprising the steps of:
(I) Binarizing the segmented result to obtain a foreground image and a background image;
(II) performing morphological closing operation on the processed foreground image, and communicating umbilical cable fracture subareas caused by segmentation errors;
(III) carrying out contour searching on the foreground image after the closing operation, calculating the area, perimeter and image moment characteristics of each contour, and removing small-area useless contours formed by noise;
(IV) finding out the horizontal bounding rectangle of each contour;
Finding out the horizontal external rectangle of each outline, combining all the external rectangles in the maximum range, and expanding the combined rectangles by 1.5-2 times to obtain a sub-image block for retaining the umbilical cable and surrounding environment information.
3. The method for identifying and early warning the end state of the salvaging ROV umbilical cable according to claim 2, wherein the normal state is that the umbilical cable is normally fixed to the ROV frame through a bearing head, the stretching is that the ROV submerges too quickly, the bending is that the umbilical cable is excessively long, the interference is that the umbilical cable interferes with a suspension cable or the ROV frame, the distortion state is that the umbilical cable end generates a distortion force due to water flow, the winding is that the umbilical cable is wound with the ROV frame, the salvaging object or an underwater object, the knotting is that the umbilical cable is knotted with the ROV frame, the breakage is that the umbilical cable surface is broken, and the breakage is that the umbilical cable is broken.
4. The method for identifying and early warning the end state of the ROV umbilical cable according to claim 3, wherein the segmentation positioning model comprises a local feature extraction module, a context feature extraction module, a feature fusion module and a segmentation module which are sequentially connected, wherein the local feature extraction module consists of at least three residual units which are connected in series, the outputs of the local feature extraction module and the context feature extraction module serve as the inputs of the feature fusion module, multi-scale fusion is realized step by step, and the segmentation module inputs fusion features output by the feature fusion module into a preset semantic segmentation model based on FCN, so that the segmentation of the umbilical cable prospect and other backgrounds is completed.
5. The method for identifying and pre-warning the end state of an ROV salvaged umbilical cable according to claim 4, wherein the first residual error unit changes a first convolution kernel with an original step length of 2 and a width of 7 in an initial stage into two groups of second convolution kernels with step lengths of 1 and 3 and a group of third convolution kernels with step lengths of 2 and 3, and the first residual error unit is used for expanding an image characteristic receptive field and reducing the number of parameters.
6. The method for identifying and early warning the end state of the salvaged ROV umbilical cable according to claim 5, wherein the contextual feature extraction module comprises a global feature extraction unit and a cross feature extraction unit, the global feature extraction unit carries out global pooling processing on an input feature map and carries out fourth convolution kernel processing with the width of 1 on the input feature map, the convolution result is up-sampled to the size of the input feature map to obtain a global feature map for obtaining maximum receptive field information of an image, the cross feature extraction unit carries out transverse convolution and vertical convolution on the input feature map to obtain a cross enhancement feature map for enhancing edge features of the umbilical cable, and then pixel-by-pixel accumulation is carried out on the global feature map and the cross enhancement feature map for enhancing contextual features.
7. The method for identifying and pre-warning the end state of an umbilical cable for salvaging an ROV according to claim 1, wherein the end state identification model of the umbilical cable is MobileNetV2 multi-category image classification model.
8. The method for identifying and early warning the end state of an umbilical cable for salvaging an ROV according to claim 7, wherein when the first-stage early warning is carried out, an operator carries out water surface display warning information or a water surface umbilical winch rapidly reel and reel the cable, when the second-stage early warning is carried out, the operator carries out water surface display warning information or ROV pause operation and immediately fine-adjusts the posture and the position, when the third-stage early warning is carried out, the operator carries out acousto-optic warning or ROV pause operation and immediately retrieves, and when the fourth-stage early warning is carried out, the operator carries out underwater high-voltage power-off or stops salvaging operation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310935781.2A CN117036792B (en) | 2023-07-28 | 2023-07-28 | Method for identifying and early warning end state of salvaging ROV umbilical cable |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310935781.2A CN117036792B (en) | 2023-07-28 | 2023-07-28 | Method for identifying and early warning end state of salvaging ROV umbilical cable |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117036792A CN117036792A (en) | 2023-11-10 |
| CN117036792B true CN117036792B (en) | 2025-08-19 |
Family
ID=88634599
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310935781.2A Active CN117036792B (en) | 2023-07-28 | 2023-07-28 | Method for identifying and early warning end state of salvaging ROV umbilical cable |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117036792B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN207835648U (en) * | 2017-11-09 | 2018-09-07 | 美钻石油钻采系统(上海)有限公司 | A kind of underwater robot light vision monitoring apparatus |
| CN116342575A (en) * | 2023-04-11 | 2023-06-27 | 盐城国睿信科技有限公司 | Intelligent recognition system and method based on image processing |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11684799B2 (en) * | 2021-08-28 | 2023-06-27 | Cutera, Inc. | Image guided laser therapy |
| CN114562950B (en) * | 2022-02-28 | 2023-08-15 | 中国船舶科学研究中心 | Umbilical cable-shaped monitoring system and device for underwater collaborative operation |
-
2023
- 2023-07-28 CN CN202310935781.2A patent/CN117036792B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN207835648U (en) * | 2017-11-09 | 2018-09-07 | 美钻石油钻采系统(上海)有限公司 | A kind of underwater robot light vision monitoring apparatus |
| CN116342575A (en) * | 2023-04-11 | 2023-06-27 | 盐城国睿信科技有限公司 | Intelligent recognition system and method based on image processing |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117036792A (en) | 2023-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111339882B (en) | Power transmission line hidden danger detection method based on example segmentation | |
| CN108961238A (en) | Display screen quality determining method, device, electronic equipment and storage medium | |
| CN109472200B (en) | An intelligent sea surface garbage detection method, system and storage medium | |
| CN108444447A (en) | A kind of fishing net in underwater obstacle avoidance system autonomous detection method in real time | |
| Zhao et al. | Research on detection method for the leakage of underwater pipeline by YOLOv3 | |
| CN116052082A (en) | A method and device for anomaly detection of power distribution substation based on deep learning algorithm | |
| CN117253192A (en) | Intelligent system and method for silkworm breeding | |
| CN113962973A (en) | An intelligent inspection system and method of unmanned aerial vehicle for transmission line based on satellite technology | |
| Petraglia et al. | Pipeline tracking and event classification for an automatic inspection vision system | |
| CN111476289A (en) | Fish shoal identification method, device, equipment and storage medium based on feature library | |
| KR20220143119A (en) | Automatic identification of training data candidates for cognitive systems | |
| CN113850166B (en) | A ship image recognition method and system based on convolutional neural network | |
| CN110969213A (en) | Ship detection method and device based on fast RCNN and electronic equipment | |
| CN119672613A (en) | A surveillance video information intelligent processing system based on cloud computing | |
| CN117788892A (en) | Vision-based AUV propeller winding diagnosis and assessment method | |
| CN117456163A (en) | Ship target detection method, system and storage medium | |
| CN117036792B (en) | Method for identifying and early warning end state of salvaging ROV umbilical cable | |
| CN118197027B (en) | Unmanned aerial vehicle scheduling checking method based on pipeline early warning and related device | |
| CN118609164A (en) | Construction area personnel detection method, device, storage medium and program product | |
| CN115511876B (en) | High-speed railway contact network bird thorn prevention identification method and system based on relative position perception | |
| CN118262258A (en) | A method and system for detecting differences in ground environment images | |
| CN113139077B (en) | Method, device, terminal and storage medium for identifying ship identity | |
| CN112926383B (en) | A Target Automatic Recognition System Based on Underwater Laser Image | |
| Zhao et al. | Subsea Pipeline Inspection Based on Contrast Enhancement Module | |
| Gupta et al. | Object Detection in Low Light using YOLOv11 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |