CN113034439B - High-speed railway sound barrier defect detection method and device - Google Patents
High-speed railway sound barrier defect detection method and device Download PDFInfo
- Publication number
- CN113034439B CN113034439B CN202110234308.2A CN202110234308A CN113034439B CN 113034439 B CN113034439 B CN 113034439B CN 202110234308 A CN202110234308 A CN 202110234308A CN 113034439 B CN113034439 B CN 113034439B
- Authority
- CN
- China
- Prior art keywords
- speed railway
- sound barrier
- image
- vehicle body
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C11/00—Photogrammetry or videogrammetry, e.g. stereogrammetry; Photographic surveying
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
- G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
- G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
- G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
- G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
- G01C21/18—Stabilised platforms, e.g. by gyroscope
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Automation & Control Theory (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
本发明提供一种高速铁路声屏障缺损检测方法及装置，方法包括：通过综合巡检车挂载数码摄像机和惯性组件，拍摄高速铁路运行环境视频数据和车体姿态数字信号模拟数据；基于多源数据配准的车体姿态补偿方法，对采集得到的高速铁路运行环境视频数据和车体姿态数字信号模拟数据进行配准；基于像素清晰度评价的最优拼接模型，对高速铁路运行环境视频进行全景图拼接；采集得到的正常声屏障图像，训练基于无监督学习的高速铁路声屏障异常检测网络，对拼接的全景图进行声屏障缺损检测。本发明实现了无监督条件下车载动态检测高速铁路声屏障缺损的目的，在一定程度上保障了高速铁路运行安全，具有较好的实用价值。
The invention provides a method and device for detecting sound barrier defects of a high-speed railway. The method includes: mounting a digital camera and an inertial component on a comprehensive inspection vehicle, and shooting high-speed railway operating environment video data and vehicle body attitude digital signal simulation data; The vehicle body attitude compensation method based on data registration registers the collected high-speed railway operating environment video data and the vehicle body attitude digital signal simulation data; based on the optimal splicing model based on pixel definition evaluation, the high-speed railway operating environment video is analyzed. Panorama image stitching: The acquired normal sound barrier images are trained to detect the abnormality of the high-speed railway sound barrier based on unsupervised learning, and the sound barrier defect detection is performed on the stitched panorama images. The invention realizes the purpose of vehicle-mounted dynamic detection of the sound barrier defect of the high-speed railway under unsupervised conditions, guarantees the operation safety of the high-speed railway to a certain extent, and has good practical value.
Description
Technical Field
The invention relates to the technical field of high-speed railway safety detection, in particular to a method and a device for detecting high-speed railway sound barrier defect.
Background
The high-speed railway operation environment safety guarantee technology is one of key technologies for high-speed railway construction planning. The whole-course closed high-speed railway operation environment provides safety guarantee for the operation of the high-speed train. The high-speed railway operation environment facilities comprise guard rails, sound barriers, ballastless track beds, track parts, signal equipment, power supply systems and the like, wherein the occurrence of abnormity of the guard rails and the sound barriers undoubtedly has direct and vital influence on the driving safety and even endangers the life safety of passengers. In the past, the sound barrier defect detection of the high-speed railway mainly adopts video data shot by fixed camera equipment installed in key sections along the high-speed railway for detection. However, the method is limited by the collection visual field, and the whole situation along the line cannot be controlled in the method, so that the method is an effective method for shooting the video data of the front line environment, guard rails and sound barriers at two sides along the line by using the vehicle-mounted camera equipment installed on the high-speed comprehensive detection vehicle and then detecting the defect state. At present, all countries in the world are equipped with a high-speed railway operation environment video detection system. Acquiring a video image through video acquisition equipment arranged on a train, superimposing line information such as a line name, mileage, speed and the like on the video image in real time, and storing the line information; the detection of the running environment of the high-speed railway is completed by manually analyzing the video data, and various abnormal conditions affecting the running safety of the train are timely found. However, the analysis of video data still relies on manual observation to determine abnormal conditions of the line environment, including sound barrier defects, equipment intrusion, intrusion of foreign objects into the line section, and so on.
In recent years, the technology based on image processing and computer vision intelligent identification has been widely applied to the detection of the appearance state abnormity of the high-speed railway facilities, the acquisition and storage of track images are realized, the subsequent manual browsing is convenient, and the automatic analysis capability is not available. With the development of image processing, pattern recognition and computer vision technologies, various intelligent analysis systems for automatic detection of abnormal states of rail facilities are coming out, and detection models based on artificial design features and machine learning have been applied to various aspects of rail detection. However, the method is limited by the insufficient feature representation capability of the artificial feature modeling method for the image, and the existing methods have the problems of insufficient generalization for different lines, insufficient detection rate in an open environment and the like. In response to these problems, researchers at home and abroad have begun to attempt to perform intelligent analysis of high-speed railway scene images using deep learning neural networks. Although the image-based intelligent detection technology has achieved a breakthrough result, the video-based intelligent detection technology has a wider demand in the fields of automatic driving, video monitoring in complex scenes, and the like. Video detection has more difficulties and higher requirements than image detection. Target objects in the video may have the problems of blurring, blocking, shape variation diversity, illumination variation diversity and the like, and a good detection result cannot be obtained only by using an image detection technology. In addition, since a video contains a large number of video frames, the direct use of an image-based detection method may cause a huge amount of calculation.
The traditional video intelligent detection method generally uses a manual feature design mode to construct the features of a target object, and then carries out subsequent operations such as detection, identification and the like on a video in a template matching mode. However, when the target object is blocked, the illumination changes and the form changes, the manually designed features cannot be adapted in real time. In addition, the manually designed features can only express partial characteristics of the target object, are easily interfered by the outside, and the feature model is more and more difficult to construct along with the gradual complexity of the detection task requirement. These limitations result in the conventional video target detection method failing to meet the actual detection task requirements.
At present, the video intelligent detection technology based on deep learning has made remarkable research and application progress, but when the technology is applied to the task of detecting sound barrier defect of high-speed railway, the following 3 defects still exist:
1) the acquired video data volume of the high-speed railway running environment is too large, and transmission and storage are not utilized;
2) the video data processing time is long, and the requirement on the detection efficiency cannot be met;
3) the high-speed railway sound barrier defect sample image is scarce, which is not beneficial to training the supervised deep convolution neural network.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method and a device for detecting the defect of a sound barrier of a high-speed railway, which specifically comprise the following technical schemes:
in a first aspect, the invention provides a method for detecting a defect of a sound barrier of a high-speed railway, which comprises the following steps:
the digital camera and the inertia component are mounted on the comprehensive inspection vehicle, and video data of the running environment of the high-speed railway and analog data of a vehicle body posture digital signal are shot;
the vehicle body attitude compensation method based on multi-source data registration is used for registering the acquired video data of the high-speed railway running environment and the vehicle body attitude digital signal analog data;
carrying out panorama stitching on the high-speed railway running environment video based on the optimal stitching model of the pixel definition evaluation;
and acquiring the obtained normal sound barrier image, training a high-speed railway sound barrier abnormity detection network based on unsupervised learning, and carrying out sound barrier defect detection on the spliced panoramic image.
Before the optimal splicing model based on pixel definition evaluation is used for splicing the panoramic image of the high-speed railway running environment video, the method further comprises the following steps:
and performing lossless information extraction on the high-speed railway running environment video data to obtain panoramic image format data.
The vehicle body attitude compensation method based on multi-source data registration is used for registering acquired high-speed railway running environment video data and vehicle body attitude digital signal analog data, and comprises the following steps:
extracting attitude angle information from high-speed railway operation environment video data based on a vanishing point detection algorithm;
and forming an input signal by the attitude angle information and the vehicle body attitude digital signal analog data, inputting the input signal into a deep learning registration network for training, and stably training the deep learning registration network to obtain the registration parameters of the waveform.
Wherein, the normal sound barrier image that the collection obtained trains the high-speed railway sound barrier anomaly detection network based on unsupervised learning, includes:
inputting the sound barrier image in a normal state into an encoder to obtain depth characteristics, and mapping the depth characteristics into an image by using a decoder to obtain a reconstructed image;
inputting the sound barrier image and the reconstructed image in the normal state into a depth convolution neural network to obtain depth characteristics corresponding to the two images respectively;
calculating reconstruction error loss of the images based on the sound barrier images and the reconstructed images in the normal state, and calculating depth feature similarity measurement loss according to the respective corresponding depth features of the two images;
and superposing the reconstruction error loss and the depth characteristic similarity measurement loss to obtain the total loss of the high-speed railway sound barrier anomaly detection network.
Wherein, carry out the sound barrier defect detection to the panorama of concatenation, include:
inputting the panoramic image into a high-speed railway sound barrier anomaly detection network to obtain depth characteristics corresponding to the panoramic image and the reconstructed image;
and calculating the difference between the depth features, and if the difference between the depth features is greater than a preset value, determining that the panoramic image is in an abnormal state.
In a second aspect, the present invention provides a device for detecting a defect in a sound barrier of a high-speed railway, comprising:
the acquisition unit is used for comprehensively inspecting the digital camera and the inertia component mounted on the vehicle and shooting video data of the running environment of the high-speed railway and analog data of digital signals of the posture of the vehicle body;
the registration unit is used for registering the acquired high-speed railway running environment video data and the vehicle body attitude digital signal analog data based on a vehicle body attitude compensation method of multi-source data registration;
the splicing unit is used for carrying out panorama splicing on the high-speed railway running environment video based on the optimal splicing model of the pixel definition evaluation;
and the detection unit is used for acquiring the obtained normal sound barrier images, training an unsupervised learning-based high-speed railway sound barrier abnormity detection network and carrying out sound barrier defect detection on the spliced panoramic image.
Further, the method also comprises the following steps:
and the optimization unit is used for performing lossless information extraction on the high-speed railway running environment video data to obtain panoramic image format data.
In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor executes the computer program to implement the steps of the method for detecting sound barrier defects of a high speed railway.
In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, implements the steps of the method for detecting a sound barrier defect of a high speed railway.
According to the technical scheme, the invention provides the method and the device for detecting the sound barrier defect of the high-speed railway, wherein video data of the running environment of the high-speed railway and analog data of the vehicle body posture digital signal are shot by mounting a digital camera and an inertia assembly on an integrated inspection vehicle; the vehicle body attitude compensation method based on multi-source data registration is used for registering the acquired video data of the high-speed railway running environment and the vehicle body attitude digital signal analog data; carrying out panorama stitching on the high-speed railway running environment video based on the optimal stitching model of the pixel definition evaluation; the obtained normal sound barrier image is collected, the high-speed railway sound barrier abnormity detection network based on unsupervised learning is trained, sound barrier defect detection is carried out on the spliced panoramic image, the purpose of vehicle-mounted dynamic detection of the sound barrier defect of the high-speed railway under unsupervised conditions is achieved, the operation safety of the high-speed railway is guaranteed to a certain extent, and the method has good practical value.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
FIG. 1 is a schematic flow chart of a high-speed railway sound barrier defect detection method in an embodiment of the invention;
FIG. 2 is an exemplary diagram of attitude changes during operation of a train in an embodiment of the present invention;
FIG. 3 is a waveform of vehicle body attitude data output by the inertial component in an embodiment of the invention;
FIG. 4 is a heterogeneous data registration model based on deep learning according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a panoramic annular sampling model of forward motion video according to an embodiment of the present invention;
FIG. 6 is a diagram illustrating definition of pixel sharpness according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a sound barrier anomaly detection network based on unsupervised learning according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a second structure of the high-speed railway sound barrier defect detection device in the embodiment of the invention;
fig. 9 is a schematic structural diagram of an electronic device in an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention provides an embodiment of a high-speed railway sound barrier defect detection method, and referring to fig. 1, the high-speed railway sound barrier defect detection method specifically comprises the following contents:
s1: the digital camera and the inertia component are mounted on the comprehensive inspection vehicle, and video data of the running environment of the high-speed railway and analog data of a vehicle body posture digital signal are shot;
in the step, videos of the running environment of the high-speed railway in different directions, which are acquired by the inspection vehicle, are synthesized. Referring to fig. 2, during detection, track geometry irregularities often cause the train to sway, for example, a difference in height between two tracks will cause the train to roll sideways about the Z-axis; the raising or lowering of the rail plane will cause the train to heave about the Y axis; while a lateral bend of the track will cause the train to deflect about the X-axis.
It should be noted that the digital video camera is rigidly connected to the vehicle body, i.e. without any damping device. Therefore, the shaking of the train inevitably causes the shaking of the camera equipment, the obtained video content is unstable, and the video scene shakes, so that the generated panoramic image has obvious distortion and distortion, and the subsequent intelligent analysis is influenced.
S2: the vehicle body attitude compensation method based on multi-source data registration is used for registering the acquired video data of the high-speed railway running environment and the vehicle body attitude digital signal analog data;
by tracking the track of the vanishing point of the steel rail in the video, three attitude angles generated by the camera along with the shaking of the vehicle body, namely a left-right deflection angle pan, a vertical pitch angle tilt and a rotation roll angle roll can be calculated. However, in practical application, the influence of the tracking and receiving imaging quality of the vanishing point, the illumination condition and the train turning is large, and the accuracy of the derived attitude angle of the camera is limited, so that the further application of the high-speed railway panoramic image is restricted.
Based on the accurate inertia subassembly on the comprehensive inspection car, including gyroscope and accelerometer, can accurate acquisition vehicle body's gesture, as shown in fig. 3. Although the conventional high-speed comprehensive inspection vehicle is provided with a uniform mileage positioning system, due to the heterogeneity of the inertial measurement unit and the video in data acquisition, hardware time delay, accumulated errors of a speed encoder and the like, the mileage counting of the inertial measurement unit and the video has errors. Therefore, how to accurately relate the vehicle body posture information acquired by the inertial measurement unit to the video frame is a difficult problem to be solved firstly.
In the step, a matching method of heterogeneous data based on deep learning is researched based on two data formats of high-speed railway operation environment video data and vehicle body attitude digital signal analog data. As shown in fig. 4, extracting attitude angle information from the high-speed railway operation environment video data based on a vanishing point detection algorithm; and forming an input signal by the attitude angle information and the vehicle body attitude digital signal analog data, inputting the input signal into a deep learning registration network for training, and stably training the deep learning registration network to obtain the registration parameters (dx, dy) of the waveform.
S3: carrying out panorama stitching on the high-speed railway running environment video based on the optimal stitching model of the pixel definition evaluation;
in this step, as shown in fig. 5, for a piece of forward motion video with a number of frames N, St is the sampling zone extracted from the t-th frame image, and Rt is the corresponding sampling region of St in the physical space scene. The definition is as follows:
st: an annular sampling band consisting of an outer sampling ring OSRt (green) and an inner sampling ring ISRt (red);
rt is the sampling area of St in the physical space scene;
constructing a splicing area: extracting a set of zonal sequences S from a video sequence1,S2,S3,...，SNIs satisfied with
Panoramic stitching:
t (-) is the image geometry correction.
Based on the above definition, the panorama stitching of forward motion video can be described as: extracting a set of zonal sequences S from a video sequence1,S2,S3,…，SNAnd satisfying that any adjacent space sampling areas are not overlapped and have no interval, namely the panoramic annular zone is used for completely sampling the physical space. Sequence of zones that will satisfy the conditionsThe columns are geometrically modified and merged together to produce a panoramic image. Therefore, the key issue in generating a panorama from a forward moving video is how to determine the positions of the stitched outer frame OSRt (green) and the stitched inner frame ISRt (red) in each frame image.
In order to ensure that the panorama has the best resolution, the position of the stitching outer frame OSRt should be located at the edge of the image with larger spatial resolution. However, for a high-speed railway, under a high-speed running condition of a train, a significant motion blur (a global motion blur is generated when a camera moves but a scene in a high-speed railway scene is still) is often generated in an acquired image, and a defocus blur may be generated when a steel rail extends to infinity. This requires a compromise between image resolution and motion blur when positioning the position of the stitched frame, and a trade-off between the two is made to determine the optimal position of the stitched frame for non-uniform rectangular sampling.
For a high-speed railway, in order to ensure the quality of the generated panoramic image, the project provides the concept of splicing definition, and establishes a splicing definition optimization model to determine the optimal splicing position. The red frame in FIG. 6 indicates the k-1 frame image fk-1ESR of splicing outer framek-1The green frame represents the k frame image fkESR of splicing outer framekThe orange frame represents the k +1 frame image fk+1ESR of splicing outer framek-1A pixel point in the k frame image
Defining the camera slave frame f under the influence of the previous frame (k-1 st frame) and the next frame (k +1 st frame)k-1Move to frame fk+1Through frame fkTime pixel (x)j,yu) The displacement of (a) is:
Δlk,j＝d(Hk(xj,yu,1)T,(xj,yu,1))；
the homography matrix H maps a pixel location to another pixel location. d (-) is the Euclidean distance between two pixel points. In a short time of 3 consecutive frames, it can be considered that
HK-1≈HK
To meet the requirements of resolution and ambiguity, the definition of pixels in video frames is defined to describe the absolute displacement of pixels between adjacent 3 frames. For frame fkPixel (x) of (2)j,yu) The splicing resolution is defined as:
in the formula,. DELTA.lk-1,jRepresenting a slave frame fk-1To frame fkA displacement of (a) (. DELTA.l)k,jRepresenting a slave frame fkTo frame fk+1σ is a constant, when (x)j,yu) When the inter-frame motion is small, HK-1And HKClose to the identity matrix I, therefore
Close to 1, indicated by (x)j,yu) The centered image block is likely to be sharpened. Otherwise
Small, meaning that the image block may contain large motion blur.
Define 4-directional splicing lines
And
the resolution of (d) is:
the following mathematical model is satisfied in the formula:
the expression above shows that the sum of the definition of the 4 splicing lines of the splicing outer frame is maximum under the condition of meeting the constraint condition of the geometric structure of the scene, and x is obtained under the condition that the sum of the definition of the 4 splicing lines reaches the maximum valuel、yu、xrAnd ydNamely the optimal position of the outer frame splicing line. And y isu、xrAnd ydAnd xlIn this regard, the following stitching optimization model can be obtained:
and converting the video splicing problem of the high-speed railway into the optimization problem to solve, and realizing image splicing.
S4: and acquiring the obtained normal sound barrier image, training a high-speed railway sound barrier abnormity detection network based on unsupervised learning, and carrying out sound barrier defect detection on the spliced panoramic image.
The high-speed rail running environment video is collected through a vehicle-mounted camera, and the influence of external factors such as different seasons, weather and the like on the image quality of the video is detected in the running process of a train. In addition, the abnormal form of the high-speed railway sound barrier abnormality detection task cannot be predicted, so that a feature template cannot be constructed by a feature modeling method, and the traditional image classification method cannot obtain a good detection effect.
With the development of the deep learning technology, the image identification method based on the supervised deep learning is gradually mature, and the identification performance is greatly improved. However, in the task of detecting the abnormal sound barrier of the high-speed railway, the abnormal samples are very difficult to obtain, the number of the samples is very small, the target abnormal state cannot be effectively expressed by directly performing feature learning through a deep network, and the accuracy rate of the detection result is low.
The image anomaly detection method based on the self-encoder (AutoEncoder) only uses normal images for training, so that the self-encoder can reconstruct the normal images, and when the abnormal images are input, the reconstruction error between the images and the reconstructed images is large, and the abnormal images can be identified. However, the abnormal state belongs to the high-level semantic features of the images, and the reconstruction error from the encoder is measured according to the pixel difference of the two images, so the detection result of the method is unstable. In addition, since the self-encoder is sensitive to image noise, blur and changes in lighting conditions, any small changes in the input image may result in a reconstruction result with large differences.
In order to solve the problems, the research provides a sound barrier anomaly detection network based on unsupervised learning, and the network is additionally provided with a two-branch deep convolutional neural network on the basis of an autoencoder, and is used for extracting the depth characteristics of an input image and a reconstructed image and carrying out similarity measurement on the two depth characteristics. The network not only requires the minimum reconstruction error of the reconstructed image and the input image, but also requires the minimum distance between the depth feature of the reconstructed image and the depth feature of the input image, namely, the high-level semantic features of the images of the same category are the same, so that the sensitivity of the self-encoder to external factors such as image noise, blurring and illumination condition change is reduced, and the purpose of improving the target anomaly detection precision is achieved. The overall architecture of the network is shown in fig. 7.
In the network training stage, firstly, the sound barrier image x in a normal state is input into an encoder to obtain depth features, and the depth features are mapped into an image by using a decoder to obtain a reconstructed image x'. Then, the image x and the reconstructed image x 'are input into a two-branch depth convolution neural network, so as to obtain the depth features f (x) and f (x') of the two images, and the weight parameters of the two-branch depth convolution neural network are shared. Finally, the reconstruction error loss of the image is calculated by using x and x ', the depth characteristic similarity measurement loss is calculated by using f (x) and f (x'), and the two losses are added to serve as the total loss of the optimization network model.
In practical use, a to-be-detected sound barrier image is input, a reconstructed image is obtained by using a self-encoder, the depth characteristics of the two images are extracted, and finally the difference between the depth characteristics is calculated. Because only the normal state image is used in the model training stage, when the image to be detected is abnormal, the difference between the depth characteristic of the reconstructed image and the depth characteristic of the image to be detected is inevitably large, and the abnormal state of the image to be detected can be known.
In the above model, the reconstruction error loss function uses the L1 distance, i.e., when each pixel of the reconstructed image is the same as each pixel of the input image, the reconstruction error between the two images is minimal. The depth feature similarity measure loss function typically uses the L2 distance, i.e., the euclidean distance of two depth feature vectors is the smallest when they belong to the same class. Thus, the global loss function of the above model is defined as follows:
f(x)＝WTx+b＝||W||||x||cos(θ)+b；
wherein N is the total number of training samples, xiIs the ith training sample image, x'iFor reconstructing the image, f (x) is extracting the depth feature vector of the sample x, W is the weight parameter, and b is the bias term. The loss function not only requires that the image reconstruction error is small, but also requires that the depth features of the reconstructed image are similar to those of the input image, i.e., the high-level semantic features of the same category of images are the same.
As can be seen from the above description, in the method for detecting a sound barrier defect of a high-speed railway provided by the embodiment of the invention, video data of a running environment of the high-speed railway and analog data of a vehicle body posture digital signal are shot by mounting a digital camera and an inertia component on an integrated inspection vehicle; the vehicle body attitude compensation method based on multi-source data registration is used for registering the acquired video data of the high-speed railway running environment and the vehicle body attitude digital signal analog data; carrying out panorama stitching on the high-speed railway running environment video based on the optimal stitching model of the pixel definition evaluation; the obtained normal sound barrier image is collected, the high-speed railway sound barrier abnormity detection network based on unsupervised learning is trained, sound barrier defect detection is carried out on the spliced panoramic image, the purpose of vehicle-mounted dynamic detection of the sound barrier defect of the high-speed railway under unsupervised conditions is achieved, the operation safety of the high-speed railway is guaranteed to a certain extent, and the method has good practical value.
In an embodiment of the present invention, before the optimal stitching model based on pixel definition evaluation performs panorama stitching on a high speed railway operation environment video, the method further includes:
and performing lossless information extraction on the high-speed railway running environment video data to obtain panoramic image format data.
In the step, a panoramic stitching model of the high-speed railway running environment video is constructed, and lossless information extraction is carried out on massive video data to obtain a lightweight panoramic image format, so that the storage and access expenses of the video data are reduced, and the video is converted into a form more suitable for manual inspection or computer analysis processing; and then, based on the generated panoramic image, providing an unsupervised learning-based high-speed railway sound barrier abnormity detection network, and realizing automatic identification of the high-speed railway sound barrier defect.
The embodiment of the invention provides a specific implementation manner of a high-speed railway sound barrier defect detection device capable of realizing all contents in the high-speed railway sound barrier defect detection method, and referring to fig. 8, the high-speed railway sound barrier defect detection device specifically comprises the following contents:
the acquisition unit 10 is used for comprehensively inspecting the digital camera and the inertia component mounted on the vehicle and shooting video data of the running environment of the high-speed railway and analog data of digital signals of the posture of the vehicle body;
the registration unit 20 is used for registering the acquired video data of the high-speed railway running environment and the analog data of the vehicle body attitude digital signals based on a vehicle body attitude compensation method of multi-source data registration;
the splicing unit 30 is used for carrying out panorama splicing on the high-speed railway running environment video based on the optimal splicing model of the pixel definition evaluation;
and the detection unit 40 is used for acquiring the obtained normal sound barrier images, training an unsupervised learning-based high-speed railway sound barrier abnormity detection network, and performing sound barrier defect detection on the spliced panoramic image.
Further, the method also comprises the following steps:
and the optimization unit is used for performing lossless information extraction on the high-speed railway running environment video data to obtain panoramic image format data.
The embodiment of the high-speed railway sound barrier defect detection apparatus provided by the present invention may be specifically used for executing the processing flow of the embodiment of the high-speed railway sound barrier defect detection method in the above embodiment, and the functions thereof are not described herein again, and reference may be made to the detailed description of the above method embodiment.
As can be seen from the above description, the high-speed railway sound barrier defect detection device provided by the embodiment of the invention shoots the video data of the running environment of the high-speed railway and the analog data of the vehicle body posture digital signal by mounting the digital camera and the inertia component on the comprehensive inspection vehicle; the vehicle body attitude compensation method based on multi-source data registration is used for registering the acquired video data of the high-speed railway running environment and the vehicle body attitude digital signal analog data; carrying out panorama stitching on the high-speed railway running environment video based on the optimal stitching model of the pixel definition evaluation; the obtained normal sound barrier image is collected, the high-speed railway sound barrier abnormity detection network based on unsupervised learning is trained, sound barrier defect detection is carried out on the spliced panoramic image, the purpose of vehicle-mounted dynamic detection of the sound barrier defect of the high-speed railway under unsupervised conditions is achieved, the operation safety of the high-speed railway is guaranteed to a certain extent, and the method has good practical value.
The application provides an embodiment of an electronic device for implementing all or part of contents in the high-speed railway sound barrier defect detection method, and the electronic device specifically includes the following contents:
a processor (processor), a memory (memory), a communication Interface (Communications Interface), and a bus; the processor, the memory and the communication interface complete mutual communication through the bus; the communication interface is used for realizing information transmission between related devices; the electronic device may be a desktop computer, a tablet computer, a mobile terminal, and the like, but the embodiment is not limited thereto. In this embodiment, the electronic device may be implemented with reference to the embodiment for implementing the method for detecting a defect in a sound barrier of a high-speed railway and the embodiment for implementing the device for detecting a defect in a sound barrier of a high-speed railway in the embodiments, and the contents thereof are incorporated herein, and repeated details are not repeated.
Fig. 9 is a schematic block diagram of a system configuration of an electronic device 9600 according to an embodiment of the present application. As shown in fig. 9, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. Notably, this fig. 9 is exemplary; other types of structures may also be used in addition to or in place of the structure to implement telecommunications or other functions.
In one embodiment, the high speed railway sound barrier defect detection function may be integrated into the central processor 9100. The central processor 9100 may be configured to control as follows:
the digital camera and the inertia component are mounted on the comprehensive inspection vehicle, and video data of the running environment of the high-speed railway and analog data of a vehicle body posture digital signal are shot; the vehicle body attitude compensation method based on multi-source data registration is used for registering the acquired video data of the high-speed railway running environment and the vehicle body attitude digital signal analog data; carrying out panorama stitching on the high-speed railway running environment video based on the optimal stitching model of the pixel definition evaluation; and acquiring the obtained normal sound barrier image, training a high-speed railway sound barrier abnormity detection network based on unsupervised learning, and carrying out sound barrier defect detection on the spliced panoramic image.
In another embodiment, the high-speed railway sound barrier defect detection device may be configured separately from the central processor 9100, for example, the high-speed railway sound barrier defect detection device may be configured as a chip connected to the central processor 9100, and the high-speed railway sound barrier defect detection function is realized by the control of the central processor.
As shown in fig. 9, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is noted that the electronic device 9600 also does not necessarily include all of the components shown in fig. 9; in addition, the electronic device 9600 may further include components not shown in fig. 9, which may be referred to in the prior art.
As shown in fig. 9, a central processor 9100, sometimes referred to as a controller or operational control, can include a microprocessor or other processor device and/or logic device, which central processor 9100 receives input and controls the operation of the various components of the electronic device 9600.
The memory 9140 can be, for example, one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The information relating to the failure may be stored, and a program for executing the information may be stored. And the central processing unit 9100 can execute the program stored in the memory 9140 to realize information storage or processing, or the like.
The input unit 9120 provides input to the central processor 9100. The input unit 9120 is, for example, a key or a touch input device. Power supply 9170 is used to provide power to electronic device 9600. The display 9160 is used for displaying display objects such as images and characters. The display may be, for example, an LCD display, but is not limited thereto.
The memory 9140 can be a solid state memory, e.g., Read Only Memory (ROM), Random Access Memory (RAM), a SIM card, or the like. There may also be a memory that holds information even when power is off, can be selectively erased, and is provided with more data, an example of which is sometimes called an EPROM or the like. The memory 9140 could also be some other type of device. Memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application/function storage portion 9142, the application/function storage portion 9142 being used for storing application programs and function programs or for executing a flow of operations of the electronic device 9600 by the central processor 9100.
The memory 9140 can also include a data store 9143, the data store 9143 being used to store data, such as contacts, digital data, pictures, sounds, and/or any other data used by an electronic device. The driver storage portion 9144 of the memory 9140 may include various drivers for the electronic device for communication functions and/or for performing other functions of the electronic device (e.g., messaging applications, contact book applications, etc.).
The communication module 9110 is a transmitter/receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter/receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as in the case of a conventional mobile communication terminal.
Based on different communication technologies, a plurality of communication modules 9110, such as a cellular network module, a bluetooth module, and/or a wireless local area network module, may be provided in the same electronic device. The communication module (transmitter/receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and receive audio input from the microphone 9132, thereby implementing ordinary telecommunications functions. The audio processor 9130 may include any suitable buffers, decoders, amplifiers and so forth. In addition, the audio processor 9130 is also coupled to the central processor 9100, thereby enabling recording locally through the microphone 9132 and enabling locally stored sounds to be played through the speaker 9131.
An embodiment of the present invention further provides a computer-readable storage medium capable of implementing all the steps in the sound barrier defect detection method for a high speed railway in the above embodiment, where the computer-readable storage medium stores a computer program, and the computer program, when executed by a processor, implements all the steps in the sound barrier defect detection method for a high speed railway in the above embodiment, for example, when the processor executes the computer program, implements the following steps:
the digital camera and the inertia component are mounted on the comprehensive inspection vehicle, and video data of the running environment of the high-speed railway and analog data of a vehicle body posture digital signal are shot; the vehicle body attitude compensation method based on multi-source data registration is used for registering the acquired video data of the high-speed railway running environment and the vehicle body attitude digital signal analog data; carrying out panorama stitching on the high-speed railway running environment video based on the optimal stitching model of the pixel definition evaluation; and acquiring the obtained normal sound barrier image, training a high-speed railway sound barrier abnormity detection network based on unsupervised learning, and carrying out sound barrier defect detection on the spliced panoramic image.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
Finally, it should be noted that: the above embodiments are only used to illustrate the technical solution of the present invention, and not to limit the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the present invention, and they should be construed as being included in the following claims and description.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110234308.2A CN113034439B (en) | 2021-03-03 | 2021-03-03 | High-speed railway sound barrier defect detection method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110234308.2A CN113034439B (en) | 2021-03-03 | 2021-03-03 | High-speed railway sound barrier defect detection method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113034439A CN113034439A (en) | 2021-06-25 |
| CN113034439B true CN113034439B (en) | 2021-11-23 |
Family
ID=76466529
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110234308.2A Active CN113034439B (en) | 2021-03-03 | 2021-03-03 | High-speed railway sound barrier defect detection method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113034439B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113935103B (en) * | 2021-12-17 | 2022-03-18 | 中国铁路设计集团有限公司 | Method for dynamically predicting insertion loss of high-iron-based sound source and sound barrier with limited long line |
| CN114373004B (en) * | 2022-01-13 | 2024-04-02 | 强联智创（北京）科技有限公司 | Dynamic image registration method |
| CN116665106A (en) * | 2023-06-02 | 2023-08-29 | 上海市政养护管理有限公司 | A video analysis-based method for hidden danger analysis of urban expressway noise barriers |
| CN117420151B (en) * | 2023-12-19 | 2024-03-01 | 中国铁路设计集团有限公司 | Vertical sound barrier detection method for high-speed railway |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780608A (en) * | 2016-11-23 | 2017-05-31 | 北京地平线机器人技术研发有限公司 | Posture information method of estimation, device and movable equipment |
| CN109359526A (en) * | 2018-09-11 | 2019-02-19 | 深圳大学 | A face pose estimation method, device and device |
| CN109712189A (en) * | 2019-03-26 | 2019-05-03 | 深兰人工智能芯片研究院（江苏）有限公司 | A kind of method and apparatus of sensor combined calibrating |
| CN109842761A (en) * | 2019-02-01 | 2019-06-04 | 中国社会科学院大学 | The quick high definition panorama imaging method and device of propulsion video |
| CN110555474A (en) * | 2019-08-28 | 2019-12-10 | 上海电力大学 | photovoltaic panel fault detection method based on semi-supervised learning |
| CN111951264A (en) * | 2020-08-28 | 2020-11-17 | 浙江中烟工业有限责任公司 | An anomaly detection and localization method for cigarette packaging based on deep learning |
| CN112419318A (en) * | 2020-12-17 | 2021-02-26 | 深圳市华汉伟业科技有限公司 | Multi-path cascade feedback-based anomaly detection method and device and storage medium |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103344946B (en) * | 2013-07-08 | 2015-03-18 | 杭州电子科技大学 | Foundation radar and aerial mobile platform radar real-time error registering method |
| CN107330439B (en) * | 2017-07-14 | 2022-11-04 | 腾讯科技（深圳）有限公司 | A method, client and server for determining the pose of an object in an image |
| CN110287873B (en) * | 2019-06-25 | 2021-06-29 | 清华大学深圳研究生院 | Non-cooperative target pose measurement method and system based on deep neural network and terminal equipment |
-
2021
- 2021-03-03 CN CN202110234308.2A patent/CN113034439B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106780608A (en) * | 2016-11-23 | 2017-05-31 | 北京地平线机器人技术研发有限公司 | Posture information method of estimation, device and movable equipment |
| CN109359526A (en) * | 2018-09-11 | 2019-02-19 | 深圳大学 | A face pose estimation method, device and device |
| CN109842761A (en) * | 2019-02-01 | 2019-06-04 | 中国社会科学院大学 | The quick high definition panorama imaging method and device of propulsion video |
| CN109712189A (en) * | 2019-03-26 | 2019-05-03 | 深兰人工智能芯片研究院（江苏）有限公司 | A kind of method and apparatus of sensor combined calibrating |
| CN110555474A (en) * | 2019-08-28 | 2019-12-10 | 上海电力大学 | photovoltaic panel fault detection method based on semi-supervised learning |
| CN111951264A (en) * | 2020-08-28 | 2020-11-17 | 浙江中烟工业有限责任公司 | An anomaly detection and localization method for cigarette packaging based on deep learning |
| CN112419318A (en) * | 2020-12-17 | 2021-02-26 | 深圳市华汉伟业科技有限公司 | Multi-path cascade feedback-based anomaly detection method and device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113034439A (en) | 2021-06-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113034439B (en) | High-speed railway sound barrier defect detection method and device | |
| CN109664820A (en) | Driving reminding method, device, equipment and storage medium based on automobile data recorder | |
| CN102426785B (en) | Traffic flow information perception method and system based on contour and local feature points | |
| CN112927303A (en) | Lane line-based automatic driving vehicle-mounted camera pose estimation method and system | |
| CN111199556A (en) | Indoor pedestrian detection and tracking method based on camera | |
| CN109144043A (en) | Method for tracking target object | |
| CN107194343B (en) | Traffic light detection method based on location-dependent convolution and Fire model | |
| CN110176022B (en) | Tunnel panoramic monitoring system and method based on video detection | |
| CN114219929A (en) | Parking number identification method and device | |
| CN117745709B (en) | A method, system, equipment and medium for detecting foreign object intrusion in railways | |
| CN116630650A (en) | A method for identifying the width of a field ridge and an unmanned vehicle device for a field ridge | |
| CN110443819A (en) | A kind of track detection method and device of monorail train | |
| CN117315547A (en) | A visual SLAM method to solve the problem of large proportion of dynamic objects | |
| CN103581536A (en) | Image capturing system and method | |
| CN107506753B (en) | Multi-vehicle tracking method for dynamic video monitoring | |
| CN112883836B (en) | Video detection method for deformation of underground coal mine roadway | |
| CN112766046B (en) | Target detection method and related device | |
| Zhao et al. | Real-world trajectory extraction from aerial videos-a comprehensive and effective solution | |
| CN113643374B (en) | Multi-camera calibration method, device, equipment and medium based on road characteristics | |
| CN119573742B (en) | Vector magnetic pattern matching method, device, equipment and medium | |
| CN118348015B (en) | Self-movable subway tunnel structure apparent disease detection equipment and method | |
| CN118587426B (en) | A lightweight dual-channel railway foreign body intrusion detection method | |
| CN119273715B (en) | IMU-assisted moving object detection method under dynamic condition | |
| JP2008276613A (en) | Moving object determination device, computer program, and moving object determination method | |
| CN116664906B (en) | Automatic recognition method and system for groove-shaped rail turnout of modern tram |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20221129 Address after: 1803-56, Floor 18, Building 2, Yard 1, Hangfeng Road, Fengtai District, Beijing 100070 Patentee after: Beijing chuangying Zhitong Technology Co.,Ltd. Address before: 100044 Beijing city Haidian District Xizhimen Shangyuan Village No. 3 Patentee before: Beijing Jiaotong University |