[go: up one dir, main page]

CN112967271B - A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model - Google Patents

A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model Download PDF

Info

Publication number
CN112967271B
CN112967271B CN202110317123.8A CN202110317123A CN112967271B CN 112967271 B CN112967271 B CN 112967271B CN 202110317123 A CN202110317123 A CN 202110317123A CN 112967271 B CN112967271 B CN 112967271B
Authority
CN
China
Prior art keywords
layer
module
convolution
network
casting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110317123.8A
Other languages
Chinese (zh)
Other versions
CN112967271A (en
Inventor
张辉
车爱博
王可
李晨
刘理
陈煜嵘
王耀南
缪志强
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hunan University
Changsha University of Science and Technology
Original Assignee
Hunan University
Changsha University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hunan University, Changsha University of Science and Technology filed Critical Hunan University
Priority to CN202110317123.8A priority Critical patent/CN112967271B/en
Publication of CN112967271A publication Critical patent/CN112967271A/en
Application granted granted Critical
Publication of CN112967271B publication Critical patent/CN112967271B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/26Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a casting surface defect identification method based on an improved deep Labv3+ network model, which comprises the following steps: s1, collecting a casting image data set to obtain a training set and a test set; step S2, constructing a network model, and performing data training and correction on the network model through a training set and a testing set to generate a defect detection network; step S3, designing a loss function of the defect detection network; and step S4, the defect detection network identifies and outputs the defect detection result of the casting and displays the detection duration. According to the invention, the surface defects of the castings are identified by adopting a deep learning method, so that the accuracy and speed of defect identification are improved, and a new idea is provided for quality detection of industrial castings.

Description

一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model

技术领域technical field

本发明涉及工业铸件缺陷检测技术领域,特别涉及一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法。The invention relates to the technical field of industrial casting defect detection, in particular to a casting surface defect identification method based on an improved DeepLabv3+ network model.

背景技术Background technique

在工业铸造生产中,各类铸件被广泛使用。然而,在生产制造的过程中,铸件仍然不可避免的会出现裂痕、缩孔等缺陷。这些缺陷会严重影响到铸件成品的质量,进而影响着整个制造产品的质量。高效的缺陷检测技术是保障铸件成品质量的重要环节之一。因此,在现代铸造行业高速发展生产时期,精确高效的铸件表面缺陷检测技术是工业产品检测的重要发展方向。In industrial casting production, various types of castings are widely used. However, in the process of production, castings still inevitably have defects such as cracks and shrinkage holes. These defects can seriously affect the quality of finished castings, which in turn affects the quality of the entire manufactured product. Efficient defect detection technology is one of the important links to ensure the quality of casting products. Therefore, in the period of rapid development and production of the modern foundry industry, accurate and efficient casting surface defect detection technology is an important development direction of industrial product detection.

目前大部分国内铸造生产仍采用人工抽样检测的方式进行铸件缺陷的检测,主要依靠检测人员肉眼观察,主观判断完成检测。该方法依赖于检测人员的先验知识,具有较强的主观性,缺乏准确性和规范性,且效率得不到保障。其他的工业铸件表面缺陷检测方法还有涡流线圈检测和超声波检测等。然而这些传统方法都存在不同的弊端,检测工作耗费大量的人力物力,最终的检测结果同样需要人工处理做出判断。同时在超声波检测和涡流线圈检测时可能会与铸件表面缺陷产生接触可能发生物理与化学的变化,进一步扩大缺陷的区域,不利于检测精度的提高。At present, most of the domestic foundry production still adopts the method of manual sampling inspection to detect casting defects, mainly relying on the inspection personnel to observe with the naked eye and subjectively judge to complete the inspection. This method relies on the prior knowledge of the detection personnel, has strong subjectivity, lacks accuracy and standardization, and cannot guarantee the efficiency. Other industrial casting surface defect detection methods include eddy current coil inspection and ultrasonic inspection. However, these traditional methods all have different drawbacks. The detection work consumes a lot of manpower and material resources, and the final detection results also require manual processing to make judgments. At the same time, during ultrasonic inspection and eddy current coil inspection, there may be contact with the surface defects of the casting, and physical and chemical changes may occur, further expanding the defect area, which is not conducive to the improvement of inspection accuracy.

发明内容SUMMARY OF THE INVENTION

针对上述问题,为改善当前铸件缺陷检测困境,本发明提出了一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法,其基于神经网络的检测方法,通过改进DeepLabv3+网络模型进行铸件缺陷检测,能够实现自动实时在线以及智能化检测铸件表面缺陷。In view of the above problems, in order to improve the current casting defect detection dilemma, the present invention proposes a casting surface defect identification method based on the improved DeepLabv3+ network model. Automatic real-time online and intelligent detection of casting surface defects.

采集的工件表面缺陷图像被制作成缺陷数据集送入到改进后的神经网络中,进而通过基于语义分割的自定义网络训练学习,最终得到训练完成的网络,用于工件表面缺陷图像检测并标记缺陷区域,进而结合缺陷检测软件,输出检测的时间,同时可替换网络模型进行不同模型检测对比,在本方法的研究下工件的缺陷检测可进行智能识别,达到高精度检测、减少人工干预的目的。The collected workpiece surface defect images are made into a defect dataset and sent to the improved neural network, and then trained and learned through a self-defined network based on semantic segmentation, and finally the trained network is obtained for image detection and marking of workpiece surface defects. The defect area is then combined with the defect detection software to output the detection time. At the same time, the network model can be replaced to compare different models. Under the research of this method, the defect detection of the workpiece can be intelligently identified, so as to achieve high-precision detection and reduce manual intervention. .

为了达到上述目的,本发明提供的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法,包括如下步骤:In order to achieve the above purpose, a method for identifying surface defects of castings based on the improved DeepLabv3+ network model provided by the present invention includes the following steps:

步骤S1、采集铸件图像数据集,获得训练集和测试集:工业相机采集铸件图像,Labelme标注铸件缺陷,生成图像数据集,并将图像数据集分成训练集和测试集;Step S1, collecting a casting image data set to obtain a training set and a test set: the industrial camera collects the casting image, Labelme marks the casting defects, generates an image data set, and divides the image data set into a training set and a test set;

步骤S2、构建网络模型,并通过训练集和测试集对网络模型进行数据训练和修正,生成缺陷检测网络;Step S2, constructing a network model, and performing data training and correction on the network model through the training set and the test set to generate a defect detection network;

步骤S3、设计缺陷检测网络的损失函数:损失函数为包括预测结果的交叉熵损失及真实值的交叉熵损失的函数;Step S3, designing the loss function of the defect detection network: the loss function is a function including the cross-entropy loss of the prediction result and the cross-entropy loss of the real value;

步骤S4、所述缺陷检测网络识别并输出铸件缺陷检测结果,并显示检测时长。Step S4, the defect detection network identifies and outputs the casting defect detection result, and displays the detection duration.

优选地,步骤S2具体包括如下步骤:Preferably, step S2 specifically includes the following steps:

步骤S21、构建改进DeepLabV3+网络模型:改进现有DeepLabV3+网络模型中的编码解码模块,所述编码解码模块采用深度可分离卷积网络,并删减通道数量;Step S21, constructing and improving the DeepLabV3+ network model: improving the encoding and decoding module in the existing DeepLabV3+ network model, the encoding and decoding module adopts a depth separable convolutional network, and the number of channels is deleted;

步骤S22、进行数据训练和修正生成缺陷检测网络:将改进DeepLabv3+网络模型作为铸件表面缺陷识别的网络模型来提取原始采集的铸件图片的铸件表面缺陷特征,通过利用训练集中的训练数据对改进DeepLabv3+网络模型进行训练,再将测试集输入到改进DeepLabv3+网络模型中对其进行修正,直至生成的缺陷检测网络的预测精度满足预测精度阈值。Step S22, perform data training and correction to generate a defect detection network: use the improved DeepLabv3+ network model as a network model for casting surface defect identification to extract the casting surface defect features of the originally collected casting images, and improve the DeepLabv3+ network by using the training data in the training set. The model is trained, and then the test set is input into the improved DeepLabv3+ network model for correction until the prediction accuracy of the generated defect detection network meets the prediction accuracy threshold.

优选地,所述步骤S4中所述输出铸件检测结果具体为通过面对工业铸件缺陷检测的可视化软件实现输出检测结果。Preferably, in the step S4, outputting the detection result of the casting is specifically realized by outputting the detection result through a visualization software facing the defect detection of the industrial casting.

优选地,所述步骤S21中,所述编码解码模块包括编码模块及解码模块,所述编码模块与所述解码模块连接,所述编码模块包括深度卷积网络层、编码器及空洞空间金字塔池化模块,所述深度卷积网络层为主干模型,输入图片通过所述深度卷积网络层进行卷积后得到的包括缺陷形状信息的低级语义特征直接传入所述解码模块中,输入图片通过编码器及所述深度卷积网络层得到的包括缺陷位置类别的高级语义特征传入所述空洞空间金字塔池化模块,所述空洞空间金字塔池化模块包括空洞卷积模块、四层卷积层及一层池化层,所述四层卷积层与所述一层池化层串行连接,所述空洞卷积模块分别采用不同膨胀率的空洞卷积块,控制编码器提取特征的分辨率,分别进行空洞卷积后得到所述四层卷积层的四个特征图,所述空洞卷积模块进行池化后得到所述一层池化层的一个特征图。Preferably, in the step S21, the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, and the encoding module includes a deep convolutional network layer, an encoder and a hole space pyramid pool The deep convolutional network layer is the backbone model, and the low-level semantic features including defect shape information obtained after convolution of the input picture through the deep convolutional network layer are directly transmitted to the decoding module, and the input picture is passed through The high-level semantic features including defect location categories obtained by the encoder and the deep convolutional network layer are passed into the hole space pyramid pooling module, and the hole space pyramid pooling module includes a hole convolution module and a four-layer convolution layer. and one layer of pooling layer, the four layers of convolution layers are connected in series with the one layer of pooling layers, and the hole convolution modules respectively use hole convolution blocks with different expansion rates to control the resolution of the features extracted by the encoder. rate, respectively perform hole convolution to obtain four feature maps of the four-layer convolution layer, and the hole convolution module obtains a feature map of the one-layer pooling layer after pooling.

优选地,对于所述主干模型得到的低级语义特征信息,所述编码模块通过一个卷积对低级语义特征进行操作,以减少通道数,运算后得到最终的低级语义特征的语义信息,传到所述解码模块的解码器中;对于所述空洞空间金字塔池化模块得到的高级语义特征,所述解码模块通过一次上采样得到最终的高级语义特征的语义信息,作为输出结果,传入到解码器中。Preferably, for the low-level semantic feature information obtained by the backbone model, the encoding module operates on the low-level semantic feature through a convolution to reduce the number of channels, and obtains the final semantic information of the low-level semantic feature after the operation, and transmits it to the In the decoder of the decoding module; for the high-level semantic features obtained by the hole space pyramid pooling module, the decoding module obtains the semantic information of the final high-level semantic features through one upsampling, and is passed to the decoder as the output result. middle.

优选地,所述编码模块还包括堆叠层、通道调整卷积层、中间特征层、中间特征层堆叠层、后处理卷积层及后处理模块,所述堆叠层为所述四层卷积层及所述一层池化层获得的特征图进行堆叠起来形成,对所述堆叠层进行卷积以调整通道,形成所述通道调整卷积层,将所述通道调整卷积层进行长宽调整,形成所述中间特征层,所述中间特征层堆叠后形成所述中间特征层堆叠层,所述中间特征层堆叠层进行卷积后形成所述后处理卷积层,再对所述后处理卷积层通过所述后处理模块进行后处理。Preferably, the encoding module further includes a stack layer, a channel adjustment convolution layer, an intermediate feature layer, an intermediate feature layer stack layer, a post-processing convolution layer and a post-processing module, and the stack layer is the four-layer convolution layer and the feature map obtained by the layer of pooling layer are stacked to form, the stacked layer is convolved to adjust the channel, the channel adjustment convolution layer is formed, and the channel adjustment convolution layer is adjusted in length and width , the intermediate feature layer is formed, the intermediate feature layer stack is formed after the intermediate feature layer is stacked, the intermediate feature layer stack is convoluted to form the post-processing convolution layer, and then the post-processing convolution layer is formed. The convolutional layer is post-processed by the post-processing module.

优选地,所述后处理模块包括像素点分类模块及像素点类别概率计算模块,所述像素点分类模块对所述后处理卷积层的图像进行每个像素点的分类,所述像素点类别概率计算模块采用softmax求解每个像素点类别的概率。Preferably, the post-processing module includes a pixel point classification module and a pixel point category probability calculation module, the pixel point classification module classifies each pixel point on the image of the post-processing convolution layer, and the pixel point category The probability calculation module uses softmax to solve the probability of each pixel category.

优选地,所述解码模块将所述编码器输出的高级语义特征采用采样因子为4的双线性上采样进行上采样,所述解码模块的输入由两部分组成,一部分来自主干模型的所述深度卷积网络层的直接输出,连接从主干模型所输出的对应的具有相同空间分辨率的低级语义特征,另一部分通过在主干模型的深度卷积网络层导入至所述空洞空间金字塔池化模块中,找到一个与编码器输出的语义特征分辨率相同的低级语义特征图,经过1*1卷积进行降通道数使之与编码器输出的语义特征所占通道比重一样,并将所述解码模块两部分输入分别得到的低级语义特征和低级语义特征图连接在一起,在连接处理后,然后再通过一个3*3细化卷积进行细化,后通过采样因子为4的双线性上采样得到最终的预测结果。Preferably, the decoding module upsamples the high-level semantic features output by the encoder using bilinear upsampling with a sampling factor of 4, and the input of the decoding module consists of two parts, one part comes from the backbone model of the The direct output of the deep convolutional network layer is connected to the corresponding low-level semantic features with the same spatial resolution output from the backbone model, and the other part is imported into the hole spatial pyramid pooling module through the deep convolutional network layer of the backbone model. , find a low-level semantic feature map with the same resolution as the semantic feature output by the encoder, reduce the number of channels through 1*1 convolution to make it equal to the channel proportion of the semantic feature output by the encoder, and decode the The low-level semantic features and low-level semantic feature maps obtained from the input of the two parts of the module are connected together. After the connection is processed, it is refined by a 3*3 thinning convolution, and then the bilinear sampling factor is 4. Sampling to get the final prediction result.

优选地,所述编码模块传入到所述解码模块的输出采用的上采样以及所述解码模块输出采用的上采样采用的均是双线性插值法,定义公式如下:Preferably, the upsampling adopted by the output of the encoding module to the decoding module and the upsampling adopted by the output of the decoding module are bilinear interpolation, and the definition formula is as follows:

srcx=desx*srcω/desω src x =des x *src ω /des ω

srcy=desy*srch/desh src y =des y *src h /des h

式中,srcx表示原图像中像素点x坐标,srcy表示原图像中像素点y坐标,desx表示目标图像中像素点x坐标,desy表示目标图像中像素点y坐标,srcω表示原图像宽度,srch表示原图像高度,desω表示目标图像宽度,desh表示目标图像高度。In the formula, src x represents the x-coordinate of the pixel in the original image, src y represents the y-coordinate of the pixel in the original image, des x represents the x-coordinate of the pixel in the target image, des y represents the y-coordinate of the pixel in the target image, and src ω represents The width of the original image, src h is the height of the original image, des ω is the width of the target image, and des h is the height of the target image.

优选地,所述步骤S3中损失函数使用交叉熵损失函数,公式如下:Preferably, the loss function in the step S3 uses a cross-entropy loss function, and the formula is as follows:

J=-[y·log(p)+(1-y)·log(1-p)]J=-[y·log(p)+(1-y)·log(1-p)]

其中y表示样本的label,正类为1,负类为0;p表示样本预测为正的概率。Where y represents the label of the sample, the positive class is 1, and the negative class is 0; p represents the probability that the sample is predicted to be positive.

本发明能够取得下列有益效果:本发明所改进的基于DeepLabv3+网络模型可用于工业铸件缺陷识别,将此识别网络可直接应用于工业铸件缺陷检测领域,在进行识别后产生缺陷识别的数字化缺陷数据,结果存储和查询可与铸件信息一一对应,提供了一定的数据反馈,该反馈可用于铸造工艺的优化,为后续的工艺提供指导,特别地指出,本方法主要以工业铸件打磨缺陷处为目标,进行铸件的缺陷检测识别。The present invention can achieve the following beneficial effects: the improved DeepLabv3+ network model based on the present invention can be used for industrial casting defect identification, and the identification network can be directly applied to the field of industrial casting defect detection, and digital defect data for defect identification is generated after identification, The result storage and query can correspond one-to-one with the casting information, providing a certain data feedback, which can be used for the optimization of the casting process and provide guidance for the subsequent process. In particular, it is pointed out that this method mainly targets the grinding defects of industrial castings , for casting defect detection and identification.

本发明采用的基于改进DeepLabV3+的网络模型,具有识别速度快,识别精确度高的优点,将传统的缺陷检测工艺改造为优良的基于深度学习算法和深度神经卷积网络的缺陷检测识别方法,有效解决了传统工艺的缺点。The network model based on the improved DeepLabV3+ adopted in the present invention has the advantages of fast recognition speed and high recognition accuracy, and transforms the traditional defect detection process into an excellent defect detection and identification method based on deep learning algorithm and deep neural convolution network, which is effective Solve the shortcomings of traditional craftsmanship.

本发明设计了一款铸件缺陷检测的可视化软件,该软件可通过设置缺陷检测的网络模型,选择需要检测的缺陷图片,检测完毕后输出检测结果和检测时间,可视化结果清晰,软件操作简单易上手,极大程度的改善了当前铸件缺陷检测有人工参与弊端的现况。The invention designs a visualization software for casting defect detection. The software can select the defect pictures to be detected by setting the network model of defect detection, and output the detection results and detection time after the detection is completed. The visualization results are clear, and the software operation is simple and easy to use. , which greatly improves the current situation that the current casting defect detection has the drawbacks of manual participation.

附图说明Description of drawings

图1为本发明的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法的流程图;Fig. 1 is the flow chart of a kind of casting surface defect identification method based on improved DeepLabv3+ network model of the present invention;

图2为本发明的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法的一较佳实施例的整体网络结构的示意图;2 is a schematic diagram of the overall network structure of a preferred embodiment of a method for identifying surface defects of castings based on an improved DeepLabv3+ network model of the present invention;

图3为图2所示的整体网络结构的编码模块的示意图;Fig. 3 is the schematic diagram of the coding module of the overall network structure shown in Fig. 2;

图4为图3所示的整体网络结构的解码模块的示意图;4 is a schematic diagram of a decoding module of the overall network structure shown in FIG. 3;

图5为本发明的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法的一较佳实施例的主干模型网络结构的示意图;5 is a schematic diagram of a backbone model network structure of a preferred embodiment of a method for identifying surface defects of castings based on an improved DeepLabv3+ network model of the present invention;

图6为本发明的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法的一较佳实施例的识别结果的示意图;6 is a schematic diagram of a recognition result of a preferred embodiment of a method for recognizing casting surface defects based on an improved DeepLabv3+ network model of the present invention;

图7为本发明的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法的一较佳实施例的Qt设计软件界面的示意图。7 is a schematic diagram of a Qt design software interface of a preferred embodiment of a method for identifying surface defects of castings based on an improved DeepLabv3+ network model of the present invention.

具体实施方式Detailed ways

为使本发明要解决的技术问题、技术方案和优点更加清楚,下面将结合附图及具体实施例进行详细描述。In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

本发明针对现有的问题,提供了一种基于改进DeepLabv3+(一种语义分割模型)网络模型的铸件表面缺陷识别方法。本发明为基于神经网络的检测方法,通过改进DeepLabv3+网络模型进行铸件缺陷检测,能够实现自动实时在线以及智能化检测铸件表面缺陷。采集的工件表面缺陷图像被制作成缺陷数据集送入到改进后的神经网络中,进而通过基于语义分割的自定义网络训练学习,最终得到训练完成的网络,用于工件表面缺陷图像检测并标记缺陷区域,进而结合缺陷检测软件,输出检测的时间,同时可替换网络模型进行不同模型检测对比,在本方法的研究下工件的缺陷检测可进行智能识别,达到高精度检测、减少人工干预的目的。Aiming at the existing problems, the present invention provides a casting surface defect identification method based on an improved DeepLabv3+ (a semantic segmentation model) network model. The invention is a detection method based on a neural network, and by improving the DeepLabv3+ network model to detect casting defects, it can realize automatic real-time online and intelligent detection of casting surface defects. The collected workpiece surface defect images are made into a defect dataset and sent to the improved neural network, and then trained and learned through a self-defined network based on semantic segmentation, and finally the trained network is obtained for image detection and marking of workpiece surface defects. The defect area is then combined with the defect detection software to output the detection time. At the same time, the network model can be replaced to compare different models. Under the research of this method, the defect detection of the workpiece can be intelligently identified, so as to achieve high-precision detection and reduce manual intervention. .

如图1所示,本发明的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法包括如下步骤:As shown in Figure 1, a method for identifying surface defects of castings based on the improved DeepLabv3+ network model of the present invention includes the following steps:

步骤S1、采集铸件图像数据集,获得训练集和测试集:工业相机采集铸件图像,Labelme(一种深度学习标注工具)标注铸件缺陷,生成图像数据集,并将图像数据集分成训练集和测试集;Step S1, collecting a casting image data set to obtain a training set and a test set: the industrial camera collects casting images, Labelme (a deep learning labeling tool) marks casting defects, generates an image data set, and divides the image data set into training set and test set set;

定义铸件缺陷的标准,并使用Labelme将每张图片的缺陷区域进行标注,标注的图片包括缺陷位置和缺陷面积,形成一个常见工业铸件表面缺陷的数据集,数据集包括共244张图片,采集的图像像素大小为1408*1580。Define the standard of casting defects, and use Labelme to mark the defect area of each image. The marked images include defect location and defect area, forming a data set of common industrial casting surface defects. The data set includes a total of 244 images. The image pixel size is 1408*1580.

铸件缺陷数据集中每张缺陷图片上仅含有一类缺陷,图像的大小统一调整为1408*1580像素。进一步地,将获取的铸件数据集分成训练集和测试集,训练集是已经将缺陷标注好的图片,共211张,测试集是未标注的铸件缺陷图片,共33张。Each defect image in the casting defect dataset contains only one type of defect, and the size of the image is uniformly adjusted to 1408*1580 pixels. Further, the obtained casting data set is divided into a training set and a test set. The training set is the pictures that have been marked with defects, a total of 211 pictures, and the test set is the unlabeled pictures of casting defects, a total of 33 pictures.

步骤S2、构建网络模型,并通过训练集和测试集对网络模型进行数据训练和修正,生成缺陷检测网络;Step S2, constructing a network model, and performing data training and correction on the network model through the training set and the test set to generate a defect detection network;

步骤S3、设计缺陷检测网络的损失函数:最终损失函数为包括预测结果的交叉熵损失及真实值的交叉熵损失的函数;Step S3, designing the loss function of the defect detection network: the final loss function is a function including the cross-entropy loss of the prediction result and the cross-entropy loss of the real value;

步骤S4、所述缺陷检测网络识别并输出铸件缺陷检测结果,并显示检测时长。Step S4, the defect detection network identifies and outputs the casting defect detection result, and displays the detection duration.

步骤S2具体包括如下步骤:Step S2 specifically includes the following steps:

步骤S21、构建改进DeepLabV3+网络模型:改进现有DeepLabV3+网络模型中的编码解码模块,所述编码解码模块采用深度可分离卷积网络,并删减通道数量;Step S21, constructing and improving the DeepLabV3+ network model: improving the encoding and decoding module in the existing DeepLabV3+ network model, the encoding and decoding module adopts a depth separable convolutional network, and the number of channels is deleted;

采用改进后的DeepLabv3+网络模型构建训练用的卷积神经网络结构,改进的是DeepLabV3+网络模型中的主干模型以及编码解码模块,该主干模型以及编码解码模块通过应用具有不同比率的空洞卷积和图像级特征来获取多尺度的卷积特征,但是参数数量过于庞大,计算时间过长,因此本方法改进了编码解码模块,将编码解码部分和解码部分的普通卷积替换成深度可分离卷积,通道数量也进行了一定量的删减操作。The improved DeepLabv3+ network model is used to build the convolutional neural network structure for training. The backbone model and the encoder-decoder module in the DeepLabV3+ network model are improved. However, the number of parameters is too large and the calculation time is too long. Therefore, this method improves the encoding and decoding module, and replaces the ordinary convolution of the encoding and decoding part and the decoding part with a depthwise separable convolution. The number of channels has also undergone a certain amount of pruning.

主干网络使用的是Xception(一种网络卷积结构)网络结构,本方法为加快网络计算速度,减少铸件缺陷图片的检测时间,将Xception网络替换为MobileNet V2(一个轻量化卷积神经网络)网络。The backbone network uses the Xception (a kind of network convolution structure) network structure. In order to speed up the network calculation speed and reduce the detection time of casting defect pictures, this method replaces the Xception network with the MobileNet V2 (a lightweight convolutional neural network) network. .

步骤S22、进行数据训练和修正生成缺陷检测网络:将改进DeepLabv3+网络模型作为铸件表面缺陷识别的网络模型来提取原始采集的铸件图片的铸件表面缺陷特征,通过利用训练集中的训练数据对改进DeepLabv3+网络模型进行训练,再将测试集输入到改进DeepLabv3+网络模型中对其进行修正,直至生成的缺陷检测网络的预测精度满足预测精度阈值。训练时通过调整训练参数达到获得良好网络预测模型的目的。Step S22, perform data training and correction to generate a defect detection network: use the improved DeepLabv3+ network model as a network model for casting surface defect identification to extract the casting surface defect features of the originally collected casting images, and improve the DeepLabv3+ network by using the training data in the training set. The model is trained, and then the test set is input into the improved DeepLabv3+ network model for correction until the prediction accuracy of the generated defect detection network meets the prediction accuracy threshold. The purpose of obtaining a good network prediction model is achieved by adjusting the training parameters during training.

训练网络设置的相关参数如下:学习率设置为0.01,加载数据(batch)的线程数目设置为4,训练步数设置为40,batch-size设置为4,使用Intel I7 CPU,GPU:Nvidia_1080TI。The relevant parameters of the training network settings are as follows: the learning rate is set to 0.01, the number of threads for loading data (batch) is set to 4, the number of training steps is set to 40, and the batch-size is set to 4, using Intel I7 CPU, GPU: Nvidia_1080TI.

如图2、图3及图4所示,所述步骤S21中,所述编码解码模块包括编码模块及解码模块,所述编码模块与所述解码模块连接,所述编码模块包括深度卷积网络层、编码器及空洞空间金字塔池化模块,所述深度卷积网络层为主干模型,进行卷积后得到的低级语义特征传入所述解码模块中,所述深度卷积网络层得到的高级语义特征传入所述空洞空间金字塔池化模块。As shown in FIG. 2 , FIG. 3 and FIG. 4 , in step S21 , the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, and the encoding module includes a deep convolutional network Layer, encoder and hole space pyramid pooling module, the deep convolutional network layer is the backbone model, the low-level semantic features obtained after convolution are passed into the decoding module, and the high-level semantic features obtained by the deep convolutional network layer are Semantic features are passed into the hole spatial pyramid pooling module.

所述步骤S21中,所述编码解码模块包括编码模块及解码模块,所述编码模块与所述解码模块连接,所述编码模块包括主干模型及空洞空间金字塔池化模块,所述主干模型为深度卷积网络层MobileNet V2,输入图片通过主干模型进行卷积后得到的包括缺陷的形状信息的低级语义特征直接传入所述解码模块中,输入图片通过编码器得到的包括缺陷位置类别的高级语义特征传入所述空洞空间金字塔池化模块。图5即为主干模型网络结构的示意图。In the step S21, the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, the encoding module includes a backbone model and a hole space pyramid pooling module, and the backbone model is a depth. In the convolutional network layer MobileNet V2, the low-level semantic features including the shape information of defects obtained after the input picture is convolved through the backbone model are directly passed into the decoding module, and the high-level semantics including the defect location category are obtained from the input picture through the encoder Features are passed into the hole space pyramid pooling module. FIG. 5 is a schematic diagram of the network structure of the backbone model.

带有缺陷的图片传入到编码模块中,通过深度卷积网络层,其中的深度神经网络模型包含了串行结构的空洞卷积,得到的低级语义特征传入解码模块中,高级语义特征传入到空洞空间金字塔池化模块中。The picture with defects is passed into the encoding module, and through the deep convolutional network layer, the deep neural network model contains the convolution of the serial structure, the obtained low-level semantic features are passed into the decoding module, and the high-level semantic features are passed to the decoding module. into the hole space pyramid pooling module.

输入图片通过所述深度卷积网络层进行卷积后得到的包括缺陷形状信息的低级语义特征直接传入所述解码模块中,输入图片通过编码器及所述深度卷积网络层得到的包括缺陷位置类别的高级语义特征传入所述空洞空间金字塔池化模块,所述空洞空间金字塔池化模块包括空洞卷积模块、四层卷积层及一层池化层,所述四层卷积层与所述一层池化层串行连接,所述空洞卷积模块分别采用不同膨胀率的空洞卷积块,控制编码器提取特征的分辨率,分别进行空洞卷积后得到所述四层卷积层的四个特征图,所述空洞卷积模块进行池化后得到所述一层池化层的一个特征图。The low-level semantic features including defect shape information obtained after the input picture is convolved through the deep convolutional network layer are directly passed into the decoding module, and the input picture obtained through the encoder and the deep convolutional network layer includes defects. The high-level semantic features of the location category are passed into the hole space pyramid pooling module, and the hole space pyramid pooling module includes a hole convolution module, a four-layer convolution layer, and a pooling layer. The four-layer convolution layer It is serially connected with the one-layer pooling layer, and the atrous convolution module adopts atrous convolution blocks with different expansion rates respectively, controls the resolution of the feature extracted by the encoder, and performs atrous convolution respectively to obtain the four-layer volume. The four feature maps of the accumulation layer are obtained after the atrous convolution module performs pooling to obtain a feature map of the one-layer pooling layer.

对于所述主干模型得到的低级语义特征信息,所述编码模块通过一个卷积对低级语义特征进行操作,以减少通道数,运算后得到最终的低级语义特征的语义信息,传到所述解码模块的解码器中;对于所述空洞空间金字塔池化模块得到的高级语义特征,所述解码模块通过一次上采样得到最终的高级语义特征的语义信息,作为输出结果,传入到解码器中。For the low-level semantic feature information obtained by the backbone model, the encoding module operates on the low-level semantic feature through a convolution to reduce the number of channels, and after the operation obtains the final semantic information of the low-level semantic feature, which is sent to the decoding module In the decoder of ; for the high-level semantic features obtained by the hole space pyramid pooling module, the decoding module obtains the semantic information of the final high-level semantic features through one upsampling, and transmits it to the decoder as an output result.

底层的输出进行空洞卷积处理,通过1*1卷积调整通道数得到b0层,膨胀率为6的空洞卷积块得到b1层,利用膨胀率为12的空洞卷积块得到b2层,利用膨胀率为18的空洞卷积块得到b3层,最后将四层卷积层和一层池堆叠起来用1*1卷积进行通道调整,之后进行成中间特征层的长宽,和处理好的中间特征层进行堆叠,用3*3卷积进行最后的处理,然后进行每个像素点的分类,最后用softmax求每个像素点类别的概率。The output of the bottom layer is processed by hole convolution, and the number of channels is adjusted by 1*1 convolution to obtain the b0 layer. The hole convolution block with an expansion rate of 18 gets the b3 layer. Finally, the four convolution layers and one pool are stacked with 1*1 convolution for channel adjustment, and then the length and width of the intermediate feature layer are processed. The intermediate feature layers are stacked, the final processing is performed with 3*3 convolution, and then each pixel is classified, and finally the probability of each pixel category is calculated by softmax.

基于改进DeepLabv3+网络模型,构建了使用空洞卷积的编码-解码结构模型,在这种编码-解码架构中,引入可控制编码器提取特征的分辨率,通过采用的空洞卷积平衡精度和耗时。采用的空洞卷积可在不损失信息的情况下,在特征提取的时候跨像素提取以此来加大感受野,让每个卷积输出都包含较大范围的信息,空洞卷积用于在编码模块中进行特征提取。Based on the improved DeepLabv3+ network model, an encoding-decoding structure model using atrous convolution is constructed. In this encoding-decoding architecture, the resolution of the features extracted by the controllable encoder is introduced, and the atrous convolution is used to balance the accuracy and time-consuming. . The atrous convolution used can increase the receptive field by cross-pixel extraction during feature extraction without losing information, so that each convolution output contains a large range of information. Atrous convolution is used in Feature extraction is performed in the encoding module.

当输出步长为16的编码模块输出时,取得了速率和精度的平衡。而当输出步长为8的编码模块输出,精度更高,但计算复杂度增加。平衡各方面优势后,本方法决定采用输出步长为16的编码模块。When the output of the encoding module with stride of 16 is output, a balance of rate and accuracy is achieved. When the output step size is 8, the coding module outputs, the precision is higher, but the computational complexity increases. After balancing the advantages of various aspects, this method decides to use an encoding module with an output step size of 16.

高级语义特征进入到所述空洞金字塔池化模块编码解码,分别与四个空洞卷积层的所述四层卷积层和一个池化层的所述一层池化层进行卷积和池化,得到五个特征图,进一步的连接成五层空洞空间金字塔池化模块。The high-level semantic features are encoded and decoded by the hole pyramid pooling module, and are convolved and pooled with the four-layer convolutional layers of the four hollow convolutional layers and the one-layer pooling layer of one pooling layer respectively. , to obtain five feature maps, which are further connected into a five-layer empty space pyramid pooling module.

然而由于对应的低级语义特征包含了较多的通道信息,可能会超过输出编码特征导致训练困难,故而在连接操作前,所述编码模块通过一个1*1的卷积对低级语义特征进行操作,以减少通道数,运算后得到最终的语义信息,所述解码模块通过一次上采样得到的输出结果传入到解码器中。However, since the corresponding low-level semantic features contain more channel information, it may exceed the output encoding features and cause training difficulties. Therefore, before the connection operation, the encoding module operates on the low-level semantic features through a 1*1 convolution. In order to reduce the number of channels, the final semantic information is obtained after the operation, and the output result obtained by the decoding module through one upsampling is passed to the decoder.

所述编码模块还包括堆叠层、通道调整卷积层、中间特征层、中间特征层堆叠层、后处理卷积层及后处理模块,所述堆叠层为所述四层卷积层及所述一层池化层获得的特征图进行堆叠起来形成,对所述堆叠层进行卷积以调整通道,形成所述通道调整卷积层,将所述通道调整卷积层进行长宽调整,形成所述中间特征层,所述中间特征层堆叠后形成所述中间特征层堆叠层,所述中间特征层堆叠层进行卷积后形成所述后处理卷积层,再对所述后处理卷积层通过所述后处理模块进行后处理。The encoding module further includes a stacking layer, a channel adjustment convolutional layer, an intermediate feature layer, an intermediate feature layer stacking layer, a post-processing convolutional layer, and a post-processing module, where the stacking layer is the four-layer convolutional layer and the The feature maps obtained by a layer of pooling layers are stacked to form, the stacked layers are convoluted to adjust the channel, the channel adjustment convolution layer is formed, and the channel adjustment convolution layer is adjusted in length and width to form the channel adjustment convolution layer. The intermediate feature layer is stacked to form the intermediate feature layer stack, the intermediate feature layer stack is convoluted to form the post-processing convolution layer, and then the post-processing convolution layer is formed. Post-processing is performed by the post-processing module.

所述后处理模块包括像素点分类模块及像素点类别概率计算模块,所述像素点分类模块对所述后处理卷积层的图像进行每个像素点的分类,所述像素点类别概率计算模块采用softmax(一种输出层的激励函数)求解每个像素点类别的概率。The post-processing module includes a pixel point classification module and a pixel point category probability calculation module, the pixel point classification module classifies each pixel point on the image of the post-processing convolution layer, and the pixel point category probability calculation module. Use softmax (an excitation function of the output layer) to solve the probability of each pixel category.

所述四层卷积层和所述一层池化层串行连接后进行1*1卷积操作后作为高级语义特征信息传入到所述解码模块,经过一次四倍上采样,与所述编码模块中的主干模型的所述深度神经网络层一起,传入到所述解码模块中。The four-layer convolutional layer and the one-layer pooling layer are serially connected and then subjected to a 1*1 convolution operation and then transmitted to the decoding module as high-level semantic feature information. The deep neural network layers of the backbone model in the encoding module are passed into the decoding module together.

所述编码模块传入到所述解码模块的高级语义特征进行1*1卷积操作后的结果和主干模型传出的低级语义特征信息连接在一起组成并行结构,后进行一次3*3卷积操作,通过一次四倍的上采样,进一步的组成了所述解码模块,最后传出预测结果。The result of the 1*1 convolution operation on the high-level semantic features passed into the decoding module by the encoding module and the low-level semantic feature information from the backbone model are connected together to form a parallel structure, and then a 3*3 convolution is performed. Operation, through a quadruple upsampling, the decoding module is further formed, and finally the prediction result is sent out.

具体地,所述解码模块将所述编码器输出的高级语义特征采用采样因子为4的双线性上采样进行上采样,所述解码模块的输入由两部分组成,一部分来自主干模型的所述深度卷积网络层的直接输出,连接从主干模型所输出的对应的具有相同空间分辨率的低级语义特征,另一部分通过在主干模型的深度卷积网络层导入至所述空洞空间金字塔池化模块中,找到一个与编码器输出的语义特征分辨率相同的低级语义特征图,经过1*1卷积进行降通道数使之与编码器输出的语义特征所占通道比重一样,并将所述解码模块两部分输入分别得到的低级语义特征和低级语义特征图连接在一起,在连接处理后,然后再通过一个3*3细化卷积进行细化,后通过采样因子为4的双线性上采样得到最终的预测结果。Specifically, the decoding module upsamples the high-level semantic features output by the encoder using bilinear upsampling with a sampling factor of 4, and the input of the decoding module consists of two parts, one part comes from the backbone model of the The direct output of the deep convolutional network layer is connected to the corresponding low-level semantic features with the same spatial resolution output from the backbone model, and the other part is imported into the hole spatial pyramid pooling module through the deep convolutional network layer of the backbone model. , find a low-level semantic feature map with the same resolution as the semantic feature output by the encoder, reduce the number of channels through 1*1 convolution to make it equal to the channel proportion of the semantic feature output by the encoder, and decode the The low-level semantic features and low-level semantic feature maps obtained from the input of the two parts of the module are connected together. After the connection is processed, it is refined by a 3*3 thinning convolution, and then the bilinear sampling factor is 4. Sampling to get the final prediction result.

所述编码模块传入到所述解码模块的输出采用的上采样以及所述解码模块输出采用的上采样采用的均是双线性插值法,定义公式如下:The upsampling adopted by the output of the encoding module input to the decoding module and the upsampling adopted by the output of the decoding module are both bilinear interpolation, and the definition formula is as follows:

srcx=desx*srcω/desω src x =des x *src ω /des ω

srcy=desy*srch/desh src y =des y *src h /des h

式中,srcx表示原图像中像素点x坐标,srcy表示原图像中像素点y坐标,desx表示目标图像中像素点x坐标,desy表示目标图像中像素点y坐标,srcω表示原图像宽度,srch表示原图像高度,desω表示目标图像宽度,desh表示目标图像高度。In the formula, src x represents the x-coordinate of the pixel in the original image, src y represents the y-coordinate of the pixel in the original image, des x represents the x-coordinate of the pixel in the target image, des y represents the y-coordinate of the pixel in the target image, and src ω represents The width of the original image, src h is the height of the original image, des ω is the width of the target image, and des h is the height of the target image.

所述步骤S3中损失函数使用交叉熵损失函数,公式如下:In the step S3, the loss function uses the cross-entropy loss function, and the formula is as follows:

J=-[y·log(p)+(1-y)·log(1-p)]J=-[y·log(p)+(1-y)·log(1-p)]

其中y表示样本的label,正类为1,负类为0;p表示样本预测为正的概率。Where y represents the label of the sample, the positive class is 1, and the negative class is 0; p represents the probability that the sample is predicted to be positive.

所述步骤S4中所述输出铸件检测结果具体为通过面对工业铸件缺陷检测的可视化软件实现输出检测结果。The outputting of the casting detection results in the step S4 is specifically to realize the outputting of the detection results through visualization software for industrial casting defect detection.

如图6及图7所示,此软件输入的是具有缺陷的工业铸件图像,可添加不同的识别模型,最终输出缺陷检测的结果图像,操作简便,可视化结果清晰,可供用户直接检测识别缺陷铸件图片。可视化软件为一款面对工业铸件缺陷检测的软件,是基于Qt的一款可视化工业软件,具有操作简单,可视化清晰以及显示缺陷检测的检测时间和任选检测网络的特点。As shown in Figure 6 and Figure 7, this software inputs images of industrial castings with defects. Different recognition models can be added, and the final image of defect detection results is output. The operation is simple and the visualization results are clear. Users can directly detect and identify defects. Casting pictures. Visualization software is a software for industrial casting defect detection. It is a visual industrial software based on Qt. It has the characteristics of simple operation, clear visualization, and displaying the detection time of defect detection and optional detection network.

本发明能够取得下列有益效果:本发明所改进的基于DeepLabv3+网络模型可用于工业铸件缺陷识别,将此识别网络可直接应用于工业铸件缺陷检测领域,在进行识别后产生缺陷识别的数字化缺陷数据,结果存储和查询可与铸件信息一一对应,提供了一定的数据反馈,该反馈可用于铸造工艺的优化,为后续的工艺提供指导,特别地指出,本方法主要以工业铸件打磨缺陷处为目标,进行铸件的缺陷检测识别。The present invention can achieve the following beneficial effects: the improved DeepLabv3+ network model based on the present invention can be used for industrial casting defect identification, and the identification network can be directly applied to the field of industrial casting defect detection, and digital defect data for defect identification is generated after identification, The result storage and query can correspond one-to-one with the casting information, providing a certain data feedback, which can be used for the optimization of the casting process and provide guidance for the subsequent process. In particular, it is pointed out that this method mainly targets the grinding defects of industrial castings , for casting defect detection and identification.

本发明采用的基于改进DeepLabV3+的网络模型,具有识别速度快,识别精确度高的优点,将传统的缺陷检测工艺改造为优良的基于深度学习算法和深度神经卷积网络的缺陷检测识别方法,有效解决了传统工艺的缺点。The network model based on the improved DeepLabV3+ adopted in the present invention has the advantages of fast recognition speed and high recognition accuracy, and transforms the traditional defect detection process into an excellent defect detection and identification method based on deep learning algorithm and deep neural convolution network, which is effective Solve the shortcomings of traditional craftsmanship.

本发明设计了一款铸件缺陷检测的可视化软件,该软件可通过设置缺陷检测的网络模型,选择需要检测的缺陷图片,检测完毕后输出检测结果和检测时间,可视化结果清晰,软件操作简单易上手,极大程度的改善了当前铸件缺陷检测有人工参与弊端的现况。The invention designs a visualization software for casting defect detection. The software can select the defect pictures to be detected by setting the network model of defect detection, and output the detection results and detection time after the detection is completed. The visualization results are clear, and the software operation is simple and easy to use. , which greatly improves the current situation that the current casting defect detection has the drawbacks of manual participation.

以上所述是本发明的优选实施方式,应当指出,对于本技术领域的普通技术人员来说,在不脱离本发明所述原理的前提下,还可以作出若干改进和润饰,这些改进和润饰也应视为本发明的保护范围。The above are the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (5)

1.一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法,其特征在于,包括如下步骤:1. a casting surface defect identification method based on improving DeepLabv3+ network model, is characterized in that, comprises the steps: 步骤S1、采集铸件图像数据集,获得训练集和测试集:工业相机采集铸件图像,Labelme标注铸件缺陷,生成图像数据集,并将图像数据集分成训练集和测试集;Step S1, collecting a casting image data set to obtain a training set and a test set: the industrial camera collects the casting image, Labelme marks the casting defects, generates an image data set, and divides the image data set into a training set and a test set; 步骤S2、构建网络模型,并通过训练集和测试集对网络模型进行数据训练和修正,生成缺陷检测网络;Step S2, constructing a network model, and performing data training and correction on the network model through the training set and the test set to generate a defect detection network; 步骤S3、设计缺陷检测网络的损失函数:损失函数为包括预测结果的交叉熵损失及真实值的交叉熵损失的函数;Step S3, designing the loss function of the defect detection network: the loss function is a function including the cross-entropy loss of the prediction result and the cross-entropy loss of the real value; 步骤S4、所述缺陷检测网络识别并输出铸件缺陷检测结果,并显示检测时长;Step S4, the defect detection network identifies and outputs the casting defect detection result, and displays the detection duration; 步骤S2具体包括如下步骤:Step S2 specifically includes the following steps: 步骤S21、构建改进DeepLabV3+网络模型:改进现有DeepLabV3+网络模型中的编码解码模块,所述编码解码模块采用深度可分离卷积网络,并删减通道数量;Step S21, constructing and improving the DeepLabV3+ network model: improving the encoding and decoding module in the existing DeepLabV3+ network model, the encoding and decoding module adopts a depth separable convolutional network, and the number of channels is deleted; 步骤S22、进行数据训练和修正生成缺陷检测网络:将改进DeepLabv3+网络模型作为铸件表面缺陷识别的网络模型来提取原始采集的铸件图片的铸件表面缺陷特征,通过利用训练集中的训练数据对改进DeepLabv3+网络模型进行训练,再将测试集输入到改进DeepLabv3+网络模型中对其进行修正,直至生成的缺陷检测网络的预测精度满足预测精度阈值;Step S22, perform data training and correction to generate a defect detection network: use the improved DeepLabv3+ network model as a network model for casting surface defect identification to extract the casting surface defect features of the originally collected casting images, and improve the DeepLabv3+ network by using the training data in the training set. The model is trained, and then the test set is input into the improved DeepLabv3+ network model to correct it, until the prediction accuracy of the generated defect detection network meets the prediction accuracy threshold; 所述步骤S21中,所述编码解码模块包括编码模块及解码模块,所述编码模块与所述解码模块连接,所述编码模块包括深度卷积网络层、编码器及空洞空间金字塔池化模块,所述深度卷积网络层为主干模型,输入图片通过所述深度卷积网络层进行卷积后得到的包括缺陷形状信息的低级语义特征直接传入所述解码模块中,输入图片通过编码器及所述深度卷积网络层得到的包括缺陷位置类别的高级语义特征传入所述空洞空间金字塔池化模块,所述空洞空间金字塔池化模块包括空洞卷积模块、四层卷积层及一层池化层,所述四层卷积层与所述一层池化层串行连接,所述空洞卷积模块分别采用不同膨胀率的空洞卷积块,控制编码器提取特征的分辨率,分别进行空洞卷积后得到所述四层卷积层的四个特征图,所述空洞卷积模块进行池化后得到所述一层池化层的一个特征图;In the step S21, the encoding and decoding module includes an encoding module and a decoding module, the encoding module is connected to the decoding module, and the encoding module includes a deep convolutional network layer, an encoder and a hole space pyramid pooling module, The deep convolutional network layer is the backbone model, and the low-level semantic features including defect shape information obtained by convolution of the input picture through the deep convolutional network layer are directly transferred to the decoding module, and the input picture is passed through the encoder and the decoder. The high-level semantic features including defect location categories obtained by the deep convolutional network layer are passed into the hole space pyramid pooling module, and the hole space pyramid pooling module includes a hole convolution module, a four-layer convolution layer and a layer of Pooling layer, the four-layer convolution layer is serially connected with the one-layer pooling layer, and the hole convolution modules respectively use hole convolution blocks with different expansion rates to control the resolution of the features extracted by the encoder, respectively. After the hole convolution is performed, four feature maps of the four-layer convolution layer are obtained, and the hole convolution module is pooled to obtain a feature map of the one-layer pooling layer; 对于所述主干模型得到的低级语义特征信息,所述编码模块通过一个卷积对低级语义特征进行操作,以减少通道数,运算后得到最终的低级语义特征的语义信息,传到所述解码模块的解码器中;对于所述空洞空间金字塔池化模块得到的高级语义特征,所述解码模块通过一次上采样得到最终的高级语义特征的语义信息,作为输出结果,传入到解码器中;For the low-level semantic feature information obtained by the backbone model, the encoding module operates on the low-level semantic feature through a convolution to reduce the number of channels, and after the operation obtains the final semantic information of the low-level semantic feature, which is sent to the decoding module In the decoder; for the high-level semantic features obtained by the hole space pyramid pooling module, the decoding module obtains the semantic information of the final high-level semantic features through one-time upsampling, which is passed into the decoder as an output result; 所述解码模块将所述编码器输出的高级语义特征采用采样因子为4的双线性上采样进行上采样,所述解码模块的输入由两部分组成,一部分来自主干模型的所述深度卷积网络层的直接输出,连接从主干模型所输出的对应的具有相同空间分辨率的低级语义特征,另一部分通过在主干模型的深度卷积网络层导入至所述空洞空间金字塔池化模块中,找到一个与编码器输出的语义特征分辨率相同的低级语义特征图,经过1*1卷积进行降通道数使之与编码器输出的语义特征所占通道比重一样,并将所述解码模块两部分输入分别得到的低级语义特征和低级语义特征图连接在一起,在连接处理后,然后再通过一个3*3细化卷积进行细化,后通过采样因子为4的双线性上采样得到最终的预测结果;The decoding module upsamples the high-level semantic features output by the encoder using bilinear upsampling with a sampling factor of 4, and the input of the decoding module consists of two parts, one part from the depth convolution of the backbone model. The direct output of the network layer is connected to the corresponding low-level semantic features with the same spatial resolution output from the backbone model, and the other part is imported into the hole spatial pyramid pooling module through the deep convolutional network layer of the backbone model. A low-level semantic feature map with the same resolution as the semantic feature output by the encoder, after 1*1 convolution, the number of channels is reduced to make it equal to the channel proportion of the semantic feature output by the encoder, and the decoding module is divided into two parts. The low-level semantic features and low-level semantic feature maps obtained from the input are connected together. After the connection processing, they are then refined by a 3*3 thinning convolution, and then the final result is obtained by bilinear upsampling with a sampling factor of 4. the forecast result; 所述编码模块传入到所述解码模块的输出采用的上采样以及所述解码模块输出采用的上采样采用的均是双线性插值法,定义公式如下:The upsampling adopted by the output of the encoding module input to the decoding module and the upsampling adopted by the output of the decoding module are both bilinear interpolation, and the definition formula is as follows: srcx=desx*srcω/desω src x =des x *src ω /des ω srcy=desy*srch/desh src y =des y *src h /des h 式中,srcx表示原图像中像素点x坐标,srcy表示原图像中像素点y坐标,desx表示目标图像中像素点x坐标,desy表示目标图像中像素点y坐标,srcω表示原图像宽度,srch表示原图像高度,desω表示目标图像宽度,desh表示目标图像高度。In the formula, src x represents the x-coordinate of the pixel in the original image, src y represents the y-coordinate of the pixel in the original image, des x represents the x-coordinate of the pixel in the target image, des y represents the y-coordinate of the pixel in the target image, and src ω represents The width of the original image, src h is the height of the original image, des ω is the width of the target image, and des h is the height of the target image. 2.根据权利要求1所述的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法,其特征在于,所述步骤S4中所述输出铸件检测结果具体为通过面对工业铸件缺陷检测的可视化软件实现输出检测结果。2. a kind of casting surface defect identification method based on improved DeepLabv3+ network model according to claim 1, is characterized in that, described in described step S4, output casting detection result is specifically by the visualization software that faces industrial casting defect detection Realize the output detection result. 3.根据权利要求1所述的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法,其特征在于,所述编码模块还包括堆叠层、通道调整卷积层、中间特征层、中间特征层堆叠层、后处理卷积层及后处理模块,所述堆叠层为所述四层卷积层及所述一层池化层获得的特征图进行堆叠起来形成,对所述堆叠层进行卷积以调整通道,形成所述通道调整卷积层,将所述通道调整卷积层进行长宽调整,形成所述中间特征层,所述中间特征层堆叠后形成所述中间特征层堆叠层,所述中间特征层堆叠层进行卷积后形成所述后处理卷积层,再对所述后处理卷积层通过所述后处理模块进行后处理。3. a kind of casting surface defect identification method based on improving DeepLabv3+ network model according to claim 1, is characterized in that, described coding module also comprises stacking layer, channel adjustment convolution layer, middle feature layer, middle feature layer stacking layer, post-processing convolutional layer and post-processing module, the stacking layer is formed by stacking the feature maps obtained by the four-layer convolutional layer and the one-layer pooling layer, and convolving the stacked layer to obtain Adjust the channel to form the channel adjustment convolution layer, and adjust the length and width of the channel adjustment convolution layer to form the intermediate feature layer. After the intermediate feature layers are stacked, the intermediate feature layer stack is formed. The post-processing convolution layer is formed after the stacking layers of the intermediate feature layers are convoluted, and then the post-processing convolution layer is post-processed by the post-processing module. 4.根据权利要求3所述的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法,其特征在于,所述后处理模块包括像素点分类模块及像素点类别概率计算模块,所述像素点分类模块对所述后处理卷积层的图像进行每个像素点的分类,所述像素点类别概率计算模块采用softmax求解每个像素点类别的概率。4. a kind of casting surface defect identification method based on improved DeepLabv3+ network model according to claim 3, is characterized in that, described post-processing module comprises pixel classification module and pixel classification probability calculation module, and described pixel classification The module classifies each pixel point on the image of the post-processing convolution layer, and the pixel point category probability calculation module uses softmax to calculate the probability of each pixel point category. 5.根据权利要求1所述的一种基于改进DeepLabv3+网络模型的铸件表面缺陷识别方法,其特征在于,所述步骤S3中损失函数使用交叉熵损失函数,公式如下:5. a kind of casting surface defect identification method based on improving DeepLabv3+ network model according to claim 1, is characterized in that, in described step S3, loss function uses cross entropy loss function, and formula is as follows: J=-[y·log(p)+(1-y)·log(1-p)]J=-[y·log(p)+(1-y)·log(1-p)] 其中y表示样本的label,正类为1,负类为0;p表示样本预测为正的概率。Where y represents the label of the sample, the positive class is 1, and the negative class is 0; p represents the probability that the sample is predicted to be positive.
CN202110317123.8A 2021-03-25 2021-03-25 A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model Active CN112967271B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110317123.8A CN112967271B (en) 2021-03-25 2021-03-25 A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110317123.8A CN112967271B (en) 2021-03-25 2021-03-25 A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model

Publications (2)

Publication Number Publication Date
CN112967271A CN112967271A (en) 2021-06-15
CN112967271B true CN112967271B (en) 2022-04-19

Family

ID=76278400

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110317123.8A Active CN112967271B (en) 2021-03-25 2021-03-25 A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model

Country Status (1)

Country Link
CN (1) CN112967271B (en)

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114332829A (en) * 2021-12-06 2022-04-12 江苏航天大为科技股份有限公司 Driver fatigue detection method based on multiple strategies
CN114332047B (en) * 2021-12-31 2024-09-06 华中科技大学 A method for constructing a surface defect detection model and its application
CN116524383A (en) * 2022-01-20 2023-08-01 北京字跳网络技术有限公司 Handle positioning method, device, wearable device, storage medium and program product
CN114529534A (en) * 2022-02-22 2022-05-24 青岛科技大学 Full-automatic coreless motor winding machine product defect detection method based on machine vision
CN114708493A (en) * 2022-02-26 2022-07-05 上海大学 Traditional Chinese medicine crack tongue diagnosis portable device and using method
CN114441552A (en) * 2022-02-28 2022-05-06 中国船舶重工集团公司第七0七研究所 High-precision optical fiber loop glue filling defect detection and correction method
CN115423740A (en) * 2022-06-17 2022-12-02 兰州交通大学 Method for identifying size of internal defect area of EPR vehicle-mounted cable terminal
CN115249243A (en) * 2022-08-15 2022-10-28 南京耘瞳科技有限公司 Equipment and method for surface defect detection of automobile engine castings based on machine vision
CN115830316A (en) * 2022-11-23 2023-03-21 齐鲁工业大学 Nematode image segmentation method and system based on deep learning and iterative feature fusion
CN116703834B (en) * 2023-05-22 2024-01-23 浙江大学 Method and device for judging and grading excessive sintering ignition intensity based on machine vision

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108986091A (en) * 2018-07-12 2018-12-11 太原科技大学 Casting defect image detecting method based on depth Hash network
CN109389599A (en) * 2018-10-25 2019-02-26 北京阿丘机器人科技有限公司 A kind of defect inspection method and device based on deep learning
CN110222681A (en) * 2019-05-31 2019-09-10 华中科技大学 A kind of casting defect recognition methods based on convolutional neural networks
CN111223088A (en) * 2020-01-16 2020-06-02 东南大学 A casting surface defect recognition method based on deep convolutional neural network

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8204291B2 (en) * 2007-10-15 2012-06-19 General Electric Company Method and system for identifying defects in a radiographic image of a scanned object

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108986091A (en) * 2018-07-12 2018-12-11 太原科技大学 Casting defect image detecting method based on depth Hash network
CN109389599A (en) * 2018-10-25 2019-02-26 北京阿丘机器人科技有限公司 A kind of defect inspection method and device based on deep learning
CN110222681A (en) * 2019-05-31 2019-09-10 华中科技大学 A kind of casting defect recognition methods based on convolutional neural networks
CN111223088A (en) * 2020-01-16 2020-06-02 东南大学 A casting surface defect recognition method based on deep convolutional neural network

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Research on Automatic Recognition of Casting Defects Based on Deep Learning;limin duan;《IEEE:Research on Automatic Recognition of Casting Defects Based on Deep Learning》;IEEE;20210122;第9卷;全文 *
基于Mask R-CNN的铸件X射线DR图像缺陷检测研究;蔡彪等;《仪器仪表学报》;20200315(第03期);全文 *

Also Published As

Publication number Publication date
CN112967271A (en) 2021-06-15

Similar Documents

Publication Publication Date Title
CN112967271B (en) A Casting Surface Defect Recognition Method Based on Improved DeepLabv3+ Network Model
CN112862811B (en) Method, equipment and device for material microscopic image defect recognition based on deep learning
CN110570396B (en) A deep learning-based industrial product defect detection method
CN111223088B (en) A casting surface defect recognition method based on deep convolutional neural network
CN111882620B (en) Road drivable area segmentation method based on multi-scale information
CN111028217A (en) Image crack segmentation method based on full convolution neural network
CN111932511B (en) A method and system for quality detection of electronic components based on deep learning
CN110473173A (en) A kind of defect inspection method based on deep learning semantic segmentation
CN108710919A (en) A kind of crack automation delineation method based on multi-scale feature fusion deep learning
CN113393438B (en) A resin lens defect detection method based on convolutional neural network
CN116402821B (en) A method for detecting quality defects in aircraft skin glue coating based on neural network
CN109035233A (en) Visual attention network and Surface Flaw Detection method
CN114972316A (en) Battery case end surface defect real-time detection method based on improved YOLOv5
CN116883393B (en) Metal surface defect detection method based on anchor frame-free target detection algorithm
CN114331869B (en) Dam face crack semantic segmentation method
CN117451716A (en) A method for detecting surface defects of industrial products
CN109815957A (en) A text recognition method based on color image in complex background
CN117576015A (en) Weld defect visual detection method based on improved YOLOv5s network
CN115810191A (en) Pathological cell classification method based on multi-attention fusion and high-precision segmentation network
CN112614113A (en) Strip steel defect detection method based on deep learning
CN118485631A (en) A glass bead defect detection method based on improved spatial pyramid pooling
CN116958074A (en) Object surface defect detection method based on yolov7
CN114429461A (en) A cross-scenario strip surface defect detection method based on domain adaptation
CN118887206A (en) A lightweight defect detection method based on contrastive knowledge distillation
CN120374598A (en) Metal defect identification method for training of small quantity of training samples

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant