[go: up one dir, main page]

CN114882400B - An aggregate detection and classification method based on AI intelligent machine vision technology - Google Patents

An aggregate detection and classification method based on AI intelligent machine vision technology Download PDF

Info

Publication number
CN114882400B
CN114882400B CN202210461094.7A CN202210461094A CN114882400B CN 114882400 B CN114882400 B CN 114882400B CN 202210461094 A CN202210461094 A CN 202210461094A CN 114882400 B CN114882400 B CN 114882400B
Authority
CN
China
Prior art keywords
image
aggregate
detection
gradient
machine vision
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210461094.7A
Other languages
Chinese (zh)
Other versions
CN114882400A (en
Inventor
袁胜
杜钰莹
上官林建
黄伟
郭灿波
马军旭
刘明堂
胡旭峰
汪璐
岳爽
李斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Henan Sanhe Hydraulic Machinery Group Co ltd
Henan Sanhe Hydraulic New Building Material Machinery Co ltd
Zhengzhou Sanhe Hydraulic Machinery Co ltd
North China University of Water Resources and Electric Power
Original Assignee
Henan Sanhe Hydraulic Machinery Group Co ltd
Henan Sanhe Hydraulic New Building Material Machinery Co ltd
Zhengzhou Sanhe Hydraulic Machinery Co ltd
North China University of Water Resources and Electric Power
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Henan Sanhe Hydraulic Machinery Group Co ltd, Henan Sanhe Hydraulic New Building Material Machinery Co ltd, Zhengzhou Sanhe Hydraulic Machinery Co ltd, North China University of Water Resources and Electric Power filed Critical Henan Sanhe Hydraulic Machinery Group Co ltd
Priority to CN202210461094.7A priority Critical patent/CN114882400B/en
Publication of CN114882400A publication Critical patent/CN114882400A/en
Application granted granted Critical
Publication of CN114882400B publication Critical patent/CN114882400B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20112—Image segmentation details
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A10/00—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE at coastal zones; at river basins
    • Y02A10/40—Controlling or monitoring, e.g. of flood or hurricane; Forecasting, e.g. risk assessment or mapping

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Mathematical Physics (AREA)
  • Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Probability & Statistics with Applications (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Quality & Reliability (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种基于AI智能机器视觉技术的骨料检测分类方法，包括以下步骤：S1，使用高清摄像头拍摄骨料运输视频，并通过5G网络进行实时传输，获得骨料图像；S2，通过图像边缘分割处理方法进行对所述骨料图像处理，提取边缘特征；S3，对处理后的骨料图像的骨料类别进行手工标注；S4，将标注后的骨料图像输入机器视觉算法中，进行训练，进行特征提取，输出为权重文件；S5，将训练好的权重放在检测系统里，对骨料进行检测识别。通过使用摄像头通过5G网络实时获取骨料图像，应用计算机图像处理技术对视频进行处理，结合AI智能算法对不同种类骨料状态图像进行检测和识别，以解决现有骨料监测方法操作复杂的问题，同时也实现了实时性处理的功能。

The present invention discloses an aggregate detection and classification method based on AI intelligent machine vision technology, comprising the following steps: S1, using a high-definition camera to shoot aggregate transportation video, and transmitting it in real time through a 5G network to obtain an aggregate image; S2, processing the aggregate image through an image edge segmentation processing method to extract edge features; S3, manually marking the aggregate category of the processed aggregate image; S4, inputting the marked aggregate image into a machine vision algorithm, training it, extracting features, and outputting it as a weight file; S5, placing the trained weights in a detection system to detect and identify the aggregate. Aggregate images are obtained in real time through a 5G network using a camera, computer image processing technology is applied to process the video, and different types of aggregate status images are detected and identified in combination with an AI intelligent algorithm to solve the problem of complex operation of existing aggregate monitoring methods, while also realizing the function of real-time processing.

Description

Aggregate detection and classification method based on AI intelligent machine vision technology
Technical Field
The invention belongs to the technical field of aggregate detection, and particularly relates to an aggregate detection and classification method based on an AI intelligent machine vision technology.
Background
Aggregate is an important component of concrete, the problem of sand aggregate classification is an important factor for determining the performance quality of concrete, common concrete (simply referred to as concrete) is composed of cement, sand, stone and water, and the aggregate generally accounts for 70% -80% of the concrete. The factors influencing the strength of concrete are many, mainly the influence of cement strength and water-cement ratio, the influence of aggregate gradation and shape, the influence of curing temperature, the influence of curing period and the like. The aggregate processing system is one of main auxiliary production systems in large-scale hydraulic and hydroelectric engineering construction, and the quality improvement of the sand aggregate has important significance for promoting the benign development of engineering construction industry, and is also extremely important for improving engineering quality and optimizing engineering cost. Different sand aggregates have different effects on the performance of concrete. For the particle size and shape of the aggregate, the specification of coarse aggregate needle-shaped particles in the current specification is wider, and the aggregate with good quality needs to have the standard particle size and shape. Therefore, the quality requirement of the sand aggregate is guaranteed, and the raw materials are reasonably selected, so that the quality of the concrete can be guaranteed. It is therefore particularly important to find a suitable aggregate classification detection method.
The problem of sand aggregate classification is an important working precondition for solving the proportion of concrete raw materials, and a plurality of image processing related technical problems are involved in working preparation and operation. The aggregate detection effect can be affected by various aggregate types, complex detection conditions, inconsistent detection distance, weather reasons, light changes, dry and wet changes, distance changes and the like in the concrete raw materials. Finding aggregate characteristics is one of the important problems, and a detection method suitable for identifying aggregate types needs to be found. The traditional aggregate detection method generally comprises screening method, particle size identification and the like, but the methods have limitations on image identification and real-time processing, are time-consuming and labor-consuming, are difficult to solve actual problems, do not have the requirements of rapidness, accuracy, real-time processing and the like, and cannot be comprehensively popularized and applied. In recent years, aggregate detection and identification techniques have developed methods with modern technology such as application of convolutional neural networks, imaging methods, measurement using backlights and high-resolution cameras, and application profiles based on laser detection analysis. Because of the application conditions and limitations of the various methods, any single method is not universal, and sometimes in practice, if only one parameter is taken as the basis of explanation, the expected effect can not be obtained easily, so that a comprehensive aggregate detection method is generally adopted in practice to improve the accuracy of aggregate detection.
Along with the development of scientific technology, the aggregate detection method and the detection equipment thereof have all made great progress and development, and the technical level of aggregate detection is continuously improved. However, various detection methods have application premises and limitations, and in practical application, most of the current methods are only to modernize instruments, and the methods are not innovative, so that a single method is sometimes difficult to obtain better effects in aggregate detection. Therefore, in the application of aggregate detection, a scientific and technological person should select a reasonable aggregate detection combination method according to the combination of the aggregate image characteristics and the machine vision technology.
At present, no special real-time monitoring design scheme aiming at aggregate detection exists in the concrete manufacturing engineering. In engineering construction, high-definition industrial monitoring equipment is generally installed, and the position and unloading state of an aggregate transport vehicle can be monitored. However, the existing monitoring equipment is generally distributed at the upper end of the transport vehicle, can only monitor, can not recognize and process images, and in addition, after the monitoring data are acquired, only human eyes of workers are required to judge aggregate types, and more effective processing and analysis are not performed.
Based on the defects existing in the prior art, the invention provides an aggregate detection and classification method based on an AI intelligent machine vision technology, which is characterized in that an aggregate image is acquired in real time through a 5G network by using a camera, then a computer image processing technology is applied to process videos, and the AI intelligent algorithm is combined to detect and identify different aggregate state images, so that the problem of complex operation of the existing aggregate monitoring method is solved, and the real-time processing function is realized.
Disclosure of Invention
Aiming at the defects of the prior art, the invention aims to provide an aggregate detection and classification method based on an AI intelligent machine vision technology.
The invention provides the following technical scheme:
an aggregate detection and classification method based on an AI intelligent machine vision technology comprises the following steps:
S1, shooting aggregate transportation videos by using a high-definition camera, and transmitting the aggregate transportation videos in real time through a 5G network to obtain aggregate images;
S2, processing the aggregate image by an image edge segmentation processing method, and extracting edge characteristics;
s3, manually marking the aggregate category of the processed aggregate image;
S4, inputting the marked aggregate image into a machine vision algorithm, training, extracting features, and outputting the features as a weight file;
s5, replaying the trained weight in a detection system, and detecting and identifying the aggregate.
Preferably, in step S2, the image edge segmentation processing method specifically includes the following steps:
A, importing a bone material image, carrying out graying treatment on the bone material image, and then solving a gradient map;
B, processing the image by using a watershed algorithm on the basis of the gradient map to obtain edge lines of the segmented image;
c, using image opening operation to remove small objects in the picture and burr interference affecting the target;
And D, carrying out adhesion segmentation, and segmenting out objects adhered together.
Preferably, in step S4, the machine vision algorithm training process specifically includes the following steps:
a, selecting 5 detection areas for each marked aggregate image;
b, respectively carrying out image convolution on each detection area to extract image features;
c, carrying out up-sampling treatment on the image, and restoring the convolved feature map into the image;
d, integrating the image features by using a tensor stitching algorithm;
e, outputting the image to a detection head, wherein the detection head part classifies the image, and judging the classification result of the identification image through a probability function;
f, identifying the result and outputting the detection result of the image.
Preferably, in the step a, the 5 detection areas are respectively the center point, the upper left, the upper right, the lower left and the lower right areas of the aggregate image.
Preferably, in step b, the image features include edge features, color features and texture features of the aggregate.
Preferably, in step c, the image is up-sampled so that the image conforms to the size of the display area, and an interpolation method is adopted, that is, a suitable interpolation algorithm is adopted between pixel points based on the original image pixels to insert new elements.
Preferably, in step C, the gradient image is thresholded to reduce over-segmentation due to gray scale variation using a processing method that modifies the gradient function so that the basin is only responsive to the target to be detected.
S6, testing a new image, judging whether the sample size is sufficient or not according to the test condition, keeping the sample size as uniform as possible, if the accuracy of the weight to a certain category is lower than 90%, increasing the corresponding sample size, and re-extracting the characteristics until the identification rate of each category of aggregate reaches more than 90%, and meeting the requirements.
Preferably, in step B, the watershed calculation process is divided into two steps, one is a sequencing process and the other is a flooding process. The gray level of each pixel is firstly ordered from low to high, and then in the process of realizing flooding from low to high, the influence domain of each local minimum value at the h-order height is judged and marked by adopting a first-in first-out (FIFO) structure. The watershed transformation is to obtain a water collecting basin image of the input image, and boundary points among the water collecting basins are the watershed. Obviously, watershed represents the input image maximum point. Therefore, to obtain edge information of an image, a gradient image is generally used as an input image, i.e
g(x,y)=grad(f(x,y))={[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5
Where f (x, y) represents the original image and grad represents the gradient operation.
Preferably, in step C, to reduce the over-segmentation generated by the watershed algorithm, the gradient function is modified, and a simple method is to thresholde the gradient image to eliminate the over-segmentation generated by small changes in gray scale. I.e.
g(x,y)=max(grad(f(x,y)),gθ)
Where gθ represents a threshold value.
The program adopts the method that the threshold value is used for limiting the gradient image so as to eliminate excessive segmentation caused by tiny change of gray values, a proper amount of areas are obtained, gray levels of edge points of the areas are ordered from low to high, then the submerged process is realized from low to high, and the gradient image is obtained by calculating with a Sobel operator.
The operator comprises two groups of 3x3 matrixes which are respectively transverse and longitudinal, and the two groups of matrixes are subjected to plane convolution with an image, so that the brightness difference approximate values of the transverse and longitudinal directions can be obtained respectively. The formula is as follows:
the lateral and longitudinal gradient approximations for each pixel of the image may be combined using the following equations to calculate the magnitude of the gradient.
Where A represents the original image and G x and G y represent the images with lateral and longitudinal edge detection, respectively.
After pretreatment, obtaining edge information, and selecting a minimum rectangular frame from the frames.
Preferably, in step S4, the following will be done using an algorithm:
picture classification, namely inputting pictures into a multi-layer convolutional neural network, outputting a characteristic vector, and feeding back to a softmax unit to predict the types of the pictures.
And (3) positioning and classifying, namely judging whether a detected target exists in the graph by using an algorithm, marking the position of the target, marking by using a frame (Bounding Box), and identifying and positioning the target.
Object detection, namely, a picture can contain a plurality of objects, and a part of single picture also has a plurality of different classified objects so as to detect the plurality of objects.
The Softmax function, the normalized exponential function, is effectively a logarithmic normalization of the gradient of the finite term discrete probability distribution. In multiple logistic regression and linear discriminant analysis, the input of the function is the result from K different linear functions, and the probability formula for sample vector x belonging to the j-th class is:
This can be seen as a composite of Softmax functions of K linear functions.
The method comprises the steps of representing targets by using an anchor Box, dividing an input picture into S, generating n anchor boxes by each small grid, performing IOU calculation on a real frame of an image and the anchor boxes generated by the small grids where the center points of the image are located, enabling the regressed frame to be Bounding Box, enabling the performance of a model to be better when the anchor boxes are closer to the real width and height, clustering width and height with representative shapes in ground truth Box of all samples in a training set by using a k-means algorithm, clustering (dimension cluster) dimensions, clustering out a plurality of anchor Box groups with different numbers, respectively applying the anchor Box groups to the model, and finally finding out the optimal anchor boxes which are balanced between the complexity and the high recall rate (HIGH RECALL) of the model.
Bounding Box formula:
Wherein a w and a h are the width and height of an anchor box,
T w and t h are the widths and heights directly predicted by the binding box,
B w and b h are the actual widths and heights of the post-conversion predictions,
This is the width and height of the output in the final prediction.
The loss function for object detection is as follows:
In the above formula, N is the total number of categories, yi is the probability of the current category obtained after the excitation function, yi judges whether the prior frame is responsible for the target (0 or 1), if yes, 0, otherwise, 1.
Compared with the prior art, the invention has the following beneficial effects:
(1) According to the aggregate detection and classification method based on the AI intelligent machine vision technology, aggregate images are obtained in real time through a 5G network by using a camera, then a computer image processing technology is applied to process videos, and different aggregate state images are detected and identified by combining an AI intelligent algorithm, so that the problem of complex operation of the existing aggregate monitoring method is solved, and the real-time processing function is realized.
(2) According to the aggregate detection and classification method based on the AI intelligent machine vision technology, aggregate edge information, texture features and aggregate particle size features are extracted according to the acquired aggregate images, the images are processed through the image processing technology, and the images are preprocessed, so that the aggregate features are more obvious.
(3) According to the aggregate detection and classification method based on the AI intelligent machine vision technology, each collected image is divided into 5 detection areas, one image can be judged for 5 times, and therefore multiple times of judgment of one-time identification can be achieved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some examples of the present invention and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a data acquisition roadmap of the invention;
FIG. 2 is a flow chart of the intelligent aggregate detection and identification method of the invention;
FIG. 3 is a schematic diagram illustrating image processing and recognition according to the present invention;
FIG. 4 is a flow chart of edge segmentation of aggregate images according to the present invention;
FIG. 5 is a schematic diagram of edge segmentation according to the present invention;
FIG. 6 is a schematic diagram of a machine vision model of the present invention;
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It will be apparent that the described embodiments are some, but not all, of the embodiments of the invention. All other embodiments, based on the embodiments of the invention, which are apparent to those of ordinary skill in the art without inventive faculty, are intended to be within the scope of the invention.
Thus, the following detailed description of the embodiments of the invention, as presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, based on the embodiments of the invention, which are apparent to those of ordinary skill in the art without inventive faculty, are intended to be within the scope of the invention.
Example 1
Referring to fig. 1-6, the detection objects of the invention are five kinds of aggregates, namely stones, small stones, machine-made sand, small machine-made sand and face sand, wherein the particle size of the stones is in the range of 3-4 cm, the particle size of the small stones is in the range of 2-3 cm, the particle size of the machine-made sand is in the range of 1-2 cm, the particle size of the face sand is in the range of 0.1-0.5 cm, and the collected aggregate images comprise different light states, the dry and wet states of the aggregates and the states with different shooting distances.
Fig. 1 is a schematic view of a scene of the invention, a high-definition camera is positioned obliquely above a vehicle, and after a transport vehicle is in place, a video is shot and a frame sequence picture is extracted. The aggregate detection model system comprises two parts, wherein one part is used for positioning a transport vehicle and acquiring a high-definition camera image, and the other part is used for processing image data and displaying results.
An aggregate detection and classification method based on an AI intelligent machine vision technology comprises the following steps:
S1, shooting aggregate transportation videos by using a high-definition camera, and transmitting the aggregate transportation videos in real time through a 5G network to obtain aggregate images;
S2, processing the aggregate image by an image edge segmentation processing method, and extracting edge characteristics;
s3, manually marking the aggregate category of the processed aggregate image;
S4, inputting the marked aggregate image into a machine vision algorithm, training, extracting features, and outputting the features as a weight file;
s5, replaying the trained weight in a detection system, and detecting and identifying the aggregate.
In step S2, the image edge segmentation processing method specifically includes the following steps:
A, importing a bone material image, carrying out graying treatment on the bone material image, and then solving a gradient map;
B, processing the image by using a watershed algorithm on the basis of the gradient map to obtain edge lines of the segmented image;
c, using image opening operation to remove small objects in the picture and burr interference affecting the target;
And D, carrying out adhesion segmentation, and segmenting out objects adhered together.
FIG. 4 is a flow chart of the image edge segmentation technique of the present invention. First, a captured image is imported. And (5) carrying out graying treatment on the image. The third step uses image open operation to remove small objects in the picture, and influences the burr interference of the target. And finally, carrying out adhesion segmentation, and segmenting out objects adhered together.
FIG. 5 is a schematic illustration of a blocking segment of the present invention that segments the edges of aggregate.
In practical application, aggregate accumulation is considered, edge information is mainly distinguished by color light and shade boundaries, and similarity between adjacent pixels is used as an important reference basis in the segmentation process, so that pixel points which are similar in spatial position and gray value (gradient solving) are connected with each other to form a closed contour.
The operation steps are that the color image is graying, then the gradient map is obtained, and finally the watershed algorithm is carried out on the basis of the gradient map, so as to obtain the edge line of the segmented image.
In real images, there is often an over-segmentation phenomenon using watershed algorithms due to the presence of noise points or other interfering factors, because of the presence of many very small local extremal points. To solve the problem of over-segmentation, a watershed algorithm based on a mark image may be used, i.e. guided by a priori knowledge, in order to obtain a better image segmentation effect.
In step B, the watershed calculation process is divided into two steps, one is a ranking process and the other is a flooding process. The gray level of each pixel is firstly ordered from low to high, and then in the process of realizing flooding from low to high, the influence domain of each local minimum value at the h-order height is judged and marked by adopting a first-in first-out (FIFO) structure. The watershed transformation is to obtain a water collecting basin image of the input image, and boundary points among the water collecting basins are the watershed. Obviously, watershed represents the input image maximum point. Therefore, to obtain edge information of an image, a gradient image is generally used as an input image, i.e
g(x,y)=grad(f(x,y))={[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5
Where f (x, y) represents the original image and grad represents the gradient operation.
The watershed algorithm has good response to weak edges, and noise in an image and fine gray level change of the surface of an object can generate the phenomenon of over-segmentation. But at the same time it should be seen that the watershed algorithm has a good response to weak edges, which is guaranteed by closed continuous edges. In addition, the closed catchment basin obtained by the watershed algorithm provides possibility for analyzing the regional characteristics of the image.
In step C, to reduce the over-segmentation generated by the watershed algorithm, the gradient function is modified, and a simple method is to thresholde the gradient image to eliminate the over-segmentation generated by small changes in gray scale. I.e.
g(x,y)=max(grad(f(x,y)),gθ)
Where gθ represents a threshold value.
The program adopts the method that the threshold value is used for limiting the gradient image so as to eliminate excessive segmentation caused by tiny change of gray values, a proper amount of areas are obtained, gray levels of edge points of the areas are ordered from low to high, then the submerged process is realized from low to high, and the gradient image is obtained by calculating with a Sobel operator.
The operator comprises two groups of 3x3 matrixes which are respectively transverse and longitudinal, and the two groups of matrixes are subjected to plane convolution with an image, so that the brightness difference approximate values of the transverse and longitudinal directions can be obtained respectively. The formula is as follows:
the lateral and longitudinal gradient approximations for each pixel of the image may be combined using the following equations to calculate the magnitude of the gradient.
Where A represents the original image and G x and G y represent the images with lateral and longitudinal edge detection, respectively.
After pretreatment, obtaining edge information, and selecting a minimum rectangular frame from the frames.
In step S4, the machine vision algorithm training process specifically includes the following steps:
a, selecting 5 detection areas for each marked aggregate image;
b, respectively carrying out image convolution on each detection area to extract image features;
c, carrying out up-sampling treatment on the image, and restoring the convolved feature map into the image;
d, integrating the image features by using a tensor stitching algorithm;
e, outputting the image to a detection head, wherein the detection head part classifies the image, and judging the classification result of the identification image through a probability function;
f, identifying the result and outputting the detection result of the image.
The final purpose that this patent expects to reach is to carry out real-time detection recognition to the aggregate, and detects the background complicacy, needs to discern under the condition of different illumination, different distances and the aggregate different water content, and the characteristic has the diversity, and simple image processing can not satisfy the judgement to sample characteristic, consequently uses machine vision algorithm to carry out the input and the training of sample under the different states.
And (3) processing the acquired images in the steps, manually marking, marking aggregate categories, inputting a machine vision algorithm, and extracting features.
The machine vision algorithm used in this patent is mainly 6 work steps. Firstly, selecting 5 detection areas, namely dividing an image into 5 working areas, judging a final result by using an odd number of areas, and selecting 5 recognition categories with the recognition probability of more than 50% as the final result. Secondly, image convolution, namely, the purpose of extracting image features is achieved by carrying out image convolution on each region, and the image features mainly comprise edge features, color features and texture features of aggregate. Thirdly, the image is up-sampled, so that the image accords with the size of a display area, and an interpolation method is adopted, namely, new elements are inserted between pixel points by adopting a proper interpolation algorithm on the basis of original image pixels. Fourth, tensor stitching, which uses tensor stitching to integrate image features, since we need to process the three channels (RGB) of the image separately before they can be recombined to generate a new image. Fifthly, outputting the image to a detection head, wherein the detection head part classifies the image, and judging the classification result of the identification image through a probability function. And sixthly, identifying the result, and outputting the detection result of the image, namely, the maximum probability result that the algorithm considers the image to be of a certain type.
Fig. 2 is a main flow chart of an aggregate detection method and system based on an AI intelligent vision processing technology, wherein the flow comprises four parts of aggregate region image acquisition, image preprocessing, image feature extraction and aggregate detection judgment. The system firstly judges whether a vehicle is in place or not, if so, acquires real-time image information, performs region selection and image preprocessing, then detects images by using a machine vision algorithm, performs recognition, records recognition results, outputs a final result if the probability of the selected 5 images is greater than 50%, judges whether the vehicle starts to pour, and if so, stops judging.
Fig. 3 is a technical route of the patent, wherein a training sample image is preprocessed, aggregate edges are segmented, feature extraction and training learning are performed, an efficient aggregate detection intelligent processing model is built, and then a tested sample image is processed and compared to obtain an actual aggregate identification processing result.
FIG. 6 is a flow chart of a machine vision algorithm process used in the present invention. Firstly, 5 areas, namely an image center point, an upper left area, an upper right area, a lower left area and a lower right area, are extracted from each image, so that multiple judgment of one-time identification is realized, each area is processed, image characteristics such as light, angles and textures are extracted through image convolution, an up-sampling treatment is carried out on the image, the convolved characteristic images are restored into the image, then a tensor stitching algorithm is used for expanding the dimension of the image, and finally a detection head prediction result is used. The final recognition result of the 5 regions is the consistent result of the highest probability, and more than 50% of the 5 regions are the same predicted result. And judging that the detection is finished after the 5 areas are identified.
In the step a, the 5 detection areas are respectively the center point, the upper left, the upper right, the lower left and the lower right areas of the aggregate image.
In step b, the image features include edge features, color features, and texture features of the aggregate.
In step c, the image is up-sampled to make the image conform to the size of the display area, and an interpolation method is adopted, namely, a proper interpolation algorithm is adopted to insert new elements between pixel points on the basis of original image pixels.
In step C, the gradient image is thresholded by a processing method that modifies the gradient function so that the basin responds only to the target to be detected, to reduce over-segmentation due to gray scale variation.
Example 2
S6, testing a new image, judging whether the sample size is sufficient according to the test condition, keeping the sample size as uniform as possible, if the accuracy of the weight to a certain category is lower than 90%, increasing the corresponding sample size, re-extracting the characteristics until the identification rate of each category of aggregate reaches more than 90%, and meeting the requirements.
In step S4, the following will be done using the algorithm:
picture classification, namely inputting pictures into a multi-layer convolutional neural network, outputting a characteristic vector, and feeding back to a softmax unit to predict the types of the pictures.
And (3) positioning and classifying, namely judging whether a detected target exists in the graph by using an algorithm, marking the position of the target, marking by using a frame (Bounding Box), and identifying and positioning the target.
Object detection, namely, a picture can contain a plurality of objects, and a part of single picture also has a plurality of different classified objects so as to detect the plurality of objects.
The Softmax function, the normalized exponential function, is effectively a logarithmic normalization of the gradient of the finite term discrete probability distribution. In multiple logistic regression and linear discriminant analysis, the input of the function is the result from K different linear functions, and the probability formula for sample vector x belonging to the j-th class is:
This can be seen as a composite of Softmax functions of K linear functions.
The method comprises the steps of representing targets by using an anchor Box, dividing an input picture into S, generating n anchor boxes by each small grid, performing IOU calculation on a real frame of an image and the anchor boxes generated by the small grids where the center points of the image are located, enabling the regressed frame to be Bounding Box, enabling the performance of a model to be better when the anchor boxes are closer to the real width and height, clustering width and height with representative shapes in ground truth Box of all samples in a training set by using a k-means algorithm, clustering (dimension cluster) dimensions, clustering out a plurality of anchor Box groups with different numbers, respectively applying the anchor Box groups to the model, and finally finding out the optimal anchor boxes which are balanced between the complexity and the high recall rate (HIGH RECALL) of the model.
Bounding Box formula:
Wherein a w and a h are the width and height of an anchor box,
T w and t h are the widths and heights directly predicted by the binding box,
B w and b h are the actual widths and heights of the post-conversion predictions,
This is the width and height of the output in the final prediction.
The loss function for object detection is as follows:
In the above formula, N is the total number of categories, yi is the probability of the current category obtained after the excitation function, yi judges whether the prior frame is responsible for the target (0 or 1), if yes, 0, otherwise, 1.
According to the invention, after the aggregate image is obtained, the edge characteristics are extracted by an image processing method, so that the aggregate characteristics are enhanced. Firstly, graying the picture, and then extracting the image edge through a watershed algorithm. After all the collected images are preprocessed, the images are manually marked, input into a machine vision algorithm, extracted in characteristics and output as a weight file. The method comprises the working processes of shooting aggregate transportation videos by using a high-definition camera, carrying out real-time transmission through a 5G network, extracting key areas in an image frame sequence of a monitoring area, starting to carry out preliminary processing on images, preprocessing by using an image processing method, carrying out edge detection, carrying out manual marking after image acquisition processing, inputting the images into a machine vision algorithm for training, and finally replaying the trained weights in a detection system for detecting and identifying the aggregates. Aggregate images are obtained in real time through a 5G network by using a camera, then a computer image processing technology is applied to process videos, and images in different aggregate states are detected and identified by combining an AI intelligent algorithm, so that the problem of complex operation of the existing aggregate monitoring method is solved, and the real-time processing function is realized.
The above description is only of the preferred embodiments of the present invention and is not intended to limit the present invention, but various modifications and variations can be made by those skilled in the art, and any modifications, equivalent substitutions, improvements, etc. within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (4)

1.一种基于AI智能机器视觉技术的骨料检测分类方法，其特征在于，包括以下步骤：1. A method for aggregate detection and classification based on AI intelligent machine vision technology, characterized in that it includes the following steps: S1，使用高清摄像头拍摄骨料运输视频，并通过5G网络进行实时传输，获得骨料图像；S1, uses a high-definition camera to shoot aggregate transportation videos and transmits them in real time through the 5G network to obtain aggregate images; S2，通过图像边缘分割处理方法进行对所述骨料图像处理，提取边缘特征；S2, processing the aggregate image by an image edge segmentation processing method to extract edge features; S3，对处理后的骨料图像的骨料类别进行手工标注；S3, manually labeling the aggregate categories of the processed aggregate images; S4，将标注后的骨料图像输入机器视觉算法中，进行训练，进行特征提取，输出为权重文件；S4, inputting the annotated aggregate image into the machine vision algorithm for training, feature extraction, and outputting it as a weight file; S5，将训练好的权重放在检测系统里，对骨料进行检测识别；S5, put the trained weights into the detection system to detect and identify the aggregate; 在步骤S2中，所述图像边缘分割处理方法，具体包括以下步骤：In step S2, the image edge segmentation processing method specifically includes the following steps: A，导入骨料图像，将骨料图像进行灰度化处理，然后再求梯度图；A, import the aggregate image, convert it into grayscale, and then calculate the gradient map; B，在梯度图的基础上使用分水岭算法处理图像，求得分段图像的边缘线；B, based on the gradient map, the watershed algorithm is used to process the image and obtain the edge line of the segmented image; C，使用图像开运算，去除图片中的小物体和影响目标的毛刺干扰；C, use image opening operation to remove small objects and burr interference that affects the target in the image; D，进行粘连分割，将粘连在一起的物体分割出来；D, perform adhesion segmentation to separate the objects that are stuck together; 在步骤S4中，机器视觉算法训练过程，具体包括以下步骤：In step S4, the machine vision algorithm training process specifically includes the following steps: a，对每个标注后的骨料图像均选取5个检测区域；a, 5 detection areas are selected for each annotated aggregate image; b，对每个检测区域分别进行图像卷积，提取图像特征；b. Perform image convolution on each detection area to extract image features; c，对图像进行上采样处理，将卷积后的特征图还原到图像中；c. Upsample the image and restore the convolved feature map to the image; d，使用张量拼接算法使图像特征整体化；d, use tensor splicing algorithm to integrate image features; e，输出到检测头，检测头部分就是对图像进行分类，通过概率函数判断出识别图像的分类结果；e. Output to the detection head. The detection head classifies the image and determines the classification result of the recognized image through the probability function; f，识别结果，输出该图像的检测结果；f, recognition result, output the detection result of the image; 在步骤C中，采用修改梯度函数使得集水盆只响应想要探测的目标的处理方法，对梯度图像进行阈值处理，以减低因灰度的变化产生的过度分割；In step C, a processing method is used to modify the gradient function so that the water collection basin only responds to the target to be detected, and the gradient image is threshold processed to reduce over-segmentation caused by grayscale changes; 在步骤B中，分水岭的计算过程分两个步骤，一个是排序过程，一个是淹没过程，首先对每个像素的灰度级进行从低到高排序，然后在从低到高实现淹没过程中，对每一个局部极小值在h阶高度的影响域采用先进先出结构进行判断及标注；In step B, the calculation process of the watershed is divided into two steps, one is the sorting process, and the other is the flooding process. First, the gray level of each pixel is sorted from low to high, and then in the flooding process from low to high, the influence domain of each local minimum value at the h-order height is judged and marked using a first-in-first-out structure; 为得到图像的边缘信息，通常把梯度图像作为输入图像，即g(x,y)＝grad(f(x,y))＝{[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5In order to obtain the edge information of the image, the gradient image is usually used as the input image, that is, g(x,y)=grad(f(x,y))={[f(x,y)-f(x-1,y)]×2f(x,y)-f(x,y-1)]×2}×0.5 式中，f(x,y)表示原始图像，grad表示梯度运算；In the formula, f(x,y) represents the original image, and grad represents the gradient operation; 所述梯度图像进行阈值处理的具体方法，即The specific method of threshold processing of the gradient image is: g(x,y)＝max(grad(f(x,y)),gθ)g(x,y)=max(grad(f(x,y)),gθ) 式中，gθ表示阈值；In the formula, gθ represents the threshold value; 程序采用方法：用阈值限制梯度图像以达到消除灰度值的微小变化产生的过度分割，获得适量的区域，再对这些区域的边缘点的灰度级进行从低到高排序，然后在从低到高实现淹没的过程，梯度图像用Sobel算子计算获得；The program adopts the method: use the threshold to limit the gradient image to eliminate the over-segmentation caused by the slight change of the gray value, obtain the appropriate area, and then sort the gray levels of the edge points of these areas from low to high, and then realize the flooding process from low to high. The gradient image is calculated using the Sobel operator; 该算子包含两组3x3的矩阵，分别为横向及纵向，将之与图像作平面卷积，即可分别得出横向及纵向的亮度差分近似值，其公式如下:The operator contains two sets of 3x3 matrices, one for horizontal and one for vertical. By performing a planar convolution with the image, the approximate horizontal and vertical brightness difference values can be obtained respectively. The formula is as follows: 图像的每一个像素的横向及纵向梯度近似值可用以下的公式结合，来计算梯度的大小，The approximate horizontal and vertical gradients of each pixel in the image can be combined with the following formula to calculate the gradient size: 式中A代表原始图像，Gx及Gy分别代表经横向及纵向边缘检测的图像；Where A represents the original image, G x and G y represent the images after horizontal and vertical edge detection respectively; 经过预处理后，获取边缘信息，框选出最小矩形框。After preprocessing, the edge information is obtained and the minimum rectangular frame is selected. 2.根据权利要求1所述一种基于AI智能机器视觉技术的骨料检测分类方法，其特征在于，在步骤a中，5个所述检测区域分别是骨料图像的中心点、左上、右上、左下、右下区域。2. According to claim 1, a method for aggregate detection and classification based on AI intelligent machine vision technology is characterized in that, in step a, the five detection areas are respectively the center point, upper left, upper right, lower left, and lower right areas of the aggregate image. 3.根据权利要求1所述一种基于AI智能机器视觉技术的骨料检测分类方法，其特征在于，在步骤b中，所述图像特征包括骨料的边缘特征、颜色特征和纹理特征。3. According to claim 1, a method for aggregate detection and classification based on AI intelligent machine vision technology is characterized in that, in step b, the image features include edge features, color features and texture features of the aggregate. 4.根据权利要求1所述一种基于AI智能机器视觉技术的骨料检测分类方法，其特征在于，在步骤c中，图像上采样处理，使得图像符合显示区域的大小，采用内插值方法，即在原有图像像素的基础上在像素点之间采用合适的插值算法插入新的元素。4. According to claim 1, a method for aggregate detection and classification based on AI intelligent machine vision technology is characterized in that in step c, the image is upsampled so that the image conforms to the size of the display area, and an interpolation method is used, that is, a new element is inserted between pixels based on the original image pixels using a suitable interpolation algorithm.
CN202210461094.7A 2022-04-28 2022-04-28 An aggregate detection and classification method based on AI intelligent machine vision technology Active CN114882400B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210461094.7A CN114882400B (en) 2022-04-28 2022-04-28 An aggregate detection and classification method based on AI intelligent machine vision technology

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210461094.7A CN114882400B (en) 2022-04-28 2022-04-28 An aggregate detection and classification method based on AI intelligent machine vision technology

Publications (2)

Publication Number Publication Date
CN114882400A CN114882400A (en) 2022-08-09
CN114882400B true CN114882400B (en) 2025-01-03

Family

ID=82672709

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210461094.7A Active CN114882400B (en) 2022-04-28 2022-04-28 An aggregate detection and classification method based on AI intelligent machine vision technology

Country Status (1)

Country Link
CN (1) CN114882400B (en)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116571330B (en) * 2023-05-12 2023-11-17 江苏乐通彩业新型建材有限公司 Intelligent dry-mixed mortar production device and manufacturing method
CN116260990B (en) * 2023-05-16 2023-07-28 合肥高斯智能科技有限公司 AI asynchronous detection and real-time rendering method and system for multipath video streams
CN116664539B (en) * 2023-06-14 2024-03-12 中南大学 Digital image processing method and device for water-permeable water-stable recycled aggregate mixture
CN118365722A (en) * 2024-03-29 2024-07-19 华中科技大学 Aggregate particle image generation method and system based on diffusion model
CN118392733B (en) * 2024-06-26 2025-02-07 苏州中材非金属矿工业设计研究院有限公司 An aggregate particle size laser online monitoring system based on AI model

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104063866A (en) * 2014-06-26 2014-09-24 中国矿业大学(北京) Method for detecting granularity in ore transmission process
CN110175658A (en) * 2019-06-26 2019-08-27 浙江大学 A kind of distress in concrete recognition methods based on YOLOv3 deep learning

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111127416A (en) * 2019-12-19 2020-05-08 武汉珈鹰智能科技有限公司 An automatic detection method for surface defects of concrete structures based on computer vision
CN114092485B (en) * 2021-09-28 2024-08-09 华侨大学 Mask rcnn-based stacked coarse aggregate image segmentation method and mask rcnn-based stacked coarse aggregate image segmentation system

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104063866A (en) * 2014-06-26 2014-09-24 中国矿业大学(北京) Method for detecting granularity in ore transmission process
CN110175658A (en) * 2019-06-26 2019-08-27 浙江大学 A kind of distress in concrete recognition methods based on YOLOv3 deep learning

Also Published As

Publication number Publication date
CN114882400A (en) 2022-08-09

Similar Documents

Publication Publication Date Title
CN114882400B (en) An aggregate detection and classification method based on AI intelligent machine vision technology
CN106651872B (en) Pavement crack identification method and system based on Prewitt operator
Akagic et al. Pavement crack detection using Otsu thresholding for image segmentation
EP3633605B1 (en) Information processing device, information processing method, and program
Dorafshan et al. Automatic surface crack detection in concrete structures using OTSU thresholding and morphological operations
CN106485223B (en) The automatic identifying method of rock particles in a kind of sandstone microsection
CN114694144B (en) Intelligent identification and rating method for non-metallic inclusions in steel based on deep learning
CN107862677A (en) The Tunnel Lining Cracks recognition methods of thresholding algorithm and system between a kind of class based on gradient
CN107169953A (en) Bridge concrete surface crack detection method based on HOG features
Abbas et al. Automated pavement distress detection using image processing techniques
CN113177924A (en) Industrial production line product flaw detection method
CN105809121A (en) Multi-characteristic synergic traffic sign detection and identification method
CN109598681B (en) No-reference quality evaluation method for image after repairing of symmetrical Thangka
CN114332026A (en) Visual detection method and device for scratch defects on surface of nameplate
CN113887524B (en) Magnetite microscopic image segmentation method based on semantic segmentation
CN107038690B (en) A Motion Shadow Removal Method Based on Multi-feature Fusion
CN118967672A (en) Industrial defect detection method, system, device and storage medium
CN113436157A (en) Vehicle-mounted image identification method for pantograph fault
CN114331986A (en) A method of dam crack identification and measurement based on unmanned aerial vehicle vision
Sorncharean et al. Crack detection on asphalt surface image using enhanced grid cell analysis
CN108647593A (en) Unmanned plane road surface breakage classification and Detection method based on image procossing and SVM
CN110211112A (en) A kind of casting defect inspection method based on filtering selective search
CN114863277A (en) Machine vision-based method for rapidly detecting irregular particle group overrun particles
CN119360385A (en) A method and system for quickly locating and identifying fine cracks in tunnel lining
CN119049041A (en) Litchi nondestructive sorting method and device, storage medium and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant