[go: up one dir, main page]

CN111169871A - A garbage truck intelligent manipulator grabbing trash can method and manipulator - Google Patents

A garbage truck intelligent manipulator grabbing trash can method and manipulator Download PDF

Info

Publication number
CN111169871A
CN111169871A CN202010032478.8A CN202010032478A CN111169871A CN 111169871 A CN111169871 A CN 111169871A CN 202010032478 A CN202010032478 A CN 202010032478A CN 111169871 A CN111169871 A CN 111169871A
Authority
CN
China
Prior art keywords
trash
circle
manipulator
garbage
image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010032478.8A
Other languages
Chinese (zh)
Inventor
陈海初
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Foshan University
Original Assignee
Foshan University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Foshan University filed Critical Foshan University
Priority to CN202010032478.8A priority Critical patent/CN111169871A/en
Publication of CN111169871A publication Critical patent/CN111169871A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B65—CONVEYING; PACKING; STORING; HANDLING THIN OR FILAMENTARY MATERIAL
    • B65F—GATHERING OR REMOVAL OF DOMESTIC OR LIKE REFUSE
    • B65F3/00—Vehicles particularly adapted for collecting refuse
    • B65F3/02—Vehicles particularly adapted for collecting refuse with means for discharging refuse receptacles thereinto
    • B65F3/04—Linkages, pivoted arms, or pivoted carriers for raising and subsequently tipping receptacles
    • B65F3/041—Pivoted arms or pivoted carriers
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Mechanical Engineering (AREA)
  • Image Analysis (AREA)

Abstract

本发明提供了一种垃圾车智能机械手抓取垃圾桶方法，包括以下步骤：通过图像采集装置获取垃圾桶的特征信息；通过垃圾桶的特征信息获取机械手相对于垃圾桶的位置信息；根据垃圾桶相对所述机械手的位置信息规划所述机械手的运动轨迹；根据所述运动轨迹控制所述机械手对作业区间内的待处理的垃圾桶进行抓取；同时本发明还提供了机械手的结构。本发明是将单目视觉与激光组合用于检测，提供一种检测精度高、成本低、获取垃圾桶特征信息方便的检测手段，通过该检测手段能够使得机械手精准的抓取垃圾桶的凸耳并完成倾倒垃圾的动作，能够使得垃圾车的抓取垃圾桶等动作不再需要人工操作，降低劳动强度，减少工作人员受伤的几率。

Figure 202010032478

The invention provides a method for grabbing a trash can by an intelligent manipulator of a garbage truck. The motion track of the manipulator is planned relative to the position information of the manipulator; the manipulator is controlled according to the motion track to grab the trash can to be treated in the working area; meanwhile, the present invention also provides the structure of the manipulator. The invention combines monocular vision and laser for detection, and provides a detection method with high detection accuracy, low cost, and convenient acquisition of characteristic information of the trash can, through which the manipulator can accurately grasp the lugs of the trash can. And completing the action of dumping garbage can make the actions such as grabbing the trash can of the garbage truck no longer need manual operation, reduce labor intensity, and reduce the probability of injury to staff.

Figure 202010032478

Description

Method for grabbing garbage can by intelligent manipulator of garbage truck and manipulator
Technical Field
The invention relates to the field of environmental sanitation equipment, in particular to a method for grabbing a garbage can by an intelligent manipulator of a garbage truck and the manipulator.
Background
At present, the continuous development of intelligent equipment at any time has appeared and has used the manipulator to carry out the operation to the garbage bin and empty collection rubbish, and the manipulator is held up the garbage bin and is overturned the back, and the garbage bin lid is opened the back and is accomplished rubbish and empty the process under self gravity.
Through mass search, the scheme that the mechanical arm in the prior art grabs the garbage can is found to be a mechanical arm disclosed by the publication number CN109250369A, a garbage truck and a garbage truck loading method, the disclosed mechanical arm has six degrees of freedom, can reach a required position in any posture in a working range, is high in flexibility and simple and convenient to operate, the garbage truck with the mechanical arm can automatically dump the garbage can by using the loading method disclosed by the invention, and the labor intensity of workers is greatly reduced. Or as the garbage truck side feeding mechanism disclosed in the publication number CN103057883B, the mechanism is simple to operate, and only one person is needed to complete feeding. Or as the intelligent mechanical arm of the garbage truck disclosed in the publication number CN105600242B, the garbage can be positioned and captured by the mechanical arm without manually hanging the garbage can and putting down the garbage can; after the garbage bin is preliminarily positioned, follow-up actions are automatically and intelligently controlled, and safety and high efficiency are achieved.
To sum up, the scheme that the manipulator of prior art snatchs the garbage bin needs the manpower to hang the bucket or artifical location, and the manipulator that can not realize full automation intellectuality snatchs the garbage bin to, because garbage bin wall intensity is certain, manipulator clamping-force can not be too big, when rubbish is heavier in the bucket, probably not embrace shakiness or even not embrace the garbage bin. In addition, most positions of the existing mechanical arm operation garbage can are not fixed, the garbage can is placed randomly, the mechanical arm is difficult to hold the garbage can, even the garbage can cannot be held enough, and the operation efficiency of the mechanical arm is seriously influenced. The actual problems of the mechanical arm technology on the garbage truck requiring urgent treatment in practical application still have a lot of unreported specific solutions.
Disclosure of Invention
The invention provides a method for a garbage truck intelligent manipulator to grab a garbage can and a manipulator to solve the problems,
in order to achieve the purpose, the invention adopts the following technical scheme:
a method for grabbing a garbage can by an intelligent manipulator of a garbage truck comprises the following steps:
(1) acquiring characteristic information of a garbage can to be processed in an operation interval through an image acquisition device;
(2) acquiring position information of the manipulator relative to the trash can according to the characteristic information of the trash can to be processed;
(3) planning a motion track of the manipulator according to the position information of the garbage can to be processed in the operation interval relative to the manipulator;
(4) and controlling the manipulator to grab the garbage can to be treated in the operation interval according to the motion track.
Optionally, in the step (1), the step of obtaining the characteristic information of the to-be-processed trash can in the operation interval through the image acquisition device includes the following steps:
(1-1) object calibration: obtaining internal parameters and distortion parameters of an image acquisition device for correcting distortion of a shot image through calibration of the image acquisition device; obtaining a conversion relation between a pixel distance and an actual physical distance through pixel calibration, namely the millimeter length corresponding to a unit pixel;
(1-2) acquiring a digital image of the garbage can and carrying out distortion removal: capturing a digital image of the trash can through an image acquisition device, and then carrying out distortion correction on the digital image of the trash can according to a calibration result of the image acquisition device to obtain a corrected image of the trash can;
(1-3) graying and smoothing: graying the corrected image of the garbage can, and carrying out weighted average on the red, green and blue components of the corrected image by different weights; then, smoothing the detection area by using a Gaussian function to obtain a smooth single-channel gray image of the garbage can;
(1-4) laser ranging: measuring the distance from the image acquisition device to the ground by using a laser range finder, and establishing a mapping relation between an actual curve edge point and an imaging edge parameter and a mathematical conversion relation between a pixel and a detection target;
(1-5) edge detection: obtaining an edge image of the garbage can by using an algorithm for quickly defining a threshold value;
(1-6) calculating the maximum pixel width of the lug: using Hough transform based on secondary gray value processing to find and calculate circles tangent to two end points of the maximum outer diameter of the lug of the garbage can, and performing secondary gray threshold processing to reduce residual noise points;
(1-7) calculating the maximum outer diameter of the lug of the garbage can: and establishing a geometric model, and calculating to obtain the maximum outer diameter and the height of the lugs of the garbage can through physical parameter conversion.
Optionally, in the step (1-4), a mapping relation between an actual curve edge point and an imaging edge parameter is established, namely, a distance between a target and the ground and related parameters are measured by a laser range finder by using a geometric similarity principle, a relation between actual deformation and imaging parameters is established, and a measured shape error is reduced; the establishment of the mathematical conversion relation between the pixels and the detection target refers to finding out the number of the corresponding pixel points of a unit millimeter in an image, and calculating the actual parameters of the measurement object by reading the sum of the pixels with the corresponding lengths at the two ends of the image.
Optionally, in the step (1-7), the method for grabbing the trash can by the intelligent manipulator of the trash truck includes the following steps:
(1-7a) defining the ith circle as the circle tangent to the curve at the leftmost side of the lug contour of the garbage can, and setting the center abscissa of the ith circle as xiThe radius of the circle is riThe leftmost point of the circle tangency can be obtained and is also the point of the left maximum outer circle;
(1-7b) defining the jth circle as a circle tangent to the curve at the rightmost side of the lug outline of the garbage can, and setting the center abscissa of the jth circle as xjThe radius of the circle is rjThe rightmost point tangent to the circle can be obtained and is also the point of the rightmost excircle on the right;
(1-7c) the distance between the maximum edge tangent points of the j-th circle and the i-th circle on the x coordinate is the pixel width w of the largest outer diameter of the garbage can lug2And finally, the maximum outer diameter of the convex lug of the garbage can be obtained by combining the parameter relation of the geometric model:
Figure BDA0002364837660000031
wherein d is the maximum outer diameter of the lug of the garbage can, and l1Is the distance from the edge point of the maximum outer diameter to the optical center of the image acquisition device, l3Height at maximum outer diameter, /)4The projection distance from the optical center of the image acquisition device to the ground imaging is set, w is the set standard width, w1A standard pixel width is set for the image.
Optionally, in the step (1-7), the physical parameter conversion refers to converting the pixel distance parameter into a physical distance parameter according to a mapping relationship between the obtained actual curve edge point and the imaging edge parameter and a mathematical conversion relationship between the pixel and the detection target.
An intelligent mechanical arm of a garbage truck is used for detecting, positioning, moving, dumping and resetting a garbage can, and comprises a grabbing part and a rotary telescopic part which are sequentially connected, and an information acquisition device and a control device which are arranged in the intelligent mechanical arm of the garbage truck; wherein,
the grabbing part is used for clamping the garbage can or loosening the garbage can after clamping the garbage can;
the rotary telescopic part is used for driving the grabbing part to move, dump and reset the garbage can;
the image acquisition device is used for acquiring the characteristic information of the garbage can to be processed in the operation interval;
the control device is used for controlling the grabbing part and the rotary telescopic part according to the information acquired by the image acquisition device.
The beneficial technical effects obtained by the invention are as follows:
1. the method combines monocular vision and laser for detection, provides a detection means with high detection precision, low cost and convenient acquisition of the characteristic information of the garbage can, and can enable the manipulator to accurately grab the lugs of the garbage can and finish the action of dumping the garbage through the detection means.
2. The method effectively improves the flexibility and the accuracy of measurement.
3. The method can avoid manual operation for actions of grabbing the garbage can and the like of the garbage truck, reduce labor intensity and reduce the probability of injury of workers.
Drawings
The invention will be further understood from the following description in conjunction with the accompanying drawings. The components in the figures are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the embodiments. Like reference numerals designate corresponding parts throughout the different views.
Fig. 1 is a schematic flow chart of a method for a garbage truck to grip a garbage can by an intelligent manipulator in one embodiment of the invention;
FIG. 2 is a schematic structural diagram of an intelligent manipulator of a garbage truck according to one embodiment of the present invention;
fig. 3 is a schematic flow chart of acquiring characteristic information of a trash can in a method for a trash truck intelligent manipulator to grab the trash can according to one embodiment of the invention;
FIG. 4 is a three-dimensional schematic diagram of the geometric relationship between the image capturing device and the trash can and various parameters according to one embodiment of the present invention;
FIG. 5 is a schematic top view of the image capturing device and the trash can according to one embodiment of the present invention;
fig. 6 is a diagram illustrating an improved hough circle detection effect in a method for a garbage truck intelligent manipulator to grab a garbage can according to one embodiment of the present invention;
fig. 7 is a diagram illustrating an improved hough circle detection effect in a method for a garbage truck intelligent manipulator to grab a garbage can according to one embodiment of the present invention;
fig. 8 is a schematic view of an application scenario of an intelligent manipulator of a garbage truck according to an embodiment of the present invention.
Description of reference numerals: 11-a grasping portion; 12-rotating the telescoping section.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more clearly apparent, the present invention is further described in detail below with reference to embodiments thereof; it should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. Other systems, methods, and/or features of the present embodiments will become apparent to those skilled in the art upon review of the following detailed description. It is intended that all such additional systems, methods, features and advantages be included within this description, be within the scope of the invention, and be protected by the accompanying claims. Additional features of the disclosed embodiments are described in, and will be apparent from, the detailed description below.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if there is an orientation or positional relationship indicated by the terms "upper", "lower", "left", "right", etc. based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not intended to indicate or imply that the device or component referred to must have a specific orientation, be constructed in a specific orientation, and be operated, and therefore, the terms describing the positional relationship in the drawings are only for illustrative purposes and are not to be construed as limitations of the present patent, and specific meanings of the terms described above will be understood by those skilled in the art according to specific situations.
The invention relates to a method for a garbage truck intelligent manipulator to grab a garbage can and a manipulator, which explain the following embodiments according to the description of the attached drawings:
the first embodiment is as follows:
an intelligent mechanical arm of a garbage truck is used for detecting, positioning, moving, dumping and resetting a garbage can, and comprises a grabbing part 11 and a rotary telescopic part 12 which are sequentially connected, and an information acquisition device and a control device which are arranged in the intelligent mechanical arm of the garbage truck; the grabbing part 11 is used for clamping the garbage can or loosening the garbage can after clamping the garbage can; the rotary telescopic part 12 is used for driving the grabbing part to move, dump and reset the garbage can; the image acquisition device is used for acquiring the characteristic information of the garbage can to be processed in the operation interval; the control device is configured to control the grasping portion 11 and the rotating and telescoping portion 12 according to the information acquired by the image acquisition device. Specifically, the above-mentioned manipulator is a well-known technology for extensive research and application, and is not described herein in detail.
A method for grabbing a garbage can by an intelligent manipulator of a garbage truck comprises the following steps:
(1) acquiring characteristic information of a garbage can to be processed in an operation interval through an image acquisition device;
(2) acquiring position information of the manipulator relative to the trash can according to the characteristic information of the trash can to be processed;
(3) planning a motion track of the manipulator according to the position information of the garbage can to be processed in the operation interval relative to the manipulator;
(4) and controlling the manipulator to grab the garbage can to be treated in the operation interval according to the motion track.
Optionally, in the step (1), the step of obtaining the characteristic information of the to-be-processed trash can in the operation interval through the image acquisition device includes the following steps:
(1-1) object calibration: obtaining internal parameters and distortion parameters of an image acquisition device for correcting distortion of a shot image through calibration of the image acquisition device; obtaining a conversion relation between a pixel distance and an actual physical distance through pixel calibration, namely the millimeter length corresponding to a unit pixel;
(1-2) acquiring a digital image of the garbage can and carrying out distortion removal: capturing a digital image of the trash can through an image acquisition device, and then carrying out distortion correction on the digital image of the trash can according to a calibration result of the image acquisition device to obtain a corrected image of the trash can;
(1-3) graying and smoothing: graying the corrected image of the garbage can, and carrying out weighted average on the red, green and blue components of the corrected image by different weights; then, smoothing the detection area by using a Gaussian function to obtain a smooth single-channel gray image of the garbage can;
(1-4) laser ranging: measuring the distance from the image acquisition device to the ground by using a laser range finder, and establishing a mapping relation between an actual curve edge point and an imaging edge parameter and a mathematical conversion relation between a pixel and a detection target;
(1-5) edge detection: obtaining an edge image of the garbage can by using an algorithm for quickly defining a threshold value;
(1-6) calculating the maximum pixel width of the lug: using Hough transform based on secondary gray value processing to find and calculate circles tangent to two end points of the maximum outer diameter of the lug of the garbage can, and performing secondary gray threshold processing to reduce residual noise points;
(1-7) calculating the maximum outer diameter of the lug of the garbage can: and establishing a geometric model, and calculating to obtain the maximum outer diameter and the height of the lugs of the garbage can through physical parameter conversion.
Optionally, in the step (1-4), a mapping relation between an actual curve edge point and an imaging edge parameter is established, namely, a distance between a target and the ground and related parameters are measured by a laser range finder by using a geometric similarity principle, a relation between actual deformation and imaging parameters is established, and a measured shape error is reduced; the establishment of the mathematical conversion relation between the pixels and the detection target refers to finding out the number of the corresponding pixel points of a unit millimeter in an image, and calculating the actual parameters of the measurement object by reading the sum of the pixels with the corresponding lengths at the two ends of the image.
Optionally, in the step (1-7), the method for grabbing the trash can by the intelligent manipulator of the trash truck includes the following steps:
(1-7a) defining the ith circle as the circle tangent to the curve at the leftmost side of the lug contour of the garbage can, and setting the center abscissa of the ith circle as xiThe radius of the circle is riThe leftmost point of the circle tangency can be obtained and is also the point of the left maximum outer circle;
(1-7b) the jthThe circle is defined as a circle tangent to the curve at the rightmost side of the lug outline of the garbage can, and the abscissa of the circle center of the jth circle is set as xjThe radius of the circle is rjThe rightmost point tangent to the circle can be obtained and is also the point of the rightmost excircle on the right;
(1-7c) the distance between the maximum edge tangent points of the j-th circle and the i-th circle on the x coordinate is the pixel width w of the largest outer diameter of the garbage can lug2And finally, the maximum outer diameter of the convex lug of the garbage can be obtained by combining the parameter relation of the geometric model:
Figure BDA0002364837660000081
wherein 2r is the maximum outer diameter of the lug of the garbage can, l1Is the distance from the edge point of the maximum outer diameter to the optical center of the image acquisition device, l3Height at maximum outer diameter, /)4The projection distance from the optical center of the image acquisition device to the ground imaging is set, w is the set standard width, w1A standard pixel width is set for the image.
Optionally, in the step (1-7), the physical parameter conversion refers to converting the pixel distance parameter into a physical distance parameter according to a mapping relationship between the obtained actual curve edge point and the imaging edge parameter and a mathematical conversion relationship between the pixel and the detection target.
Example two:
(1) acquiring characteristic information of a garbage can to be processed in an operation interval through an image acquisition device;
(2) acquiring position information of the manipulator relative to the trash can according to the characteristic information of the trash can to be processed;
(3) planning a motion track of the manipulator according to the position information of the garbage can to be processed in the operation interval relative to the manipulator;
(4) and controlling the manipulator to grab the garbage can to be treated in the operation interval according to the motion track.
Optionally, in the step (1), the step of obtaining the characteristic information of the to-be-processed trash can in the operation interval through the image acquisition device includes the following steps:
(1-1) object calibration: obtaining internal parameters and distortion parameters of an image acquisition device for correcting distortion of a shot image through calibration of the image acquisition device; obtaining a conversion relation between a pixel distance and an actual physical distance through pixel calibration, namely the millimeter length corresponding to a unit pixel; specifically, a digital image of the irregular bulge is captured by a camera and stored in an electronic computer. Meanwhile, camera calibration and pixel calibration are carried out on the camera, camera internal parameters and distortion parameters for correcting distortion of a shot image are obtained through the camera calibration, and the conversion relation between the pixel distance and the actual physical distance can be obtained through the pixel calibration. And then, carrying out distortion correction on the acquired image according to the calibration result.
(1-2) acquiring a digital image of the garbage can and carrying out distortion removal: capturing a digital image of the trash can through an image acquisition device, and then carrying out distortion correction on the digital image of the trash can according to a calibration result of the image acquisition device to obtain a corrected image of the trash can;
(1-3) laser ranging: measuring the distance from the image acquisition device to the ground by using a laser range finder, and establishing a mapping relation between an actual curve edge point and an imaging edge parameter and a mathematical conversion relation between a pixel and a detection target; i.e. | in FIG. 44。
Let the standard width be w mm and the standard pixel width in the image be w1The width of the pixel at the maximum outer diameter is w2. Pixel height of maximum outer diameter is h1And (4) millimeter. From the geometric relationships of fig. 4 and 5, one can obtain:
Figure BDA0002364837660000091
wherein r1 and r, h1 and h have definite mapping relation, namely r1＝f1(r),h1＝f2(h) The following formula can be obtained by conversion:
Figure BDA0002364837660000092
Figure BDA0002364837660000093
through the above formula, the problem of finding the diameter 2r and the height h of the maximum outer diameter of the lug can be converted into the pixel width w of the maximum outer diameter position of the lug2And the pixel height h of the maximum outer diameter of the lug1To a problem of (a). w is a2And h2By the following steps, the monocular vision is solved.
(1-4) graying and smoothing: graying the corrected image of the garbage can, and carrying out weighted average on the red, green and blue components of the corrected image by different weights; then, smoothing the detection area by using a Gaussian function to obtain a smooth single-channel gray image of the garbage can; graying the digital image containing the garbage can lugs acquired by the camera in the step (1-2), and processing the detection area by using a Gaussian function in a Canny operator to achieve the purposes of smoothing and enhancing the garbage can and garbage can lug edge images.
(1-5) edge detection: obtaining an edge image of the garbage can by using an algorithm for quickly defining a threshold value; a variable instruction function of defining a threshold value for human-computer interaction is designed, when a related instruction is pressed, the threshold value of high and low threshold values in a Canny algorithm can be changed, the image is initially segmented, and the efficiency is improved. The improved Canny edge detection effect is shown in fig. 7.
(1-6) calculating the maximum pixel width of the lug: using Hough transform based on secondary gray value processing to find and calculate circles tangent to two end points of the maximum outer diameter of the lug of the garbage can, and performing secondary gray threshold processing to reduce residual noise points; specifically, using the curve detection of hough transform based on secondary gray-scale value processing proposed in this embodiment:
(1-6a) description and conversion of lug Curve parameters
The ledge edge requires primarily the maximum vertex coordinates and dimensions. Describing the problem as finding a tangent arc simplifies the problem. And finding and calculating the maximum outer diameter of the lug by using Hough circle transformation.
(1-6b) principle of detecting arc with tangent curve
(1-6b-1) drawing a circle on the parameter plane by using each pixel point on the image as the center of a circle and using the known radius r (or radius value range), accumulating the results and mapping the edge points on the image space into the parameter space.
(1-6b-2) to obtain an accumulation unit A (x, y, r).
(1-6b-3) finding out a peak point on the parameter plane, wherein the position corresponds to the center of a circle on the image.
(1-6b-4) size-sort the accumulation units A, i.e., size-sort the circles present in the image. If n pixel points exist in the image, n accumulation units are correspondingly arranged. The units are sorted and the accumulation unit with the maximum value is found. Comparing the first few larger summation units, the maximum point on the arc tangent to the lug can be found. The detection effect is shown in fig. 6.
(1-6c) secondary gray threshold processing.
Left and right contours (for example, straight lines) are detected by hough transform, and the gray values of all pixel points (including no edge point) in the region between the two contours are assigned to 0. I.e. all white dots in the interval become black, the remaining noise points can be reduced. The effect of the treatment is shown in figure 8.
(1-7) calculating the maximum outer diameter of the lug of the garbage can: establishing a geometric model, and calculating to obtain the maximum outer diameter and the height of a lug of the garbage can through physical parameter conversion; the method specifically comprises the following steps:
(1-7a) the center abscissa of the ith circle (the leftmost circle is tangent to the curve) is set as xiThe radius of the circle is riThe leftmost point of the circle's tangency, also the point of the left greatest bulge, is obtained, as shown by point A in FIG. 8iPosition shown.
(1-7b) setting the center abscissa of the jth circle (the rightmost edge is tangent to the curve) as xjThe radius of the circle is rjThe rightmost point of the circle, which is also the point at the right most bulge, is obtained, as shown by point Aj in fig. 8.
(1-7c) the distance between the maximum edge tangent points of the j-th circle and the i-th circle on the x coordinate is the pixel width w of the largest outer diameter of the garbage can lug2And finally, the maximum outer diameter of the convex lug of the garbage can be obtained by combining the parameter relation of the geometric model:
Figure BDA0002364837660000111
an intelligent mechanical arm of a garbage truck is used for detecting, positioning, moving, dumping and resetting a garbage can, and comprises a grabbing part 11 and a rotary telescopic part 12 which are sequentially connected, and an information acquisition device and a control device which are arranged in the intelligent mechanical arm of the garbage truck; wherein,
the grabbing part 11 is used for clamping the garbage can or loosening the garbage can after clamping the garbage can;
the rotary telescopic part 12 is used for driving the grabbing part 11 to move, dump and reset the garbage can;
the image acquisition device is used for acquiring the characteristic information of the garbage can to be processed in the operation interval;
the control device is configured to control the grasping portion 11 and the rotating and telescoping portion 12 according to the information acquired by the image acquisition device.
Specifically, the above-mentioned manipulator is a well-known technology for extensive research and application, and is not described herein in detail.
In conclusion, the invention provides a garbage truck intelligent manipulator garbage can grabbing method and a manipulator, the method combines monocular vision and laser for detection, provides a detection means with high detection precision, low cost and convenient garbage can characteristic information acquisition, can enable the manipulator to accurately grab the lugs of the garbage cans and finish garbage dumping actions through the detection means, and simultaneously can effectively improve the measurement flexibility and the measurement accuracy, and most importantly, the actions of grabbing the garbage cans and the like of the garbage truck can be free from manual operation, so that the labor intensity is reduced, and the probability of injury of workers is reduced.
Although the invention has been described above with reference to various embodiments, it should be understood that many changes and modifications may be made without departing from the scope of the invention. That is, the methods, systems, and devices discussed above are examples. Various configurations may omit, substitute, or add various procedures or components as appropriate. For example, in alternative configurations, the methods may be performed in an order different than that described, and/or various components may be added, omitted, and/or combined. Moreover, features described with respect to certain configurations may be combined in various other configurations, as different aspects and elements of the configurations may be combined in a similar manner. Further, elements therein may be updated as technology evolves, i.e., many elements are examples and do not limit the scope of the disclosure or claims.
Specific details are set forth in the description in order to provide a thorough understanding of the exemplary configurations including implementations. However, configurations may be practiced without these specific details, e.g., well-known circuits, processes, algorithms, structures, and techniques have been shown without unnecessary detail in order to avoid obscuring the configurations. This description provides example configurations only, and does not limit the scope, applicability, or configuration of the claims. Rather, the foregoing description of the configurations will provide those skilled in the art with an enabling description for implementing the described techniques. Various changes may be made in the function and arrangement of elements without departing from the spirit or scope of the disclosure.
It is intended that the foregoing detailed description be regarded as illustrative rather than limiting, and that it be understood that it is the following claims, including all equivalents, that are intended to define the spirit and scope of this invention. The above examples are to be construed as merely illustrative and not limitative of the remainder of the disclosure. After reading the description of the invention, the skilled person can make various changes or modifications to the invention, and these equivalent changes and modifications also fall into the scope of the invention defined by the claims.

Claims (6)

1.一种垃圾车智能机械手抓取垃圾桶方法，其特征在于，包括以下步骤：1. a garbage truck intelligent manipulator grabbing trash can method, is characterized in that, comprises the following steps: (1)通过图像采集装置获取作业区间内的待处理的垃圾桶的特征信息；(1) Obtain the characteristic information of the trash can to be processed in the operation area through the image acquisition device; (2)通过待处理的垃圾桶的特征信息获取机械手相对于垃圾桶的位置信息；(2) obtain the position information of the manipulator relative to the trash can through the characteristic information of the trash can to be processed; (3)根据作业区间内的待处理的垃圾桶相对所述机械手的位置信息规划所述机械手的运动轨迹；(3) planning the motion track of the manipulator according to the position information of the trash can to be processed in the working area relative to the manipulator; (4)根据所述运动轨迹控制所述机械手对作业区间内的待处理的垃圾桶进行抓取。(4) Controlling the manipulator to grab the trash can to be disposed of in the work area according to the motion trajectory. 2.如权利要求1所述的一种垃圾车智能机械手抓取垃圾桶方法，其特征在于，步骤(1)中，通过图像采集装置获取作业区间内的待处理的垃圾桶的特征信息包括以下步骤：2. The method for grabbing a trash can by an intelligent manipulator of a garbage truck as claimed in claim 1, wherein in step (1), the characteristic information of the trash can to be processed in the operation section is obtained by the image acquisition device, comprising the following: step: (1-1)对象标定：通过图像采集装置标定来获取用于校正拍摄图像畸变的图像采集装置内部参数和畸变参数；通过像素标定来得到像素距离与实际物理距离的转换关系，即单位像素对应的毫米长度；(1-1) Object calibration: The internal parameters and distortion parameters of the image acquisition device used to correct the distortion of the captured image are obtained through the calibration of the image acquisition device; the conversion relationship between the pixel distance and the actual physical distance is obtained through pixel calibration, that is, the unit pixel corresponds to mm length; (1-2)获取垃圾桶的数字图像并去畸变：通过图像采集装置捕获垃圾桶的数字图像，然后根据图像采集装置的标定结果对垃圾桶的数字图像进行畸变校正，得到垃圾桶的校正图像；(1-2) Acquire the digital image of the trash can and remove the distortion: capture the digital image of the trash can through the image acquisition device, and then perform distortion correction on the digital image of the trash can according to the calibration result of the image acquisition device to obtain the corrected image of the trash can ; (1-3)灰度化与平滑：将垃圾桶的校正图像进行灰度化，将其红、绿、蓝三个分量以不同的权值进行加权平均；然后用高斯函数对检测区域进行平滑处理，获得垃圾桶的平滑单通道灰度图像；(1-3) Grayscale and smoothing: grayscale the corrected image of the trash can, and perform a weighted average of its red, green, and blue components with different weights; then use a Gaussian function to smooth the detection area process to obtain a smooth single-channel grayscale image of the trash can; (1-4)激光测距：用激光测距仪测量图像采集装置到地面的距离，建立实际曲线边缘点与成像边缘参数之间的映射关系、像素与检测目标的数学转换关系；(1-4) Laser ranging: measure the distance from the image acquisition device to the ground with a laser range finder, and establish the mapping relationship between the actual curve edge points and the imaging edge parameters, and the mathematical conversion relationship between pixels and detection targets; (1-5)边缘检测：使用快速定义阈值的算法，得到垃圾桶的边缘图像；(1-5) Edge detection: use the algorithm to quickly define the threshold to obtain the edge image of the trash can; (1-6)凸耳最大像素宽度求取：使用基于二次灰度值处理的霍夫变换，寻找和计算与垃圾桶凸耳最大外径两端点处相切的圆，并进行二次灰度阈值处理，减少剩余的噪点；(1-6) Obtaining the maximum pixel width of the lug: Using the Hough transform based on the secondary gray value processing, find and calculate the circle tangent to the two ends of the maximum outer diameter of the trash can lug, and perform secondary graying Degree thresholding to reduce the remaining noise; (1-7)计算垃圾桶凸耳最大的外径：建立几何模型，通过物理参数转换，计算得到垃圾桶的凸耳最大外径与凸耳高度。(1-7) Calculate the maximum outer diameter of the trash can lug: establish a geometric model, and calculate the maximum outer diameter and lug height of the trash can through the conversion of physical parameters. 3.如权利要求2所述的一种垃圾车智能机械手抓取垃圾桶方法，其特征在于，步骤(1-4)中，建立实际曲线边缘点与成像边缘参数之间的映射关系，是指利用几何相似原理，用激光测距仪测量目标到地面的距离和相关参数，建立实际变形与成像参数的关系，减少测量的形状误差；所述建立像素与检测目标的数学转换关系，是指找出一幅图像中单位毫米对应像素点的个数，再通过读取图像两端上对应长度的像素之和，计算出测量对象的实际参数。3. a kind of garbage truck intelligent manipulator grabbing trash can method as claimed in claim 2, is characterized in that, in step (1-4), establishes the mapping relation between actual curve edge point and imaging edge parameter, refers to Using the principle of geometric similarity, a laser range finder is used to measure the distance and related parameters from the target to the ground, to establish the relationship between the actual deformation and imaging parameters, and to reduce the shape error of the measurement; the establishment of the mathematical conversion relationship between the pixel and the detection target refers to finding The number of pixels corresponding to a unit millimeter in an image is obtained, and then the actual parameters of the measurement object are calculated by reading the sum of the pixels of the corresponding lengths on both ends of the image. 4.如权利要求2所述的一种垃圾车智能机械手抓取垃圾桶方法，其特征在于，步骤(1-7)中，计算垃圾桶凸耳最大的外径，包括下述步骤：4. a kind of garbage truck intelligent manipulator grabbing trash can method as claimed in claim 2, is characterized in that, in step (1-7), calculates the largest outer diameter of trash can lug, comprises the following steps: (1-7a)将第i个圆定义为在垃圾桶的凸耳轮廓的最左边与曲线相切的圆，设第i个圆的圆心横坐标为xi，该圆半径为ri，便可得到该圆相切的最左点，也为左边最大外圆处的点；(1-7a) The i-th circle is defined as the circle tangent to the curve at the leftmost edge of the outline of the lug of the trash can, and the abscissa of the center of the i-th circle is x i , and the radius of the circle is r i , then The leftmost point where the circle is tangent can be obtained, which is also the point at the largest outer circle on the left; (1-7b)将第j个圆定义为在垃圾桶的凸耳轮廓的最右边与曲线相切的圆，设第j个圆的圆心横坐标为xj，该圆半径为rj，便可得到该圆相切的最右点，也为右边最大外圆处的点；(1-7b) Define the j-th circle as the circle tangent to the curve at the rightmost edge of the outline of the trash can. Let the abscissa of the center of the j-th circle be x j , and the radius of the circle is r j , then The rightmost point where the circle is tangent can be obtained, which is also the point at the largest outer circle on the right; (1-7c)x坐标上的第j和i个圆的最大边缘切点间的距离就是垃圾桶凸耳最大的外径的像素宽度w2，结合几何模型的参数关系，最终可求得垃圾桶凸耳最大的外径：(1-7c) The distance between the maximum edge tangent point of the jth circle and the i circle on the x coordinate is the pixel width w 2 of the largest outer diameter of the trash can lug. Combined with the parameter relationship of the geometric model, the garbage can finally be obtained. Maximum outer diameter of barrel lugs:
Figure FDA0002364837650000021
Figure FDA0002364837650000021
 式中，d为垃圾桶凸耳最大的外径，l1为最大外径边缘点到图像采集装置光心的距离，l3为最大外径处的高度，l4为图像采集装置光心到地面成像的投影距离，w为设定标准宽度，w1为图像中设定标准的像素宽度。In the formula, d is the maximum outer diameter of the trash can lug, l1 is the distance from the edge point of the maximum outer diameter to the optical center of the image acquisition device, l3 is the height at the maximum outer diameter, and l4 is the distance from the optical center of the image acquisition device to the optical center. The projection distance of the ground imaging, w is the standard width, and w 1 is the standard pixel width in the image. 5.如权利要求2所述的一种垃圾车智能机械手抓取垃圾桶方法，其特征在于，步骤(1-7)中，所述物理参数转换是指根据获得的实际曲线边缘点与成像边缘参数之间的映射关系，以及像素与检测目标的数学转换关系，把像素距离参数转化为物理距离参数。5. a garbage truck intelligent manipulator grabbing trash can method as claimed in claim 2, is characterized in that, in step (1-7), described physical parameter conversion refers to according to the actual curve edge point and imaging edge obtained The mapping relationship between parameters, as well as the mathematical conversion relationship between pixels and detection targets, convert pixel distance parameters into physical distance parameters. 6.一种垃圾车智能机械手，用于对垃圾桶进行检测、定位、移取、倾倒、复位，其特征在于，所述垃圾车智能机械手包括依次连接的抓取部以及旋转伸缩部，所述垃圾车智能机械手还包括安装于其内的信息采集装置和控制装置；其中，6. A garbage truck intelligent manipulator for detecting, positioning, removing, dumping, and resetting the trash can, characterized in that the garbage truck intelligent manipulator comprises a grasping part and a rotating telescopic part that are connected in sequence, and the The garbage truck intelligent manipulator further includes an information collection device and a control device installed therein; wherein, 所述抓取部，用于对垃圾桶进行夹取动作或在夹取垃圾桶后进行松开动作；The grasping part is used for gripping the trash can or performing a loosening action after gripping the trash can; 所述旋转伸缩部，用于带动抓取部进行对垃圾桶移动、倾倒、复位的操作；The rotating telescopic part is used to drive the grabbing part to move, dump and reset the trash can; 所述图像采集装置，用于获取作业区间内的待处理的垃圾桶的特征信息；The image acquisition device is used for acquiring characteristic information of trash cans to be disposed of in the work area; 所述控制装置，用于根据所述所述图像采集装置获取的信息对所述抓取部以及所述旋转伸缩部进行操控。The control device is configured to control the grasping part and the rotating telescopic part according to the information acquired by the image acquisition device.
CN202010032478.8A 2020-01-13 2020-01-13 A garbage truck intelligent manipulator grabbing trash can method and manipulator Pending CN111169871A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010032478.8A CN111169871A (en) 2020-01-13 2020-01-13 A garbage truck intelligent manipulator grabbing trash can method and manipulator

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010032478.8A CN111169871A (en) 2020-01-13 2020-01-13 A garbage truck intelligent manipulator grabbing trash can method and manipulator

Publications (1)

Publication Number Publication Date
CN111169871A true CN111169871A (en) 2020-05-19

Family

ID=70652806

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010032478.8A Pending CN111169871A (en) 2020-01-13 2020-01-13 A garbage truck intelligent manipulator grabbing trash can method and manipulator

Country Status (1)

Country Link
CN (1) CN111169871A (en)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112633168A (en) * 2020-12-23 2021-04-09 长沙中联重科环境产业有限公司 Garbage truck and method and device for identifying barrel turning action of garbage truck
CN113031635A (en) * 2021-02-26 2021-06-25 上海高仙自动化科技发展有限公司 Attitude adjusting method and device, cleaning robot and storage medium
CN113183138A (en) * 2021-04-26 2021-07-30 上海锵玫人工智能科技有限公司 Garbage carrying and sorting robot and control method thereof
CN113666028A (en) * 2021-07-27 2021-11-19 南京航空航天大学 Garbage can detecting and grabbing method based on fusion of laser radar and camera
CN116309535A (en) * 2023-04-21 2023-06-23 浙江科技学院 Method and system for recycling and dumping garbage bins for garbage recycling vehicles
US20230382228A1 (en) * 2020-10-15 2023-11-30 Oshkosh Corporation Refuse vehicle with electric chassis
US12502965B2 (en) * 2023-08-15 2025-12-23 Oshkosh Corporation Refuse vehicle with electric chassis

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1321383A1 (en) * 2001-12-18 2003-06-25 Faun Umwelttechnik GmbH & Co. Refuse collection vehicle with grabbing means for refuse collection containers
CN102549374A (en) * 2009-09-30 2012-07-04 Og技术公司 A method and apparatus of a portable imaging-based measurement with self calibration
CN103350845A (en) * 2013-06-14 2013-10-16 滕明明 Automatic garbage collection vehicle
CN103426166A (en) * 2013-07-09 2013-12-04 杭州电子科技大学 Robot hand-eye co-location method based on laser and single eye
CN106289678A (en) * 2016-07-18 2017-01-04 广东工业大学 A kind of cylinder abnormity drum song detection device and recognition methods
US20170137218A1 (en) * 2014-04-24 2017-05-18 Mytmule, Llc Material Handling System
CN109178728A (en) * 2018-06-29 2019-01-11 长沙中联重科环境产业有限公司 Operating system, method and the garbage truck of unmanned feeding garbage truck
CN109250369A (en) * 2018-12-07 2019-01-22 长沙中联重科环境产业有限公司 The charging method of mechanical arm, garbage truck and garbage truck
CN110342153A (en) * 2019-07-29 2019-10-18 齐霄强 A kind of dustbin identification grasping means based on three-dimensional point cloud
CN110539992A (en) * 2019-10-21 2019-12-06 南京可信机器人研究院有限公司 mechanical arm, garbage truck and feeding method of garbage truck

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP1321383A1 (en) * 2001-12-18 2003-06-25 Faun Umwelttechnik GmbH & Co. Refuse collection vehicle with grabbing means for refuse collection containers
CN102549374A (en) * 2009-09-30 2012-07-04 Og技术公司 A method and apparatus of a portable imaging-based measurement with self calibration
CN103350845A (en) * 2013-06-14 2013-10-16 滕明明 Automatic garbage collection vehicle
CN103426166A (en) * 2013-07-09 2013-12-04 杭州电子科技大学 Robot hand-eye co-location method based on laser and single eye
US20170137218A1 (en) * 2014-04-24 2017-05-18 Mytmule, Llc Material Handling System
CN106289678A (en) * 2016-07-18 2017-01-04 广东工业大学 A kind of cylinder abnormity drum song detection device and recognition methods
CN109178728A (en) * 2018-06-29 2019-01-11 长沙中联重科环境产业有限公司 Operating system, method and the garbage truck of unmanned feeding garbage truck
CN109250369A (en) * 2018-12-07 2019-01-22 长沙中联重科环境产业有限公司 The charging method of mechanical arm, garbage truck and garbage truck
CN110342153A (en) * 2019-07-29 2019-10-18 齐霄强 A kind of dustbin identification grasping means based on three-dimensional point cloud
CN110539992A (en) * 2019-10-21 2019-12-06 南京可信机器人研究院有限公司 mechanical arm, garbage truck and feeding method of garbage truck

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230382228A1 (en) * 2020-10-15 2023-11-30 Oshkosh Corporation Refuse vehicle with electric chassis
CN112633168A (en) * 2020-12-23 2021-04-09 长沙中联重科环境产业有限公司 Garbage truck and method and device for identifying barrel turning action of garbage truck
CN112633168B (en) * 2020-12-23 2023-10-31 长沙中联重科环境产业有限公司 Garbage truck and method and device for identifying garbage can overturning action of garbage truck
CN113031635A (en) * 2021-02-26 2021-06-25 上海高仙自动化科技发展有限公司 Attitude adjusting method and device, cleaning robot and storage medium
CN113183138A (en) * 2021-04-26 2021-07-30 上海锵玫人工智能科技有限公司 Garbage carrying and sorting robot and control method thereof
CN113183138B (en) * 2021-04-26 2024-06-28 上海锵玫人工智能科技有限公司 Garbage carrying and sorting robot and control method thereof
CN113666028A (en) * 2021-07-27 2021-11-19 南京航空航天大学 Garbage can detecting and grabbing method based on fusion of laser radar and camera
CN113666028B (en) * 2021-07-27 2022-05-20 南京航空航天大学 Garbage can detecting and grabbing method based on fusion of laser radar and camera
CN116309535A (en) * 2023-04-21 2023-06-23 浙江科技学院 Method and system for recycling and dumping garbage bins for garbage recycling vehicles
US12502965B2 (en) * 2023-08-15 2025-12-23 Oshkosh Corporation Refuse vehicle with electric chassis

Similar Documents

Publication Publication Date Title
CN111169871A (en) A garbage truck intelligent manipulator grabbing trash can method and manipulator
CN111791239B (en) Method for realizing accurate grabbing by combining three-dimensional visual recognition
CN114758236B (en) A non-specific shape object recognition, positioning and manipulator grasping system and method
CN110342153B (en) Garbage can identification and grabbing method based on three-dimensional point cloud
CN113666028B (en) Garbage can detecting and grabbing method based on fusion of laser radar and camera
CN110315525A (en) A kind of robot workpiece grabbing method of view-based access control model guidance
CN109785317A (en) Vision system of automatic palletizing truss robot
CN108827154A (en) A kind of robot is without teaching grasping means, device and computer readable storage medium
CN113927606B (en) Robot 3D vision grabbing method and system
CN115861780B (en) A YOLO-GGCNN-based robotic arm detection and grasping method
CN114419437A (en) Workpiece sorting system based on 2D vision and control method and control device thereof
CN113602799A (en) A kind of airport luggage handling system and control method thereof
CN119526413B (en) Unmanned aerial vehicle mechanical arm control system and method based on machine vision
CN110640741A (en) Grabbing industrial robot with regular-shaped workpiece matching function
CN114074331A (en) Disordered grabbing method based on vision and robot
CN113034526B (en) Grabbing method, grabbing device and robot
CN117132655A (en) Filling barrel opening position measuring method based on machine vision
CN110728712A (en) Furnace mouth positioning method and device based on machine vision, storage medium and electronic terminal
CN115797332B (en) Object grabbing method and device based on instance segmentation
JP2020034526A (en) Work position and orientation recognition device and picking system
CN119017394B (en) Image processing method and device
CN110919648A (en) Automatic picking and stacking device and method based on raspberry group
CN117021099B (en) A human-computer interaction method based on deep learning and image processing for arbitrary objects
CN115922703B (en) Grabbing method, grabbing device and robot
CN113955343B (en) Intelligent classification equipment and classification method for disorderly garbage based on 3D vision

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200519

RJ01 Rejection of invention patent application after publication