ES2795500A1 - AUTONOMOUS DEVICE FOR AUTOMATIC VINE PRUNING (Machine-translation by Google Translate, not legally binding) - Google Patents

Abstract

Autonomous device for automatic pruning of vines. The present invention discloses an autonomous device for the automatic pruning of vines that allows automating the process of pruning the trellised vines. An artificial vision module (12) is configured to receive the point cloud images (20), of a vine with branches (18), and generate, by combining and processing the images (20), a 3D model only of the vine with branches and send the 3D model to the artificial intelligence module (14), which is configured to identify the branch to be pruned, calculate the 3D position of the cutting position of the branch to be pruned and transfer the cutting position and a cutting path to the robotic arm motor module (15) that connects with the robotic arm (6, 7) to bring the cutting element (22) to the cutting position following the cutting path and actuate the cutting element. cut.

Description

DESCRIPTION

Object of the invention

The object of the invention is an autonomous device for the automatic pruning of vines. The object of the present invention is to allow the automatic pruning of trellis and / or trellises without any interaction of individuals.

To achieve the above objective, the device combines, among others, artificial vision, neural networks, artificial intelligence, robotics and specific mechanics.

Technical Field of the Invention

The technical field to which the present invention belongs is that of devices that, by means of image analysis, recognize objects such as vines for their subsequent treatment.

Background of the Invention

Ways to automate tasks such as vine pruning are currently being sought. Most of the solutions provided are based on what is known as "vine trellis", "trellis vine" or "trellis vine". A vine trellis is a support to support and guide the vine plant, as it is not capable of supporting its own weight as they grow. The greatest advantage of trellised vines is that they allow the mechanization of the crop, and therefore a very important saving in labor, since there are no branches hanging between the streets, the machinery can circulate without stepping on or breaking branches in any time of the growing cycle.

There are many ways to mount a trellis or trellis, but the purpose is the same: the conduction of the aerial parts of the vineyard. Basically you can distinguish between two types, the vine and the trellis in line. A trellis line is basically made up of:

• two head or end posts, one at the beginning and one at the end of each row, which are about 40 cm longer than the intermediate ones and are stronger, as they will withstand a significant tension. They are usually nailed with an inclination of about 60 °;

• intermediate posts that will normally be placed every 5 vines or every 6 meters. They are not as strong as the extremes. All the intermediate posts are drilled, along their entire length, at a distance that will allow us to place the wires at different heights depending on the vegetative growth of the vine at that time.

• wire strand for the vine formation phase and another for vegetation (they can be 2, 3 or 4 strands). The thread can be from 0.2mm to 0.6mm depending on the type of load it will bear. Normally those for formation are 0.4 / 0.6mm and those for vegetation 0.4 / 0.2mm.

Although the trellis line facilitates the growth of the vine branches in a two-dimensional plane, the branches cannot be placed during their growth in exact places so that a machine by default can know the exact cutting point. Also, not all branches need to be cut at the same time or at the same cutting point. Finally the branches do not have any pattern or symmetry with each other, but instead grow with different angles and shapes.

Given the aforementioned difficulties, it would be desirable to find a vine pruning device that could take into account all the randomness with which vine branches grow at the time of pruning them.

Description of the Invention

The present invention discloses an autonomous device for the automatic pruning of vines that allows automating the process of pruning the trellised vines.

The device of the present invention combines artificial intelligence in a novel way, supported by neural networks, artificial vision, robotics and specific mechanics, being able to move completely autonomously, without dependence on any other element. It would simply be enough for the user (farmer) to place the device of the present invention in the first lane to start automatic pruning. It is also necessary for the user (farmer) to define, through an application or similar, the extension of the vineyard to be pruned. From that moment on, the device of the present invention moves completely autonomously without human intervention.

In an embodiment of the invention, the autonomous automatic pruning device for vines comprises a mobile platform and a work cabin, where:

• the mobile platform comprises at least one propulsion system;

• the work cabin includes: a general control module; two interior walls parallel to each other, separated by a predetermined distance; two robotic arms, where each articulated robotic arm comprises at least one cutting element; at least two depth cameras per side wall that generate point cloud images.

Where the general control module comprises an artificial vision module, an artificial intelligence module and a robotic arm motor module. And where the artificial vision module is configured to receive the point cloud images, of a vine with branches located between the interior walls, and generate, by combining and treating the point cloud images, a 3D model only of the vine with branches and send said 3D model to the Artificial intelligence module, which is configured to identify the branch to be pruned, calculate the 3D position of the cutting position of said branch to be pruned and transfer said cutting position and a path of cut to the robotic arm drive module that connects to the robotic arm to bring the cutting element to the cutting position following the cutting path and actuate said cutting element.

In another embodiment of the invention, the mobile platform comprises a chassis, a battery to power the robotic arms and a second battery to power the mobile platform and the work cabin.

In another embodiment of the invention, the drive system is connected to the general control module. Furthermore, the propulsion system comprises at least one omnidirectional and traction wheel, a steering motor that guides the omnidirectional and traction wheel, a traction motor that provides movement to the omnidirectional and traction wheel, and an odor sensor. The work cabin can additionally comprise a navigation module, which comprises a GPS module connected to a GPS antenna, two front navigation cameras, two rear navigation cameras, LIDAR sensors both on the front and on the rear, and at least one inertial sensor. The work cabin can additionally comprise an obstacle detector photocell in such a way that the autonomous automatic pruning device for vines is self-moving by means of the propulsion system based on GPS coordinates obtained in the navigation module and based on obstacles detected by the photocell and from an environment captured by the LIDAR sensors, the inertial sensor and the front and rear navigation cameras. The general control module can additionally comprise a remote control module, which in turn comprises a WiFi / Bluetooth and 4G / 5G communication means with a user device so that the remote control module is configured so that the device autonomous automatic pruning for vines works according to a working mode selected from: autonomous mode, in which the autonomous device is configured to analyze the environment and auto-move within the environment by means of the propulsion system; and, client / server mode, in which the standalone device is configured to receive instructions in real time from the user device and execute said instructions in real time.

In another embodiment of the invention, the artificial vision module is configured to receive 2D images of the vine generated by the depth cameras, where the 2D images are combined with the point cloud images to obtain the 3D model. This improves the location of the buds. For this embodiment, the machine vision module is configured to:

• clean noise from point cloud images and 2D images using filters;

• separate the foreground from the background of point cloud images and 2D images using 5x5 matrices;

• establish correspondences between the point cloud images and between the 2D images generated by the depth cameras;

• detect obstacles formed by wires, cables and trellis posts contained in the point cloud images and in the 2D images;

• detect a vine structure;

• generate a 3D model of the vine only, removing obstacles.

In another embodiment of the invention, the artificial vision module is configured to:

• clean noise from point cloud images using filters;

• separate the foreground from the background of the point cloud images, preferably using 5x5 matrices;

• establish correspondences between the point cloud images generated by the depth cameras;

• detect obstacles formed by wires, cables and trellis posts contained in the point cloud images;

• detect a vine structure;

• generate a 3D model of the vine only, removing obstacles.

In another embodiment of the invention, the artificial intelligence module is configured to:

• interpret and study the vine morphologically;

• rebuild the vine in a 3D model;

• add the obstacles detected in the artificial vision module to the 3D model;

• calculate and introduce some buds of the vineyards in the 3D model;

• identify the branches to be cut;

• Identify some cutting points to make on the vine.

In another embodiment of the invention, the robotic arm comprises a cutting element, such as a scissors or a blade, an artificial vision camera and a laser sensor controlled by an on-board vision module and cutting included in the module of general control. The robotic arm may comprise an upper fixed platform, to which three motors are attached, from each of which a pair of bars is suspended. Each pair of bars is attached to a mobile platform where the cutting element, the artificial vision camera and the laser sensor are integrated.

In another embodiment of the invention, the robotic arm can comprise an upper platform, which connects to an arm rotation motor at one of the ends of the upper platform by means of an upper motor, which vertically moves the upper platform, the other end of the upper platform is connected with a lower platform driven by a lower motor through a belt that allows vertical displacement of the lower platform, which is divided into elbow, arm, forearm, wrist, which are connected by an elbow motor, a forearm motor and a wrist motor, respectively. At the end of the lower platform are the cutting element, the artificial vision camera and the laser sensor.

In another embodiment of the invention, the robotic arm motor module may additionally comprise a sub-module for generating trajectories that it comprises a cost function to calculate the different cutting paths of each robotic arm.

In another embodiment of the invention, the autonomous automatic pruning device for vines may additionally comprise a second cabin and a second mobile platform. By means of two cabins and two mobile platforms, the device of the present invention can prune two rows of vines simultaneously. This entails saving time and costs mainly.

Brief description of the Figures.

Figure 1 shows the autonomous device for the automatic pruning of vines according to an embodiment of the invention.

Figure 2 shows another perspective of the autonomous device for automatic pruning of vines shown in figure 1.

Figure 3 shows a top view of the autonomous device for automatic grapevine pruning shown in figure 1.

Figure 4 shows a block diagram of the modules that make up the autonomous device for automatic pruning of vines shown in figure 1.

Figure 5 shows two perspectives of the autonomous device for automatic pruning of vines shown in figure 1 with side walls.

Figure 6 shows a perspective of the autonomous device for the automatic pruning of vines shown in figure 1 with the depth cameras pointing to the same point.

Figure 7 shows an embodiment of the robotic arm according to the present invention.

Figure 8 shows another embodiment of the robotic arm according to the present invention.

Figure 9 shows a point cloud image representing a vine and obstacles made by the artificial vision module.

Figure 10 shows a transformation of the point cloud into a vine carried out by the artificial intelligence module.

Figure 11 shows a calculation of the shape of a vine resulting from its characteristic mean made by the artificial intelligence module.

Figure 12 shows an intelligent processing of the vine carried out by the artificial intelligence module.

Figure 13 shows a generation of a complete 3D model carried out by the artificial intelligence module.

Figure 14 shows the identification of some buds on a vine carried out by the artificial intelligence module.

Figure 15 shows the different points of a branch calculated by the artificial intelligence module and the different angles with which a cutting element can access a cutting point.

Description of an embodiment.

In view of the aforementioned figures, and in accordance with the numbering adopted, an example of a preferred embodiment of the invention can be seen in them, which comprises the parts and elements that are indicated and described in detail below.

The list of references used in the figures is as follows:

1. - Autonomous automatic pruning device for vines.

2. - Mobile platform.

3. - Work cabin.

4. - Chassis of the mobile platform.

5. - Propulsion system: 5th - traction and omnidirectional wheel;

5b - steering motor; 5c - traction motor;

5d - odometric sensor on each wheel.

6. - “Y” type robotic arm.

7. - Robotic arm type “L”.

8. - General control module.

9. - Depth cameras.

10. - Photocell.

11. - Navigation module: 11a - GPS module; 11b - GPS antenna;

11c - front navigation cameras; 11d - rear navigation cameras; 11e - LIDAR type laser sensors (front and rear); 11f - inertial sensor.

12. - Machine Vision Module.

13. - Remote control module: 13a - WiFi; 13b - 4G.

14. - Artificial Intelligence Module: 14a - Artificial Intelligence algorithm. 15. - Robotic arms motor module: Sub-module for the generation of trajectories (15a).

16. - Embedded vision module and cut.

17. - Batteries: 17th - robotic arms battery;

17b - other components.

18. - Vine: 18a - branch of the vine.

19. - Obstacles: wires and metal posts.

20. - Point cloud image.

21. - Inside wall of the work cabin.

22. - Robotic arm cutting element.

23. - Artificial vision camera of the robotic arm.

24. - Robot arm laser sensor.

25. - Fixed upper platform.

26. - Motors of the robotic arm 6.

27. - Robotic arm bars.

28. - Mobile platform of the robotic arm.

29. - Arm motors: 29a - upper; 29b - lower; 29c - forearm; 29d - elbow; 29e - wrist.

30. - Robotic arm transmission belt.

31. - Rotation motor of the robotic arm.

32. - Upper platform of the robotic arm.

33. - Lower platform of the robotic arm: 33a - elbow; 33b - arm;

33c - forearm; 33d - wrist.

34. - User device such as “Smartphone” or tablet.

35. - image obtained by applying a graph to the point cloud 20.

36. - Image obtained from the graph resulting from the mean of the characteristic points.

37. - image obtained from the intelligent processing of the graph.

38. - 3D model of the vine

39. - 2D image of the vine.

40. - cutting point located at or very close to the bud of a branch (approx. 1 cm).

41. - approach angles to the cut-off point.

Throughout the present description, the term "connected" can mean that the elements that are "connected" can share electrical connection for power and / or data connection for communication between both elements. The data connection can be one-way or two-way. Alternatively or additionally, the term "connected" can mean that the elements that are "connected" can be "mechanically" joined by one or more parts of each element.

Thus, as seen in Figure 1, in a possible preferred embodiment of the autonomous device 1, the autonomous device 1 is essentially comprised of the mobile platform 2 and the work cabin 3, which work together. The mobile platform is configured to move the work cabin and provide all the necessary means for pruning as described below.

As seen in figure 2, the mobile platform 2 is made up of the chassis 4 and the propulsion system 5. The propulsion system 5 comprises the driving wheels 5a, the steering motors 5b, the traction motors 5c and the sensor. odometer 5d on each drive wheel 5a. Specifically, in the example of figure 2, the propulsion system 5 comprises four driving wheels with their corresponding steering motors, traction motors and odomometric sensors. As each driving wheel has its own steering motor and traction motor, each wheel can be oriented independently of the rest. This allows to move autonomously through the vineyard by detecting the row of vineyards. All this thanks to artificial intelligence algorithms together with environmental sensors, such as LIDAR-type laser sensors, inertial sensors, odometric sensors on the wheels, navigation cameras and GPS antenna.

For its part, the work cabin 3 is composed of two articulated robotic arms 7 (alternatively 6) (see Figures 1 to 3), one on each side of the work cabin 3 and oriented opposite each other. Depth cameras 9 are located on the work cabin 3, between two and three (with three increases the robustness) depth cameras 9 on each side of the work cabin 3 and the photocell 10 is located in the center of the work cabin so that the vineyard is right in front of the robotic arms. All this set of elements thanks to vision algorithms and artificial intelligence will be able to interact with the vineyard by pruning it.

The autonomous device for automatic grapevine pruning 1 works with two batteries 17 (see figure 3). A first battery 17a dedicated to powering only the robotic arms 7 (alternatively 6) and a second battery 17b to power the rest of the components that make up the autonomous device for automatic vine pruning.

As can be seen in figure 3, the autonomous device for the automatic pruning of vines 1 comprises the general control module 8, which is located on the mobile platform 2, although it could be located in the work cabin 3, as that its location is not relevant. The general control module 8 allows to control the movement of the mobile platform 2, and therefore of the autonomous device for the automatic pruning of vines 1. The mobile platform 2 will stop at each vineyard for a time to operate with the vine, when the moving platform 2 you have to change lanes you will have a continuous movement to leave a lane in which you have worked and get into the next lane. Additionally, the general control module 8 is capable of calculating the autonomy of the batteries 17a-17b to pre-warn a user of the need to recharge the batteries.

As shown in the block diagram of figure 4, the general control module 8 is connected with the rest of the modules (11, 12, 13, 14, 15, 16) of which the autonomous device is composed for the automatic pruning of vines 1 and is also connected with the elements distributed by the mobile platform 2 and the work cabin 3, such as depth cameras 9, photocell 10, propulsion system 5, inertial sensors 11f, LIDAR sensors 11e, the antenna GPS 11a, front 11c and rear 11d navigation cameras. Thus, the general control module 8 is connected with the navigation module 11, the artificial vision module 12, the remote control module 13, the artificial intelligence module 14 and the motor module of robotic arms 15. The robotic arms motor module 15 in turn comprises the on-board vision and cutting module 16.

The robotic arm drive module 15 controls the movements of the robotic arms 6,7. Each robotic arm 6,7 has at its end the cutting element 22 (scissors or a blade), the artificial vision camera 23 and the laser sensor 24. The on-board vision module and cutting 16 by means of the artificial vision camera 23 and / or the laser sensor 24, allows a precise approach to the branch to be cut. In addition, the on-board vision and cutting module 16 has access to the machine vision camera 23 and the laser sensor 24 to support other modules as described below. The robotic arms 6,7 embodiments are shown in Figures 7 and 8, respectively. The shape of the robotic arm 6 shown in figure 7 has an upper fixed platform 25, to which three motors 26 are attached, from each of which the pair of bars 27 is suspended. Each pair of bars 27 is attached to the mobile platform 28, where the cutting elements 22, the artificial vision camera 23 and the laser sensor 24 are integrated. The other embodiment of the robotic arm 7 is shown in figure 8. The robotic arm 7 shown in figure 8 it has the arm rotation motor 31 which connects the robotic arm 7 itself with the chassis and which allows 360 ° rotations of the robotic arm 7. Additionally, the robotic arm 7 has the upper platform 32. At one end of the upper platform 32, there is the upper motor 29a that connects with the rotation motor of the arm 31. The upper motor 29a vertically moves the upper platform 32. The upper platform 32 connects at the other end with the lower platform 33 driven by the lower motor 29b through the belt 30 that allows the vertical movement of the lower platform 33. The lower platform 33 is divided into four parts, elbow 33a - arm 33b - forearm 33c - wrist 33d, which are connected by elbow motor 29d, forearm motor 29c and wrist motor 29e, respectively. At the end of the lower platform 33 are the cutting element 22 (scissors or a blade), the artificial vision camera 23 and the laser sensor 24.

The navigation module 11 is in charge of checking its environment at all times to determine how the autonomous device 1 has to move to the next planned point. For this purpose, the navigation module 11 comprises the GPS module 11a connected to the GPS antenna 11b, which assists the general control module 8 to determine the position of the autonomous device 1 and the possible trajectories to follow depending on the operating mode. . Additionally, the navigation module 11 comprises the inertial sensor 11f, the cameras of front navigation 11c, the rear navigation cameras 11d and the two sensors (front and rear) laser type LIDAR 11e (see figures 4 and 5).

For example, when the autonomous device 1 reaches the end of a lane or row in the vineyard, the navigation module 11 helps the general control module 8 to calculate the path that the autonomous device 1 must travel, which will continue to travel the farm. lane by lane depending on the configuration of the terrain made by the user through GPS. To make the lane change, the autonomous device 1 will move perpendicular to the lane traveled until it comes to focus on the next lane to travel and once there, the device will re-enter the lane, traveling it until it reaches the end and back to start to finish traveling all the rows of vineyards within the limitation of the land or vineyard indicated by the mobile application (land area bounded by GPS). The general control module 8 can detect when it is in a suitable position to enter the lane or row by calculating the position of the autonomous device 1 by means of the navigation module 11 and knowing the environment by means of the artificial vision module 12, which uses a sensor front LIDAR type laser and two front navigation cameras, and one rear LIDAR type laser sensor and two rear navigation cameras. In total four cameras, two in the front and two rear in the back.

As shown in the block diagram of figure 4, the remote control module 13 can comprise the Wifi / Bluetooth module 13a and the 4G / 5G module 13b for wireless communications with other remote devices 34 of a user such as telephones. smart phones so that the autonomous device 1 can operate based on two main modes:

• Autonomous mode, in which the autonomous device 1 moves through the vineyard automatically assisted by the odometric sensors 5d on each wheel (to verify the movement of each wheel), as well as the steering motors 5b, the traction motors 5c , the navigation module 11 and the artificial vision module 12, including inertial sensors 11f (to determine inclination), the LIDAR sensors 11e, the navigation cameras 11c-11d and the GPS module 11a. The combination of the elements described above (11, 5b, 5c, 5d, 11a-11f, 12) allows the stand-alone device 1 to be positioned without fail in the optimal workspace for pruning. The only action that the user will have to perform is to delimit the plot using GPS coordinates so that the autonomous device 1 understands which is the work area, once the starting point (the beginning of a vineyard lane) has been established, the autonomous device 1 may move along the vineyard without the need for human intervention, performing the task of pruning and changing lanes autonomously when the current lane ends. When the mobile platform 2 is working in this mode, it will stop at each vineyard so that the corresponding modules carry out the pruning action. The mobile platform 2 detects each vineyard thanks to the photocell 10. The photocell 10 is raised at a distance of 37 cm from the ground, this height is optimally calculated so that there is only one presence or the vineyard (trunk of the vine) , or as a trellis post. When the sensor of the photocell 10 is cut, it indicates that there is either a vineyard or a trellis post. It is then that the artificial vision module 12 comes into play, which determines whether it is vid or not. In the case of being a vine, the work process that is detailed in said artificial vision module 12 begins, otherwise it will communicate to continue the march of the mobile platform 2 until the next cut of the photocell 10.

• Remote mode, in which standalone device 1 is configured to execute and receive instructions from a user and execute those instructions in real time. The user manually drives the autonomous device 1 through a remote device 34 of the client aided by a dedicated mobile application, such as a smart phone (“Smartphone”) or digital tablet (“Tablet”) with bidirectional communication that, through a dedicated application , allows the user to control the autonomous device 1 of the present invention, constantly exchanging operating data of the autonomous device 1 with the remote control module 13 that may be useful to the user.

In order to carry out the pruning, the autonomous device for the automatic pruning of vines 1 has to analyze the vines located inside the work cabin 3, and more specifically between the internal walls 21 parallel to each other. For this, the autonomous device 1 comprises the artificial vision module 12 which is supported by the depth cameras 9. Figure 6 shows how the four depth cameras 9 are oriented towards the same point. To analyze a vine that is located inside the work cabin 3, the artificial vision module 12 is connected with the depth cameras 9 to scan the interior of the work cabin 3 as explained below.

The artificial vision module 12 relies on depth images that generate the point cloud images 20, and optionally 2D images (photography type), already that the cameras used to capture the vine are depth cameras and allow both types of format to be captured. The 9 depth cameras surround the vineyard from different points and at different angles to obtain greater precision of the vineyard when captured from different points. The depth images are used to recompose and superimpose all the point clouds, and optionally the 2D images, from the 9 different depth cameras, making them coincide in a single 3D model of the vine. This 3D model 18 generated by the artificial vision module 12 is only about the vineyard, removing from this 3D model the obstacles 19 (wires, mast and / or pole of the trellised vineyards), since said information is now not interesting to process, it is added later to the 3D model. When this artificial vision module finishes generating the 3D model without obstacles, it will provide the artificial intelligence module 14. Once the 3D model of the vineyard (vineyard only) has been recomposed thanks to the artificial vision module 12, the artificial intelligence module 14 , will begin its function.

When the artificial vision module 12 sweeps the interior of the work booth 3, the 3D model formed by a cloud of points is obtained as shown in figure 9. The result of the recomposition in the 3D model is a cloud of dots that intermingle the vine 18 and the elements surrounding the vine that are referred to as "obstacles" 19 in the present specification. The detected obstacles 19 are eliminated by the artificial vision module 12 before sending the 3D model to the artificial intelligence module 14. The artificial vision module 12 processes and triangulates the information obtained from the four depth cameras 9 located per cabin (two depth cameras on each side) with the aim of recomposing the vine in the 3D model, generating the 3D model 18 of the pure vineyard, removing the noise that the images and other elements that are now not of interest such as the post may have metal and trellis wires. That is, the artificial vision module 12 takes point cloud images, and optionally 2D images, of the vine from the four cameras per work cabin, analyzes them to discriminate the vine from obstacles (common metal wires and posts in the “trellised” vines) and later it is provided to the artificial intelligence module 14, which also works both with the 3D model of the pure vineyard, and optionally with the 2D images, performing different operations. The machine vision module 12 has a margin of error of less than 1 millimeter with reality. With this precision, the artificial vision module 12 makes it possible to correctly detect the vineyard model in a real environment.

In addition, to increase precision, the artificial vision module 12 can be connected with the on-board vision and cutting module 16, through the artificial vision camera 23 and / or the laser sensor 24 located at the ends of the cutting tool. , accurately measure the distance from the analyzed vine and the cut points calculated before cutting each branch. That is, the on-board vision and cutting module 16 approaches or approaches the distances it receives from the artificial intelligence module 14 and this can be fed back when the arm approaches the cut-off point thanks to the on-board vision and cutting module 16 that at all times it checks the distance to the cutting point and if the cutting tool is correctly positioned in the position, then it is when it cuts and checks whether or not the branch has been cut. If it is not cut, you can retry up to at least two additional times.

The artificial vision module 12 mainly carries out the following functionalities:

• clean noise from images (point clouds and optionally 2D images) taken with different filters;

• separation of the foreground from the background;

• correspondence between images (point clouds and optionally 2D images) from the different depth cameras.

• detection of trellis wires and posts;

• detection of the vineyard structure; and,

• generation of a pure 3D model of the vineyard, without other elements such as cables or poles.

The “clean noise from images taken with different filters” functionality consists of applying various filters to eliminate “noise” or non-useful information when capturing both the 2D images and the point cloud images.

Within the functionality of "separation of the foreground from the background", the work cabin 3 will be composed of four depth cameras 9 and two robotic arms 6,7 as shown in figures 7 and 8, in the arrangement of the work booth shown in any of figures 1 to 3. In this way, each group of two depth cameras 9 per side works in two planes: foreground and background. Obstacles (lattice cables, poles) and the vine itself. The bottom would then be the internal face of the side wall 21 of the work cabin 3 that is behind the vine. That is, between the internal walls of the work cabin, there are only 6 or 7 robotic arms and the trellis vine. The rest of the elements of the autonomous device 1 are either flush with the interior walls or behind the interior walls and distributed throughout the chassis of the mobile platform. The inside of the cab side wall can be any color, but the best results are obtained for a uniform blue color and a matte white. The separation of the foreground from the background will be based on SVM (Support Vector Machine) algorithms. Specifically in RBF-SVM, (Radial Basis Function Support Vector Machine), which is a set of supervised learning algorithms. The artificial vision module 12 comprises libraries that make it possible to label each pixel as cable, pole or vine based on its shape, color and the color of neighboring pixels. The characteristics that are analyzed by SVM are 5x5 pixel matrices, calculating the average of the opposite pixels of the matrix to make said characteristics invariant to rotation.

The "correspondence between images from the different cameras" functionality consists of identifying the correspondence between the same points that are captured from the different depth cameras, both in 2D images and in point cloud captures. Making that, in the case of the point cloud, all the captures taken coincide in the same space, creating in this way a first 3D model of the vine and the obstacles.

Once the above process has been carried out, the functionality "detection of the wires and trellis posts" will detect obstacles by identifying the characteristic objects with straight lines belonging to posts or wires and removing them from the 3D model. To detect the cables and poles, artificial vision techniques will be used, such as the detection of straight lines, the Hough transform or the differentiation of elements by color with algorithms such as RegionGrowingRGB.

The functionality of "detection of the vineyard structure" for the detection of the structure of the vine, the Machine Vision 12 module uses as its base the standard structure of a vine: a large branch that rises from the ground and ends in a head or irregularly shaped base or a kind of T between the trunk and the main branch (depending on the variety), later separating into smaller branches which in turn divide into smaller ones.

The machine vision algorithm to ensure that all non-vine elements from the point cloud are removed, it will also subtract the information determined in the previous step when it detected the trellis posts and wires.

After carrying out this vine identification step and the previous step of eliminating obstacles through the “detection of wires and trellis posts” functionality, a “pure” 3D model will be generated 18, that is, it will only contain the vineyard in 3D without any other obstacles.

With the information provided by the artificial vision module 12, the artificial intelligence module 14 begins to transform the point cloud (provided by the artificial vision module 12) into a component that it can understand and process to know how to perform pruning. this vineyard.

The artificial intelligence module 14 has a complex artificial intelligence algorithm that is only prepared to receive the point cloud of the vineyard without having other elements such as cables and poles (see figure 9), that is, it would only be the point cloud representing the vine 18 without obstacles such as wires and metal posts 19 (see figure 9). It is for this reason that the artificial vision module 12 sends to the artificial intelligence module 14, a 3D model (see figure 9) only of the vineyard 18 eliminating all the image noise and other captured elements, such as wires and the mast of the trellis 19 that may interfere with the understanding of the vineyard by the artificial intelligence algorithm 14a. It is then when the processing of the point clouds of the vineyard begins, making a skeleton of the same and taking a characteristic summary of the vineyard to be able to interact with it.

The artificial intelligence algorithm 14a includes functions of decomposition and reconstruction of objects in images, as well as algorithms of segmentation and contraction and interpretation. That is, the point cloud of the 3D model representing vine 18 (figure 9) is processed by the artificial intelligence algorithm that performs the steps of: decomposition 35 (figure 10), reconstruction 36 (figure 11), segmentation 37 (figure 12), contraction and interpretation 38 (figure 13) to obtain the characteristic information of the morphology of the vine. Once the characteristic information on the morphology of the vine has been obtained, the artificial intelligence module 14 determines which branches are to be cut (but not the cutting point) based on the physiology of the vine and the chosen pruning method (the standalone device 1 can prune up to three different pruning methods: Guyot pruning, Cordon pruning and double Guyot pruning).

For the identification of the branches to be cut, the Artificial Intelligence module 14 takes as a model the expected behavior of a real person pruning the vineyard. This person would select the branches that they do not want to cut based on criteria (length, position, angle from the head or base, distance under the cables, or if the branch grows from the head, or from the trunk or from another branch) and cut the rest. These criteria are trained with parameterized neural networks with a cost function, so that the aforementioned parameters (length, thickness, position, angle, etc ...) will have an associated cost based on previously predefined rules, therefore that by having the vine represented in 3D, the artificial intelligence algorithm of the Artificial Intelligence module 14 has a certain cost associated with each of the branches that make up the vine based on the programmed criteria and in the end it is decided what combination of cuts to be carried out is the one that minimizes said cost function.

Once the branches that have been chosen for pruning have been selected, the artificial vision module 12 is used again, which has saved the information related to the point cloud of the obstacles 19 (wires and posts) and that were initially eliminated by the artificial vision module 12. This information is put back together to the 3D model processed by artificial intelligence module 14.

The artificial intelligence module 14 adds to the 3D model the buds of the branches 40 of the vineyard, crucial for determining the exact cut point. The identification of the buds (see figure 14) is carried out with “machine learning” algorithms (neural networks) type “TensorFlow” on the 2D images, identifying the buds and then the coordinates of the detected buds will be passed to the 3D model.

The position or coordinates of the buds (detected in the 2D images) in the 3D model will be made using mathematical formulas, depending on the position and angle of each depth camera, together with the measurement of certain characteristic points between different depth cameras. (triangulation). This triangulation will allow, starting from a 2D with X and Y coordinates of the objects, to pass to a 3D model with X, Y, Z and RX, RY, RZ coordinates.

When the 3D model has all the calculated information (processed vineyard, obstacles detected such as wires and posts, buds detected and branches selected for pruning), the artificial intelligence module 14 proceeds to calculate the exact position of the cutting point of the branch to prune. This information about the cut-off point and detected objects will then be provided to the robotic arms motor module 15. Figure 15 shows the different characteristic points of a branch calculated by the artificial intelligence module 14 and the different approach angles 41 with which the cutting element can access the cutting point.

The artificial intelligence module 14 performs the following steps:

• transforms the point cloud into a graph capable of processing characteristic points (see figure 10);

• calculates the shape of a graph resulting from the mean of the characteristic points (see figure 11);

• carries out an intelligent processing of the graph (see figure 12);

• generates the complete 3D model, processed and interpreted (see figure 13);

In the context of the present specification, a graph is a set of objects called vertices or nodes joined by links called edges or arcs, which allow representing binary relationships between elements of a set.

Intelligent processing consists of assigning a value to each node of the graph based on the characteristics of that point on the vine. For example, if it is a node of the head of the vine, it will be assigned a high value so that the head is never cut off. If it is the beginning of a fine branch, a low value will be assigned so that it is considered as an approximate cut-off point. If it is the end of the branch, it will be assigned a high value since cutting above the branch or at a distance greater than 20 cm is not interesting. A priori, it is more interesting to cut either at a distance of 4 or 5 centimeters from the beginning of the branch, or the branch flush with the head (0cm from the branch) so that it disappears completely.

Therefore, the artificial intelligence module 14 mainly carries out the following functionalities:

• Interpretation of the vine and morphological study.

• Reconstruction of the vine in 3D.

• Introduction into the 3D object of the obstacles detected in the artificial vision module.

• Calculation and introduction of the buds of the vineyards in 3D model.

• Identification of the branches to be cut.

• Identification of the cutting points to be made on the vine.

The artificial intelligence module 14 sends the cutting points to be made on the vine to the robotic arms motor module 15. The robotic arms motor module 15 calculates the path to be followed by the robotic arms 6,7 to reach all the points of established cutting avoiding obstacles such as wires, posts and other branches that may interfere. For this, the robotic arms motor module 15 comprises the trajectory generation sub-module 15a that uses algorithms for calculating obstacle-free trajectories based on ROS's "Grasp Server" and "Parallel Execution" libraries.

It must be taken into account that the cutting point can have several options of cutting angles (see figure 15), so that a single cutting point can generate different trajectories of the arms, and the robotic arms motor module 15 will always execute the one that is minimal in time costs.

Since the autonomous automatic pruning device for vines 1 has two robotic arms 6,7 per work cabin 3, the device 1 has access to each vine from two sides. This will allow the maneuvering of one or another robotic arm depending on its accessibility to the chosen cut-off point. The selection of the 6,7 robotic arm in charge of making the cut will be carried out through a mathematical cost algorithm, the cost of time in positioning to reach the different cut points is calculated for each cut point and thanks to In this way, the robotic arms motor module 15 can identify the movement time of the planning before executing it and discard or use it depending on whether it meets the minimum time. The costs of each cutting point will be calculated and updated in real time according to the position of the arms with respect to the vineyard. The robotic arm drive module 15 has implemented a priority algorithm to calculate the order of the cut points giving priority to the most accessible cut points.

Figure 15 shows the robotic arms motor module 15 calculating the planning that one of the robotic arms 6,7 has to perform to bring the cutting element 22 (scissors or blade) to the cutting point, in addition to showing the cut point shows all the possible orientations of this point, which can be made by the robot arm 6,7 to execute a cut of the branch correctly.

Finally, when the robotic arm 6,7 is positioned at the cutting point, the scissors will be activated which, supported by the artificial vision camera 23 and the laser sensor 24, will verify that the branch has been cut in the indicated position and updating the 3D model of the vineyard, this last process is carried out by the on-board vision module and section 16 that is detailed below the summary.

Therefore, the robotic arms motor module 15 mainly carries out the following functionalities:

• Generation of trajectories of the robotic arms free of collision and minimizing the path.

• Control of the robotic arms.

• Verification of the cuts made by updating the 3D environment.

The on-board vision and cutting module 16 controls the elements located at the end of the robotic arm 6,7 that are the cutting element 22, such as a scissors or a blade, the artificial vision camera 23 and the laser sensor 24. By means of the indicated elements (artificial vision camera and a laser sensor), the on-board vision and cutting module 16 can verify if the branch has been cut correctly. If the branch was cut correctly, the on-board vision and cut module 16 informs the artificial intelligence module 14 of its cut and this last module updates the status of the vineyard in 3D to continue operating with the rest of the branches. In addition, through the artificial vision camera and the laser sensor, the on-board vision and cutting module 16 provides support to the artificial intelligence module 14 to closely verify whether the position of the chosen cut-off point is correct or whether it must be corrected by a few millimeters . The on-board vision and cutting module 16 determines that the pruning has been carried out correctly, and therefore, the on-board vision and cutting module 16 is an additional safety mechanism to ensure that the cut has been carried out correctly.

Claims (14)

1. - Autonomous automatic pruning device for vines (1), characterized in that it comprises a mobile platform (2) and a work cabin (3), where: • the mobile platform (2) comprises at least one propulsion system (5); • the work cabin (3) comprises: or a general control module (8); or two interior walls (21) and parallel to each other, separated by a predetermined distance; or two robotic arms (6,7), where each articulated robotic arm comprises at least one cutting element (22); or at least two depth cameras (9) for each side wall that generate point cloud images (20); and, where the general control module (8) comprises: or an artificial vision module (12); or an artificial intelligence module (14); or a robotic arms motor module (15); where the artificial vision module (12) is configured to receive the point cloud images (20), of a vine with branches (18) located between the interior walls (21), and generate, by combining and treating the images point cloud (20), a 3D model (18) only of the vine with branches and send said 3D model to the Artificial intelligence module (14), which is configured to identify the branch to be pruned, calculate the position in 3D of the cutting position (40) of said branch to be pruned and transferring said cutting position and a cutting path to the robotic arms motor module (15) that connects with the robotic arm (6,7) to carry the element of cutting (22) to the cutting position following the cutting path and actuate said cutting element. 2. - Autonomous automatic pruning device for grapevines, according to claim 1, characterized in that the mobile platform (2) comprises a chassis (4), a battery (17a) to power the robotic arms (6,7) and a second battery (17b) to feed the mobile platform (2) and the work cabin (3). 3. - Autonomous automatic pruning device for vines, according to claim 1, characterized in that the drive system (5) is connected to the general control module (8), and where the drive system (5) comprises at least one traction wheel and omnidirectional (5a), a steering motor (5b) that guides the traction wheel and omnidirectional (5a), a traction motor (5c) that provides movement to the traction and omnidirectional wheel (5a), and an odometric sensor (5d). 4. - Autonomous automatic pruning device for vines, according to claim 3, characterized in that the work cabin (3) additionally comprises a navigation module (11), which comprises a GPS module (11a) connected to a GPS antenna (11b), two front navigation cameras (11c), two rear navigation cameras (11d), LIDAR sensors (11e) both at the front and at the rear and at least one inertial sensor (11f); and where the work cabin (3) additionally comprises a photocell (10) detecting obstacles in such a way that the autonomous automatic pruning device for vines (1) is self-moving by means of the propulsion system (5) based on some coordinates GPS obtained in the navigation module (11) and based on obstacles detected by the photocell (10) and an environment captured by the LIDAR sensors (11e), the inertial sensor (11f) and the frontal navigation cameras (11c ), and rear (11d). 5. - Autonomous automatic pruning device for vines, according to claim 1, characterized in that the artificial vision module (12) is configured to receive 2D images (39) of the vine generated by the depth cameras (9), where said 2D images (39) are combined with the point cloud images (20) to obtain the 3D model. 6. - Autonomous automatic pruning device for vines, according to claim 4, characterized in that the general control module (8) additionally comprises a remote control module (13), which in turn comprises WiFi / communication means Bluetooth (13a) and 4G / 5G (13b) with a user device (34) so that the remote control module (13) is configured so that the autonomous automatic pruning device for vines (1) works according to a mode of job selected from: • autonomous mode, in which the autonomous device (1) is configured to analyze the environment and auto-move within the environment by means of the propulsion system (5); • client / server mode, in which the standalone device (1) is configured to receive instructions in real time from the user device (34) and to execute said instructions in real time. 7. - Autonomous automatic pruning device for vines, according to claim 1, characterized in that the artificial vision module (12) is configured to: • clean noise from point cloud images (20) using filters; • separate the foreground from the background of the point cloud images (20) (using 5x5 matrices); • establish correspondences between the point cloud images (20) generated by the depth cameras; • detect obstacles (19) formed by wires, cables and trellis posts contained in the point cloud images (20); • detect a vine structure (18); • generate a 3D model (18) only of the vine, removing obstacles. 8. - Autonomous automatic pruning device for vines, according to claim 5, characterized in that the artificial vision module (12) is configured to: • clean noise from point cloud images (20) and from 2D images using filters; • separating the foreground from the background of the point cloud images (20) and the 2D images (using 5x5 matrices); • establish correspondences between the point cloud images (20) and between the 2D images generated by the depth cameras; • detect obstacles (19) formed by wires, cables and trellis poles contained in the point cloud images (20) and in the 2D images; • detect a vine structure (18); • generate a 3D model of the vine only, removing obstacles. 9. - Autonomous automatic pruning device for vines, according to claim 1, characterized in that the artificial intelligence module (14) is configured to: • interpret and study the vine morphologically; • rebuild the vine in a 3D model; • add the obstacles detected in the artificial vision module to the 3D model; • calculate and introduce some buds of the vineyards in the 3D model; • identify the branches to be cut; • Identify some cutting points to make on the vine. 10. - Autonomous automatic pruning device for vines, according to any one of the preceding claims, characterized in that the robotic arm (6,7) comprises a cutting element (22), an artificial vision camera (23) and a sensor laser (24) controlled by an on-board vision and cutting module (16) included in the general control module (8). 11. - Autonomous device of automatic pruning for vines, according to claim 10, characterized in that the robotic arm (6) comprises a fixed upper platform (25), to which three motors (26) are attached, of each of the which are suspended a pair of bars (27); Each pair of bars (27) is attached to a mobile platform (28), where the cutting element (22), the artificial vision camera (23) and the laser sensor (24) are integrated. 12. - Autonomous automatic pruning device for vines, according to claim 10, characterized in that the robotic arm (7) comprises an upper platform (32), which connects to a rotation motor of the arm (31) by one of the ends of the upper platform (32) by means of an upper motor (29a), which vertically moves the upper platform (32), the other end of the upper platform (32) is connected with a lower platform (33) driven by a motor lower (29b) through a strap (30) that allows the vertical movement of the lower platform (33), which is divided into elbow (33a), arm (33b), forearm (33c), wrist (33d), which are connected by an elbow motor (29d), a forearm motor (29c) and a wrist motor (29e), respectively; At the end of the lower platform (33) the cutting element (22), the artificial vision camera (23) and the laser sensor (24) are located. 13. - Autonomous automatic pruning device for vines, according to any of the preceding claims, characterized in that the robotic arms motor module (15) additionally comprises a trajectory generation sub-module (15a) that comprises a cost function for calculate the different cutting paths of each robotic arm (6,7). 14. - Autonomous automatic pruning device for vines, according to any of the preceding claims, characterized in that it additionally comprises a second cabin (3) and a second mobile platform (2).