WO2025014976A2 - System and method for automatically gripping and cutting fruits and plants - Google Patents

Abstract

A gripper assembly of a device for harvesting produce from a plant includes a blade, a pivot axis coupled to the blade, and a pair of fingers including a first finger and a second finger configured to pivot about the pivot axis. The gripper assembly may also include an actuation system, configured to actuate the first finger. The gripper assembly may also include a spring steel configured to bias the second finger to keep the second finger at a stable position when no force is exerted against the second finger, such that when the first finger is actuated, the pair of fingers are caused to open and close to grip a stem of a target produce item, or swipe over the blade to cause the blade to cut the stem of the target produce item to detach the target produce item from a plant.

Description

SYSTEM AND METHOD FOR AUTOMATICALLY GRIPPING AND CUTTING

FRUITS AND PLANTS

CROSS REFERENCE TO RELATED APPLICATION

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63/526,144, filed July 11, 2023, which is incorporated by reference herein in its entirety.

BACKGROUND

1. FIELD OF ART

[0002] The disclosure generally relates to the field of robotics, and more specifically to an automated produce-harvesting robot.

2. DESCRIPTION OF THE RELATED ART

[0003] Harvesting produce (such as fruits or vegetables) can be labor intensive, requiring individual detachment of each produce item from the plant. Further, delicate fruits and vegetables are prone to damage during the detachment process. Take strawberries, for instance, with their soft flesh that can easily get crushed if excessive force is exerted while picking. Damaged produce significantly diminishes their desirability and ultimately reduces their value.

[0004] Moreover, it is beneficial to reduce contact with the harvested fruits or vegetables due to food safety concerns. When handled manually, there is an increased risk of transferring mold or bacteria from one item to another. Hence, there is a pressing need for an automated tool that can delicately remove fruits or vegetables from plants and gently place them into trays or packages.

SUMMARY

[0005] Embodiments described herein relate to a gripper assembly of a device for harvesting produce from a plant. The gripper assembly includes a body, a blade coupled to the body, a pivot axis coupled to the body, and a pair of fingers including a first finger and a second finger, both configured to move about the pivot axis. In particular, the first finger or the second finger is configured to open or close to release or grip, respectively, a stem of a target produce item. While gripping the stem of the target produce item, the first finger and the second figure are configured to swipe over the blade, causing the blade to cut the stem of the target produce item to detach the target produce item from a plant.

[0006] Embodiments described herein also relate to an autonomous robot. The autonomous robot includes a robotic arm, the gripper assembly, and a camera configured to capture images. The autonomous robot also includes one or more processors, and a non-transitory computer-readable medium, storing instructions. When the instructions are executed by the one or more processors, the one or more processors cause the camera to capture images around the plant, analyze the captured images to identify the target produce item, and cause the robotic arm to move the gripper assembly toward the stem of the target produce item. In some embodiments, when the instructions are executed by the one or more processors, the one or more processors also cause the camera to capture images around a package, analyze the captured images to identify a placement position within the package, cause the robotic arm to move the gripper assembly toward the placement position, and cause the gripper assembly to release the detached target produce item at the placement position within the package.

BRIEF DESCRIPTION OF DRAWINGS

[0007] The disclosed embodiments have other advantages and features which will be more readily apparent from the detailed description, the appended claims, and the accompanying figures (or drawings). A brief introduction of the figures is below.

[0008] Figure (FIG.) 1A illustrates a perspective view of an autonomous robot for harvesting produce, according to one or more embodiments.

[0009] FIG. IB illustrates an exploded view of an autonomous robot for harvesting produce, according to one or more embodiments.

[0010] FIG. 1C illustrates an exploded view of a gripper assembly of an autonomous robot for harvesting produce.

[0011] FIG. 2A illustrates a top view of an autonomous robot for harvesting produce in a picking configuration, according to one or more embodiments. [0012] FIG. 2B illustrates a side view of an autonomous robot for harvesting produce in a picking configuration, according to one or more embodiments.

[0013] FIG. 3 A illustrates a top view of an autonomous robot for harvesting produce in a placement configuration, according to one or more embodiments.

[0014] FIG. 3B illustrates a side view of an autonomous robot for harvesting produce in a placement configuration, according to one or more embodiments.

[0015] FIG. 4 illustrates a top view of a gripper assembly of an autonomous robot for harvesting produce with its fingers closed and in stable position, according to one or more embodiments.

[0016] FIG. 5 A illustrates a top view of a gripper assembly of an autonomous robot for harvesting produce with its fingers open, according to one or more embodiments.

[0017] FIG. 5B illustrates a top view of a gripper assembly of an autonomous robot for harvesting produce with its fingers closed and in a biased position.

[0018] FIG. 6 illustrates a bottom view of a gripper assembly of an autonomous robot that performs a sequence of movements for cutting and holding a stem of a produce item, according to one or more embodiments.

[0019] FIG. 7 illustrates a perspective view of a gripper assembly of an autonomous robot for harvesting produce comprising a spring steel, according to one or more embodiments.

[0020] FIG. 8A and FIG. 8B illustrate an isometric view of an autonomous robot for harvesting produce, according to one or more embodiments.

[0021] FIG. 9A illustrates a perspective view of a gripper assembly on an autonomous robot performing a harvesting action sequence (from left to right) for detaching a produce item from a plant, according to one or more embodiments.

[0022] FIG. 9B illustrates a top view of a gripper assembly on an autonomous robot performing a harvesting action sequence (from left to right) for detaching a produce item from a plant, according to one or more embodiments.

[0023] FIG. 10 illustrates spring steel movement in a harvesting action sequence of an autonomous robot for detaching a produce item from a plant, according to one or more embodiments.

[0024] FIG. 11 illustrates a releasing action sequence of an autonomous robot for releasing a produce item detached from a plant, according to one or more embodiments. [0025] FIG. 12 illustrates spring steel movement in a releasing action sequence of an autonomous robot for releasing a produce item detached from a plant, according to one or more embodiments.

[0026] FIG. 13 illustrates an autonomous robot moving a gripper assembly to a target strawberry, according to one or more embodiments.

[0027] FIG. 14 illustrates the gripper assembly of an autonomous robot gripping a stem of a target strawberry, according to one or more embodiments.

[0028] FIG. 15 illustrates the gripper assembly of an autonomous robot cutting a gripped stem of a target strawberry to harvest the target strawberry, according to one or more embodiments.

[0029] FIG. 16 illustrates the gripper assembly of an autonomous robot moving a harvested strawberry to a placement position.

[0030] FIG. 17 illustrates the gripper assembly of an autonomous robot releasing a harvested strawberry at a placement position.

DETAILED DESCRIPTION

[0031] The Figures (FIGS.) and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.

[0032] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures. It is noted that wherever practicable similar or like reference numbers may be used in the figures and may indicate similar or like functionality. The figures depict embodiments of the disclosed system (or method) for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.

[0033] Embodiments described herein include a gripper assembly of an autonomous robot for harvesting produce, which not only reduces the risk of damaging the produce but also reduces the amount of handling required during the harvesting process.

[0034] The gripper assembly (also referred to as an end-effector assembly) of the device or the autonomous robot for harvesting produce includes a body, a blade coupled to the body, a pivot axis also coupled to the body, and a pair of fingers (including a first finger and a second finger) configured to move about the pivot axis. In particular, when the pair of fingers move about the pivot axis, the pair of fingers are configured to open or close to release or grip the stem of the target produce item. While gripping the stem of the target produce item, the pair of fingers swipe over the blade, causing the blade to cut the stem of the target produce item to detach the target produce item from a plant.

[0035] In some embodiments, the gripper assembly further includes an actuation system configured to actuate the first finger, causing the first finger to pivot about the pivot axis. In some embodiments, the gripper assembly further includes a spring configured to bias the second finger, causing the second finger to maintain at a stable position when no force is exerted against the second finger. When the second finger is at the stable position, the actuation system is configured to cause the first finger to pivot in a first pivot direction, which in turn causes the pair of fingers to open. Alternatively, the actuation system may cause the first finger to pivot in a second pivot direction toward the second finger, which in turn causes the pair of fingers to close. When the pair of fingers are closed, the actuation system may further cause the first finger to pivot further in the second pivot direction, causing both the first finger and the second finger to pivot together in the second pivot direction to swipe over the blade, which in turn causes the spring to bend.

[0036] In some embodiments, the actuation system includes a pinion gear, a rack, and an actuator. The pinion gear includes a first set of teeth, and the rack includes a second set of teeth configured to mesh with the first set of teeth. The actuator is configured to actuate the pinion gear, causing the pinion gear to rotate. When the pinion gear rotates, the first set of teeth and the second set of teeth mesh with each other, causing the rack to move in a linear direction. An end of the rack is coupled with a portion of the first finger. The portion of the first finger and the pivot axis do not overlap, such that when the rack moves in the linear direction, the portion of the first finger moves with the end of the rack in the linear direction, causing the first finger to pivot about the pivot axis. [0037] When the second finger is at the stable position, the pinion gear is configured to rotate in a first rotational direction, which causes the rack to move in a first linear direction, which in turn causes the first finger to pivot in a first pivot direction away from the second finger. As such, the pair of fingers are opened. Alternatively or in addition, the pinion gear is also configured to rotate in a second rotational direction, which causes the rack to move in a second linear direction opposite to the first linear direction, which in turn causes the first finger to pivot in a second pivot direction toward the second finger. As such, the pair of fingers are closed. [0038] When the pair of fingers are closed, the pinion gear may further rotate in the second rotational direction, which causes the rack to further move in the second linear direction, which in turn causes the second finger to pivot in the second direction, which causes both the first finger and the second finger to pivot together in the second direction to swipe over the blade. As such, a stem of a target produce item can be cut by the blade. Further, since the second finger is pushed by the first finger away from its stable position, the spring is caused to bend.

[0039] When the spring is bent, the pinion gear may rotate back in the first rotational direction, which causes the rack to move in the first linear direction, which in turn causes the first finger to pivot in the first pivot direction. At the same time, the bent spring causes the second finger to also pivot in the first pivot direction along with the first finger until the second finger reaches the stable position and the spring is no longer bent.

[0040] Because the spring is able to maintain the second finger at a stable position, while still allowing it to pivot with the first finger away from the stable position, only the first finger is actuated, reducing the mechanical complexity of the gripper assembly without compromising its performance.

[0041] In some embodiments, after the stem of the target produce item is cut, an actuator (which is a different actuator from the actuator that actuates the first finger) causes the gripper assembly to switch from a picking configuration (in which the fingers are oriented horizontally) to a placement configuration (in which the fingers are oriented vertically). The switching from the picking configuration to the placement configuration causes a position of the target produce item to switch from an upright position to a sideway position. Responsive to switching the gripper assembly to the placement configuration, the arm moves the gripper assembly to orient and aim the target produce item to a target pocket of a package, and the pair of fingers open to release the stem of the target produce item, placing the target produce item at the target pocket of the package.

[0042] In some embodiments, the device further includes a camera configured to identify the target produce item, and identify the stem of the target produce item. Responsive to identifying the stem of the target produce item, the arm moves the gripper assembly toward the stem of the target produce item.

[0043] Some existing gripper assemblies may include two grippers and two blades. The grippers are made from foam. Such foam grippers need to compress a fair amount to make two blades overlap to make a clean cut of the stem of a produce item. If the stem is thicker, the foam gripper needs to compress even further to make the cut. Meanwhile, the foam gripper needs to stick out further than the blades to grab onto the stem before cutting. The foam gripper loses grip force overtime as the foam plastically deforms and compresses.

[0044] Unlike such existing gripper assemblies, which rely on foam grippers, the embodiments described herein rely on the elasticity of the spring. By using spring, e.g., (but not limited to) spring steel or a spring steel beam, the embodiments described herein can develop a same consistent grip force over more cycles of loading. For example, unlike foam grippers, a beam of isotropic material or a spring will bend to an angle proportional to the force applied to it, deforming elastically. As such, the grip force increases as the first finger contacts the second finger, pivoting in the second “closing” direction past the point where the two fingers meet. The large angle the spring steel beam deflects, the further the two fingers are actuated in that direction by a motor, and the greater the grip force becomes. The embodiments described herein ensure enough deflection has been imparted on the spring, and therefore enough grip force is developed to hold the produce item before the blade completes a full cut through the stem. This provides a significant improvement over the existing foam gripper assemblies.

[0045] Some other existing gripper assemblies may have independently actuatable fingers and/or cut mechanisms. For example, such an existing gripper may include a pair of fingers positioned above or below a pair of cutting scissor blades that are activated sequentially to grip and cut a stem. Such an existing gripper is much more complex, and requires multiple forms of actuation and multiple overlapping blades, which need to be independently activated. Unlike such an existing gripper which requires multiple actuators and multiple overlapping blades, the embodiments described herein only include one actuator to actuate both the first finger and the second finger, and a single blade, reducing complexity, which in turn reduces manufacturing, operating, and/or maintenance costs. This provides a significant improvement in design compared to the existing gripper assemblies.

[0046] FIG. 1 A illustrates a perspective view of an autonomous robot 100 for harvesting produce, according to one or more embodiments. FIG. IB illustrates an exploded view of an autonomous robot 100, according to one or more embodiments. The autonomous robot 100 includes a gripper assembly 001, a pitch axis actuator 002 configured to rotate the gripper assembly 001, a pitch axis actuator horizontal mounting bracket 003 configured to mount the pitch axis actuator 002 onto the gripper assembly 001, an end-effector camera 004 configured to take images of surrounding areas, an end-effector camera USB cable 005 configured to transmit data between the end-effector camera 004 and a computing device (not shown), a robotic arm 006 configured to move the gripper assembly 001, and an end-effector tool flange 007 configured to attach the camera 004 and the arm 006 to the gripper assembly 001.

[0047] FIG. 1C illustrates an exploded view of the gripper assembly 001, according to one or more embodiments. The gripper assembly 001 includes a first finger 008, a second finger 009, a spring steel 010, a blade 011, a rack cover 012, a pinion gear 013, a gripper finger actuator 014 (e.g., a motor), a rack 015, a gear cover 016, a pitch axis actuator hinge frame 017, a gripper finger actuator vertical mounting bracket 018, and one or more shoulder bolts 019. In some embodiments, there are five shoulder bolts, including a first shoulder bolt for the pivot axis that couples the first and the second fingers, a second shoulder bolt for the pivot axis that couples the first finger and the end of the rack, and three additional shoulder bolts for locking the rack to be only movable in a linear direction.

[0048] FIG. 2 A illustrates a top view of an autonomous robot 100 for harvesting produce in a picking configuration, according to one or more embodiments. FIG. 2B illustrates a side view of an autonomous robot 100 for harvesting produce in a picking configuration, according to one or more embodiments. When the autonomous robot 100 is in the picking configuration, the autonomous robot 100 is configured to identify a target produce item, and a stem of the target produce item; move the gripper assembly 001 toward the stem of the target produce item; and cause the fingers 008, 009 of the gripper assembly 001 to open or close to grip or release the stem of the target produce item.

[0049] FIG. 3 A illustrates a top view of an autonomous robot for harvesting produce in a placement configuration, according to one or more embodiments. FIG. 3B illustrates a side view of an autonomous robot for harvesting produce in a placement configuration, according to one or more embodiments. When the autonomous robot 100 is in the placement configuration, the autonomous robot 100 is configured to cause the fingers 008, 009 of the gripper assembly 001 to continue to grip the stem of the target produce item; move the gripper assembly 001 to a placement position; and cause the fingers 008, 009 of the gripper assembly 001 to open to release the stem of the target produce item, placing the target produce item at the placement position.

[0050] In some embodiments, the camera 004 is configured to rotate among multiple positions. When the autonomous robot 100 is in the picking configuration, the gripper assembly 001 is oriented in a first orientation, causing the fingers to be oriented in a horizontal direction, and the camera 004 is rotated to a first position facing a horizontal direction where the plant is in a field of view. When the autonomous robot 100 is in the placement configuration, the gripper assembly is oriented in a second orientation, causing the fingers to be oriented in a vertical direction, and the camera 004 is rotated to a second position facing a vertical direction where a reference of the placement position is in the field of view.

[0051] Labels DI through D13 in FIGs. 2A-2B and 3A-3B denote various dimensions of certain mechanical components of the gripper assembly 001. In an example embodiment, the dimensions may be as follows: DI is approximately 17.55 mm, D2 is approximately 6.20 mm, D3 is approximately 169.20 mm, D4 is approximately 154.15 mm, D5 is approximately 85.30 mm, D6 is approximately 5.00 mm, D7 is 18.15 mm, D8 is 7.15 mm, D9 is approximately 9.15 mm, D10 is approximately 72.11 mm, Dll is approximately 173.86 mm, D12 is approximately 26.00 mm, and D13 is approximately 33.50 mm. The term "approximately" is used to indicate that the specified dimensions may vary slightly, typically due to manufacturing tolerances.

[0052] It should be noted that the example embodiment and dimensions described above are illustrative in nature and are not intended to be limiting or exclusive. Depending on a size range of the target produce that the gripper assembly 001 is intended to harvest, alternative dimensions may be implemented to commodate specific use cases. Such variations are encompassed within the scope of the present disclosure.

[0053] FIG. 4 illustrates a top view of a gripper assembly of an autonomous robot for harvesting produce, according to one or more embodiments. As shown in FIG. 4, the gripper assembly 001 includes a rack 015 and a pinion gear 013 configured to transfer motor power efficiently to the fingers 008, 009. The gripper assembly 001 also includes a blade Oil overlapping the second finger 009. The second finger 009 is biased by a spring steel 010. The spring of the spring steel 010 keeps the second finger 009 at its stable position when no force is exerted against the second finger 009. [0054] Figs 5A and 5B illustrate a motion range of a gripper assembly 001 of an autonomous robot for harvesting produce, according to one or more embodiments. In particular, FIG. 5A shows a top view of the gripper assembly 001 with the fingers 008, 009 open, and ready to grip a plant stem, according to one or more embodiments. As the fingers 008, 009 transition from the open position shown in FIG. 5A to the closed and stable position shown in FIG. 4, they can effectively grip a plant stem.

[0055] FIG. 5B shows a top view of the gripper assembly 001 with the fingers 008, 009 closed and biased against the spring steel 010, according to one or more embodiments. When the fingers 008, 009 shift from the closed position shown in FIG. 4 to the biased position shown in FIG. 5B, a plant stem gripped by the fingers 008, 009 will slide across the blade Oil, resulting in the blade cutting the plant stem.

[0056] The movements of the fingers 008, 009 are driven by a movement of rack 015 and the pinion gear 013. The rack 015 is a bar having teeth on one side, and the pinion gear 013 is a rotatable wheel having teeth configured to mesh with the teeth on the rack. The pinion gear 013 is powered by a gripper finger actuator 014 (not shown in FIG. 5 A or 5B) and configured to rotate about an axis 117. When the pinion gear 013 rotates about the axis 117, the teeth of the pinion gear 013 and the teeth of the rack 015 mesh with each other to translate the rotational movement of the pinion gear 013 to a linear movement of the rack 015. An end of the rack 015 is attached to a connection portion 113 of the finger 008. Note, the connection portion 113 of the first finger 008 attached to the rack 015 does not overlap the pivot axis 121 about which the fingers 008, 009 pivot, such that when the rack 015 moves left or right, the connection portion 113 is caused to move left or right, respectively, which in turn causes the finger 008 to pivot about the pivot axis 121.

[0057] Referring to FIG. 5A, when the rack 015 moves towards the right, the first finger 008 is caused to pivot counterclockwise away from the second finger 009, causing the fingers 008, 009 to open. When the rack 015 moves towards left, the finger 008 is caused to pivot clockwise toward the second finger 009, causing the fingers 008, 009 to close. When the rack 015 moves further towards the left, the closed fingers 008, 009 rotate together even further to bring a stem through the blade 011 and cause the spring steel 010 to deform, as shown in FIG. 5B. When the rack 015 moves towards the right again, the first finger 008 opens, while the spring steel 010 pushes the second finger 009 to its stable position to prevent the second finger from rotating further beyond its stable position, as shown in FIG. 5A.

[0058] As shown in FIG. 5B, the second finger 009 is biased by the spring steel 010. When the first finger 008 is caused to move clockwise beyond the stable position of the second finger 009, the first finger 008 exerts a force against the second finger 009, causing the second finger 009 to also pivot about the pivot axis 121 together with the first finger 008, swiping over the blade 011, as shown in FIG. 5B. On the other hand, when the first finger 008 is caused to move counterclockwise, the spring steel 010 pushes the second finger 009 to follow the first finger 008 to move back to its stable position in parallel with the rack 015, as shown in FIG. 4. As such, the rack 015 and pinion gear 013 are configured to trigger the movement of both the first finger 008 and second finger 009. [0059] FIG. 6 illustrates a bottom view of a gripper assembly of an autonomous robot that performs a sequence of movements for cutting and holding a stem of a produce item, according to one or more embodiments. The gripper assembly 001 includes a pivot axis 121. Both fingers 008, 009 are able to move about the same pivot axis 121. Further, the first finger 008 has a connection portion 113 that is coupled with a rack (not shown in FIG. 6 or FIG. 7). Notably, the pivot axis 121 and the connection portion 113 do not overlap each other, such that when the connection portion 113 moves linearly with the rack 015, the first finger 008 is caused to pivot about the pivot axis 121. As shown, only finger 008 is actuated by the rack 015, but the first finger 008 can push the second finger 009 to move together with the first finger 008 around the same pivot axis 121. The fingers 008, 009 that grip a stem swipe over the blade Oil, causing the stem to be cut by the blade 011. After the swipe motion, the first finger 008 moves back to release the stem, while the spring steel 010 pushes the second finger 009 back to its stable position.

[0060] FIG. 7 illustrates perspective views of a gripper assembly 001 of an autonomous robot for harvesting produce comprising a spring steel 010, according to one or more embodiments. The spring steel 010 is coupled to the second finger 009. The spring steel 010 is elastic. In FIG. 7, depicted on the left side, when a force is exerted against the spring steel 010, the spring steel 010 deforms (e.g., bends) to allow the second finger 009 to pivot about the pivot axis 121 (not shown in FIG. 7). This mechanical action results in a stem of a produce item being cut after it is gripped by the fingers 008, 009.

[0061] During the gripping process, the rack 015 drives the first finger 008 to pivot about the pivot axis 121. Once the first finger 008 contacts the second finger 009, the fingers 008, 009 secure a plant stem between them. Subsequent to gripping the stem, a cutting of the stem may be performed. During the cutting process, the rack 015 continues to drive the first finger 008 to pivot about the pivot axis 121. This movement forces the first finger 008 to press against the second finger 009, which in turn presses against the spring steel 010. As a result, the spring steel 010 bends, permitting the second finger 009 to pivot with the first finger 008. When the first and second fingers 008, 009 pivot further, the blade Oil maintains its position, such that the plant stem secured between the fingers 008, 009 slides across the stationary blade Oil, facilitating a cutting operation.

[0062] In FIG. 7, on the right, following the cutting of the stem, the rack 015 retracts, which causes the first finger 008 to pivot back. This action reduces and eventually eliminates the force applied to the second finger 009, which in turn reduces and eventually eliminates the force exerted against the spring steel 010. When no force is exerted against the spring steel 010, the spring steel 010 returns to its stable position, consequently the second finger 009 also moves back to its stable position in parallel with the rack 015. Additional details about the operation of spring steel 010 are described below with respect to FIGS. 10 and 12.

[0063] As explained with reference to FIGS. 6-7, the spring steel 010 is capable of pushing the second finger 009 back to its stable position during the release motion of the gripper assembly 001, such that a separate actuation system is not required to move the second finger 009, and the same actuation system (e.g., the gripper finger actuator 014, the rack 015 and pinion gear 013) can control the relative positions of the second finger 009 and first finger 008, and cause both the second finger 009 and first finger 008 to move and pivot.

[0064] FIG. 8A and FIG. 8B illustrate an isometric view of an autonomous robot 100 for harvesting produce, according to one or more embodiments. The autonomous robot 100 includes a gripper assembly 001, a pitch axis actuator 002, one or more actuator adapters (e.g., pitch axis actuator horizontal mounting bracket 003, pitch axis actuator hinge frame 017, gripper finger actuator vertical mounting bracket 018, shown in FIGS. IB and 1C), end-effector tool flange 007, and end-effector camera 004. In some embodiments, the overall design of the end effector is pointy and long, which helps the tool to enter in between stems.

[0065] FIG. 9A illustrates a perspective view of a gripper assembly on an autonomous robot performing a harvesting action sequence (from left to right) for detaching a produce item from a plant, according to one or more embodiments. FIG. 9B illustrates a top view of the gripper assembly on the autonomous robot performing the harvesting action sequence (from left to right) for detaching the product item from a plant, according to one or more embodiments.

[0066] Only finger 008 gets actuated by the gripper finger actuator 014 (e.g., a motor). Once the finger 008 touches the stem, the other finger 009 is caused to rotate together with the finger 008, pushing the spring steel 010 outward. During the swipe motion, the stem gets swiped over the blade Oil and gets cut. The spring steel 010 deflection provides the force between fingers 008, 009 to hold onto the stem while the stem gets cut. After the stem gets cut, the force between fingers is provided by the motor 014.

[0067] For example, first, a rack 015 coupled with a first finger 008 is actuated in a second linear direction 902, causing the first finger 008 to pivot in a second pivot direction 904 toward a second finger 009 (biased by spring steel 010) to grip a stem of a target produce item (e.g., a strawberry), shown in the left and middle images of FIG. 9A or FIG. 9B. After that, the rack 015 is further actuated in the second linear direction 902, causing the first finger 008 to move further in the second pivot direction 904, which in turn causes the spring steel 010 to deform, such that the second finger 009 pivots together with the first finger 008 to swipe the stem over the blade 011 to cause the stem to be cut by the blade Oil, shown in the right image of FIG. 9 A or FIG. 9B.

[0068] FIG. 10 illustrates spring movement in a harvesting action sequence of an autonomous robot for detaching a produce item from a plant, according to one or more embodiments. First, the spring steel 010 and the second finger 009 are in a stable position (also referred to as a neutral position) before and after the fingers 008, 009 grip the stem of the target produce item, shown in the left and middle images of FIG. 10. After that, the force from the first finger 008 acts against the second finger 009, causing the second finger 009 to pivot in the second pivot direction 904 with the first finger 009, which in turn causes the spring steel 010 to bend, shown in the right image of FIG. 10.

[0069] FIG. 11 illustrates a releasing action sequence of an autonomous robot for releasing a produce item detached from a plant, according to one or more embodiments. Only finger 008 gets actuated by a gripper finger actuator 014 (not shown in FIG. 11). As the gripper finger actuator 014 rotates the pinion gear 013 counterclockwise 910 (also referred to as a first rotation direction), the rack 015 moves backward in the first linear direction 906, which causes the first finger 008 to pivot in the first pivot direction 908 and open. As the first finger 008 opens, the second finger 008 gets pushed to its stable position by the spring steel 010. When the fingers 008, 009 lose grip on the stem, the produce item gets released.

[0070] As illustrated, in some embodiments, during the releasing action, the fingers 008, 009 are in a vertical direction. First, the rack 015 is actuated in a first linear direction 906, causing the first finger 008 to pivot in a first pivot direction 908, while the second finger 009 is pushed by the spring steel 010 back to its resting position, shown in the left image of FIG. 11. Further, the rack 015 is actuated further in the first linear direction 906, causing the first finger 008 to pivot away from the second finger 009 in the first pivot direction 908 while the second finger 009 maintains its position by the spring steel 010, thereby opening the fingers 008, 009 and releasing the stem of the target produce item, as shown in the middle and right images of FIG. 11.

[0071] FIG. 12 further illustrates spring movement in a releasing action sequence of an autonomous robot for releasing a produce item detached from a plant, according to one or more embodiments. First, the spring steel 010 bends, and the force acts against the second finger 009, shown in the left and middle images of FIG. 12. After that, the first finger 008 moves, and the second finger 009 biased by the spring steel moves with the first finger 008 to its stable position where the spring steel is flat at the neutral position, as shown in the right image of FIG. 12. [0072] FIG. 13 illustrates an autonomous robot moving a gripper assembly 001 to a target strawberry, according to one or more embodiments. In some embodiments, this is a first step of the harvesting action. The gripper 001 approaches the target fruit. In some embodiments, the fingers 008, 009 are closed initially to maintain the pointy tip in order to enter a tight gap between stems. In some embodiments, the autonomous robot includes one or more processors, and a non-transitory computer-readable medium, storing instructions. When the instructions are executed by the one or more processors, the autonomous robot causes the camera 004 to capture images around a target plant, analyzes the captured images to identify the target produce item, and causes the robotic arm to move to the gripper assembly 001 toward the stem of the target produce item and cause the gripper finger actuator 014 to move and as a result move the gripper assembly 001 and open or close fingers 008 , 009 to grip and cut the stem of the target produce item as described herein above with reference to FIGs. 4, 5A, 5B, 6, 7, 9A, 9B, and 10-17.

[0073] In some embodiments, images featuring different produce items are collected and labeled. Machine learning models (e.g., object detection models) are trained over these images. For example, a first machine learning model can be trained to detect strawberries that are ripe for harvest, a second model can be trained to identify ripe tomatoes, and additional models can be developed to recognize other types of produce.

[0074] Various machine learning techniques may be used to train the models, such as convolutional neural networks (CNNs), region-based CNN (R-CNN), You Only Look Once (YOLO), Single Shot Multibook Detector (SSD), U-Net, Mask R-CNN, Neural Architecture Search (NAS), transformers in vision, ensemble techniques, among others. The trained machine learning models may be deployed onto the autonomous robot and/or the gripper assembly 001. The autonomous robot or the gripper assembly 001 uses the machine learning models to process a captured image to identify a produce item.

[0075] FIG. 14 illustrates an autonomous robot capturing a target stem of a target strawberry, according to one or more embodiments. In some embodiments, this is a second step of the harvesting action. The gripper 001 opens the left finger 008, and moves forward to put the stem in between gripper fingers 008, 009. The gripper 001 then moves downward to minimize the stem length after cutting. The end-effector camera 004 is used to fine adjust the position of the gripper 001 to ensure a better grip.

[0076] FIG. 15 illustrates an autonomous robot cutting a captured target stem of a target strawberry to harvest the target strawberry, according to one or more embodiments. In some embodiments, this is a third step of the harvesting action. In some embodiments, in continuous motions, the first finger 008 closes, grabs the stem, pushes the second finger 009 and the spring steel 010 together, swipes the stem over the blade Oil, and cuts the stem.

[0077] FIG. 16 illustrates an autonomous robot moving a harvested strawberry to a placement position, according to one or more embodiments. FIG. 17 illustrates an autonomous robot releasing a harvested strawberry at a placement position, according to one or more embodiments. This is the last step of the harvesting action. The pitch axis actuator 002 moves the gripper assembly 001 downward. The robotic arm 006 rotates and moves the gripper assembly 001 to place the strawberry in a placement position 162 (e.g., a correct pocket) of a package 160. The first gripper finger 008 opens and packs the produce item in its package 160.

[0078] In some embodiments, the autonomous robot causes the camera 004 to capture images around a package 160, and analyzes the captured images to identify a placement position 162 within the package 160. Responsive to identifying the placement position 162 within the package 160, the autonomous robot causes the robotic arm 006 to move the gripper assembly 001 toward the placement position 162, which may include three-dimensional movements. When the fingers 008, 009 are positioned above the placement position 162 within a threshold height, the autonomous robot causes the gripper assembly 001 to release the detached produce item at the placement position 162 within the package 160.

[0079] In some embodiments, images featuring different packages and placement positions within the packages are collected and labeled. Machine learning models (e.g., object detection models) are trained over these images to identify a package and a placement position within the package. For example, a first machine learning model may be trained to detect strawberry packages, and a second machine learning model may be trained to detect tomato packages, and so on.

[0080] Again, various machine learning techniques, including those described above, may be used to train these models. The trained machine learning models may be deployed onto the autonomous robot and/or the gripper assembly 001. The autonomous robot or the gripper assembly 001 uses the machine learning models to process a captured image to identify a package and a placement position within the package.

[0081] In some embodiments, during a picking operation, the autonomous robot applies a first set of machine learning models trained to identify produce items. Responsive to identifying a produce item by the first set of machine learning models, the autonomous robot causes the gripper assembly 001 to approach the produce item, grip a stem of the produce item, and separate the produce item from its plant. In some embodiments, during the picking operation, the autonomous robot causes the fingers 008, 009 of the gripper assembly to be oriented horizontally.

[0082] After the produce item is separate from the plant, the autonomous robot performs a placement operation. During a placement operation, the autonomous robot applies a second set of machine learning model trained to identify packages. Responsive to identifying a package 160 and a placement position 162 in the package 160, the autonomous robot causes the gripper assembly 001 to move to the placement position 162 in the package 160 and release the produce item at the placement position 162 within the package 160. In some embodiments, during the placement operation, the autonomous robot causes the fingers 008, 009 of the gripper assembly 001 to be oriented vertically.

Additional Considerations

[0083] Reference in the specification to “one embodiment” or to “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least one embodiment. The appearances of the phrase “in one embodiment” or “an embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

[0084] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps (instructions) leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared and otherwise manipulated. It is convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. Furthermore, it is also convenient at times, to refer to certain arrangements of steps requiring physical manipulations or transformation of physical quantities or representations of physical quantities as modules or code devices, without loss of generality.

[0085] However, all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or the like, refer to the action and processes of a computer system, or similar electronic computing device (such as a specific computing machine), that manipulates and transforms data represented as physical (electronic) quantities within the computer system memories or registers or other such information storage, transmission or display devices.

[0086] Certain aspects of the embodiments include process steps and instructions described herein in the form of an algorithm. It should be noted that the process steps and instructions of the embodiments can be embodied in software, firmware or hardware, and when embodied in software, could be downloaded to reside on and be operated from different platforms used by a variety of operating systems. The embodiments can also be in a computer program product which can be executed on a computing system.

[0087] The embodiments also relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the purposes, e.g., a specific computer, or it may comprise a computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic- optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, application specific integrated circuits (ASICs), or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. Memory can include any of the above and/or other devices that can store information/data/programs and can be transient or non-transient medium, where a non-transient or non-transitory medium can include memory/storage that stores information for more than a minimal duration. Furthermore, the computers referred to in the specification may include a single processor or may be architectures employing multiple processor designs for increased computing capability.

[0088] The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various systems may also be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the method steps. The structure for a variety of these systems will appear from the description herein. In addition, the embodiments are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the embodiments as described herein, and any references herein to specific languages are provided for disclosure of enablement and best mode. [0089] Throughout this specification, some embodiments have used the expression “coupled” along with its derivatives. The term “coupled” as used herein is not necessarily limited to two or more elements being in direct physical or electrical contact. Rather, the term “coupled” may also encompass two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other, or are structured to provide a thermal conduction path between the elements.

[0090] Likewise, as used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0091] In addition, use of the “a” or “an” are employed to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of embodiments. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise. The use of the term and/or is intended to mean any of: “both”, “and”, or “or.”

[0092] In addition, the language used in the specification has been principally selected for readability and instructional purposes, and may not have been selected to delineate or circumscribe the inventive subject matter. Accordingly, the disclosure of the embodiments is intended to be illustrative, but not limiting, of the scope of the embodiments.

[0093] While particular embodiments and applications have been illustrated and described herein, it is to be understood that the embodiments are not limited to the precise construction and components disclosed herein and that various modifications, changes, and variations may be made in the arrangement, operation, and details of the methods and apparatuses of the embodiments without departing from the spirit and scope of the embodiments.

Claims

CLAIMS WHAT IS CLAIMED IS:

1. A gripper assembly of a device for harvesting produce from a plant, the gripper assembly comprising: a body; a blade coupled to the body; a pivot axis coupled to the body; and a pair of fingers including a first finger and a second finger, both configured to move about the pivot axis to: open or close to release or grip, respectively, a stem of a target produce item; and while gripping the stem of the target produce item, swipe over the blade, causing the blade to cut the stem of the target produce item to detach the target produce item from a plant.

2. The gripper assembly of claim 1, further comprising: an actuation system configured to actuate the first finger, causing the first finger to pivot about the pivot axis.

3. The gripper assembly of claim 2, further comprising a spring steel configured to bias the second finger, causing the second finger to maintain at a stable position when no force is exerted against the second finger.

4. The gripper assembly of claim 3, wherein: when the second finger is at the stable position, the actuation system is configured to cause the first finger to pivot in a first pivot direction, which in turn causes the pair of fingers to open; or when the second finger is at the stable position, the actuation system is configured to cause the first finger to pivot in a second pivot direction toward the second finger, which in turn causes the pair of fingers to close.

5. The gripper assembly of claim 4, wherein: when the pair of fingers are closed, the actuation system is configured to cause the first finger to pivot further in the second pivot direction, which in turn causes the second finger to pivot with the first finger in the second pivot direction to swipe over the blade, which in turn causes the spring steel to bend.

6. The gripper assembly of claim 4, wherein: when the spring steel is bent, the actuation system is configured to cause the first finger to pivot in the first pivot direction, which in turn causes the second finger biased by the bent spring steel to pivot with the first finger in the first pivot direction until the second finger reaches the stable position.

7. The gripper assembly of claim 3, the actuation system comprising: a pinion gear comprising a first set of teeth; a rack comprising a second set of teeth configured to mesh with the first set of teeth; and an actuator configured to actuate the pinion gear, causing the pinion gear to rotate; wherein when the pinion gear rotates, the first set of teeth and the second set of teeth mesh with each other, causing the rack to move in a linear direction, wherein an end of the rack is coupled with a portion of the first finger, and the portion of the first finger and the pivot axis do not overlap, such that when the rack moves in the linear direction, the portion of the first finger moves with the end of the rack in the linear direction, causing the first finger to pivot about the pivot axis.

8. The gripper assembly of claim 7, wherein: when the second finger is at the stable position, the pinion gear is configured to rotate in a first rotational direction, which causes the rack to move in a first linear direction, which in turn causes the first finger to pivot in a first pivot direction away from the second finger, which in turn causes the pair of fingers to open; and when the second finger is at the stable position, the pinion gear is configured to rotate in a second rotational direction, which causes the rack to move in a second linear direction opposite to the first linear direction, which in turn causes the first finger to pivot in a second pivot direction toward the second finger, which in turn causes the pair of fingers to close.

9. The gripper assembly of claim 8, wherein: when the pair of fingers are closed, the pinion gear is configured to rotate further in the second rotational direction, the rack further moves in the second linear direction, and the first finger further pivots in the second pivot direction, causing both the first finger and the second finger to pivot together in the second pivot direction to swipe over the blade, which in turn causes the spring steel to bend.

10. The gripper assembly of claim 8, wherein: when the spring steel is bent, the pinion gear is configured to rotate in the first rotational direction, the rack moves in the first linear direction, the first finger pivots in the first pivot direction, and the spring steel causes the second finger to also pivot in the first pivot direction along with the first finger until the second finger reaches the stable position and the spring steel is no longer bent.

11. An autonomous robot, comprising: a robotic arm; a gripper assembly of claim 1 coupled to the robotic arm; a camera configured to capture images; one or more processors; and a non-transitory computer-readable medium, storing instructions, when the instructions are executed by the one or more processors, causing the autonomous robot to: capture images around the plant by the camera; analyze the captured images to identify the target produce item; and cause the robotic arm to move the gripper assembly toward the stem of the target produce item.

12. The autonomous robot of claim 11 , wherein the instructions further cause the autonomous robot to: capture images around a package by the camera; analyze the captured images to identify a placement position within the package; cause the robotic arm to move the gripper assembly toward the placement position; and cause the gripper assembly to release the detached target produce item at the placement position within the package.

13. The autonomous robot of claim 12, wherein the one or more processors are further configured to: move the robotic arm to cause the fingers of the gripper assembly to be oriented horizontally when the gripper assembly is picking the target produce item, and move the robotic arm to cause the fingers of the gripper assembly to be oriented vertically when the gripper assembly is placing the target produce item in the placement position.