WO2021198731A1

WO2021198731A1 - An artificial-intelligence-based method of agricultural and horticultural plants' physical characteristics and health diagnosing and development assessment.

Info

Publication number: WO2021198731A1
Application number: PCT/IB2020/053083
Authority: WO
Inventors: Soroush SARABI; Mohammadreza Mohammadi; Ali SARABI; Mehdi RAZPOUSH NAZARI; Ameneh SHADLO; Ali SOLTANMORADI; Hanieh TAVASOLI
Original assignee: Individual
Current assignee: Individual
Priority date: 2020-04-01
Filing date: 2020-04-01
Publication date: 2021-10-07
Anticipated expiration: 2022-10-01

Abstract

a method of developing plant's physical characteristics data collecting by imaging methods for specific agricultural and horticultural products to be utilized in conjunction with any using machine learning, machine vision, and deep learning methods (all of the deep learning algorithms) to diagnose plants physical characteristics status.wherein the pixel matrix in the image is defined by deep learning techniques to pre-learned models of pixel values and the image library comprises images of agricultural and horticultural product's physical characteristics. wherein the problems are any physical plant problem (the diseases include any disease identifiable by visual inspection of plant leaves, stems, and entire plant or any physical changes). wherein extracting relevant images is performed by deep learning and machine vision methods and the pixel model results from multi images from a plurality of cameras received from an image capture device within the agricultural and horticultural products.

Description

An artificial-intelligence-based method of agricultural and horticultural plants' physical characteristics and health diagnosing and development assessment.

a method of developing plant's physical characteristics data collecting by imaging methods for specific agricultural and horticultural products to be utilized in conjunction with any using machine learning, machine vision, and deep learning methods (all of the deep learning algorithms) to diagnose plants physical characteristics status.wherein the pixel matrix in the image is defined by deep learning techniques to pre-learned models of pixel values and the image library comprises images of agricultural and horticultural product's physical characteristics.

wherein the problems are any physical plant problem (the diseases include any disease identifiable by visual inspection of plant leaves, stems, and entire plant or any physical changes). wherein extracting relevant images is performed by deep learning and machine vision methods and the pixel model results from multi images from a plurality of cameras received from an image capture device within the agricultural and horticultural products.

Physics (G), computing; calculating; counting (G06), image data processing or generation, in general (G06T) and Artificial intelligence.

The United Nations FAO (Food and Agriculture Organization) states that the world population would increase by another 2 billion in 2050 while the additional land area under cultivation will only account to 4% at that time. In such circumstance more efficient farming practices can be attained using the recent technological advancements and solutions to current bottlenecks in farming. A direct application of AI (Artificial Intelligence) or machine intelligence across the farming sector could act to be an epitome of shift in how farming is practiced today. Farming solutions which are AI powered enables a farmer to do more with less, enhancing the quality, also ensuring a quick GTM (go-to-market strategy) strategy for crops. There are patents in the field that we are considering:

method and apparatus for recommending food and drink based on artificial intelligence based user status

Document Type and Number: United States Patent Application 20200042865

A method and a device for recommending food based on an artificial intelligence based user status are disclosed. The method includes obtaining use information from a mobile terminal and an external terminal connected to the mobile terminal, determining a user status through an AI device, and determining a preferred food of the user based on the user status, thereby providing convenience for the user's life. The device for recommending food based on the artificial intelligence based user status can be associated with an artificial intelligence module, an unmanned aerial vehicle (UAV), a robot, an augmented reality (AR) device, a virtual reality (VR) device, devices related to 5G services, and the like.

In this patent for recommending, by an artificial intelligence based mobile terminal, food based on an artificial intelligence based user status, the method comprising. wherein the user status includes at least one of an emotional condition or a healthy condition. But our patent, we use artificial-intelligence-based method of agricultural and horticultural plants' physical characteristics and health diagnosing and development assessment.

Harvest advisory modeling using field-level analysis of weather conditions and observations and user input of harvest condition states and tool for supporting management of farm operations in precision agriculture

United States Patent 9292796

A modeling framework for evaluating the impact of weather conditions on farming and harvest operations applies real-time, field-level weather data and forecasts of meteorological and climatological conditions together with user-provided and/or observed feedback of a present state of a harvest-related condition to agronomic models and to generate a plurality of harvest advisory outputs for precision agriculture. A harvest advisory model simulates and predicts the impacts of this weather information and user-provided and/or observed feedback in one or more physical, empirical, or artificial intelligence models of precision agriculture to analyze crops, plants, soils, and resulting agricultural commodities, and provides harvest advisory outputs to a diagnostic support tool for users to enhance farming and harvest decision-making, whether by providing pre-, post-, or in situ-harvest operations and crop analyzes.

The project associated with harvest advisory modeling using field-level analysis of weather conditions and observations and user input of harvest condition states and tool for supporting management of farm operations in precision agriculture but our project about the plan is to diagnose diseases of plants and trees in gardens and agricultural fields and through drones and artificial intelligence can be detected and treated plant disease. Many inventions have been registered in the field of precision agriculture, but not all of the inventions have been designed to assess and diagnose plant diseases based on artificial intelligence.

assessing plant physiology, which includes soil quality, water, temperature, nutrition, etc. is of great importance in understanding plant status and plant damage. Mastering the physical database of plants, especially agricultural and horticultural products, allows us to come up with more precise guidelines on how to improve the growth process and identify plant damage. There are several ways to do this. In the traditional way, quality control supervisors and specialist human resources with Observing the plant closely and according to their experience and expertise determine the status of the plant. Doing closely so through this method is very time-consuming, and there is a possibility of human error.

Our purpose is to provide a way based upon the combined application of ICT solutions such as precision equipment, the Internet of Things (IoT), sensors and actuators, geo-positioning systems, Big Data, Unmanned Aerial Vehicles (UAVs, drones), robotics, etc. that assessment the physical properties of plants faster and more efficiently and Provide value-added to farmers in the form of better decision-making or more efficient operation and management. the method comprising in 3 ways: 1.Flying or moving the unmanned vehicle in the agricultural and horticultural products corridors, 2.Positioning the unmanned vehicle (with or without robotic arms) in the height of plants to capture whole or part of plant body, 3.Capturing images via the cameras, the images of whole or part of plant body agricultural and horticultural plants, and processing the images to diagnosing plant physical characteristics parameters.

Physiology is the study of all the processes happening in living organisms, such as respiration, excretion and in the case of plants, photosynthesis, transpiration, etc. In the case of agricultural and horticultural productions, their genetic make-up and environmental factors play a significant role in regulating their growth.

Understanding plant physiology is the key to successful agriculture. Plant physiology studies processes that determine plant growth, development, and economic production. Physiologist draws information from fundamental research and works on the whole plant level, solving practical agriculture problems, which limit plant growth and overall production. For example, how various environmental factors influence, nutrient/water uptake, air exchange, photosynthesis/respiration, and production and partitioning of different resources affecting growth.

Once the breeder understands the physiology of plants they are trying to cultivate, it becomes easy for them to deal with problems such as environmental stress as well as insect invasions. It is the knowledge of genetics and environment and their effects on agricultural and horticultural productions that made it possible for scientists to develop breeds of agricultural and horticultural productions that can give yield with good quality and quantity. The environment is not always in favor of plants, as there are multiple ways through which the environment causes adverse effects on agricultural and horticultural productions. These unfavorable environmental factors affect the growth of plants by altering their physiological processes.

The only way plant cultivators can achieve higher-yielding varieties is by designing genotype that has the combination of best physiological processes resulting in agricultural and horticultural productions that are structurally and physiologically more effective in a particular environment.

When we study plant physiology, we can observe the activities of plants in relation to their external environmental conditions, and we can well predict- How a plant will behave under different environmental conditions.

Plant physiology is usually divided into three major parts:

The physiology of nutrition and metabolism, which deals with the uptake, transformations, and release of materials, and also their movement within and between the cells and organs of the plant.

The physiology of growth, development, and reproduction, which is concerned with these aspects of plant function

Environmental physiology, which seeks to understand the manifold responses of plants to the environment. The part of environmental physiology which deals with effects of and adaptations to adverse conditions and which is receiving increasing attention is called stress physiology.

In some cases, poor environmental conditions damage a plant directly. In other cases,environmental stress weakens a plant and makes it more susceptible to disease or insect attack.

Environmental factors that affect plant growth include light, temperature, water, humidity, and nutrition. It is important to understand how these factors affect plant growth and development.

A mineral element is considered essential to plant growth and development if the element is involved in plant metabolic functions, and the plant cannot complete its life cycle without the element. Usually, the plant exhibits a visual symptom indicating a deficiency in a specific nutrient, which normally can be corrected or prevented by supplying the nutrient. Terms commonly used to describe levels of nutrients in plants: calcium (new leaves are misshapen or stunted. Existing leaves remain green) iron (young leaves are yellow/ white green veins) nitrogen (upper leaves are light green, lower leaves are yellow. Bottom leaves are yellow and shriveled) potassium (yellowing tips and edges, especially in young leaves. Dead or yellow patches or spots develop on leaves) zinc (chlorosis between veins. Yellowing tips and margins. Spreeding gray-brown spots) manganese (yellow spots and/or elongated holes between veins) phosphorous (leaves are darker than normal. Less of leaves) magnesium (lower leaves turn yellow from edge inward. Veins remain green).

The faster the temperature drops, the lower the temperature, and the longer the temperature stays low, the greater the damage to plants.

Insects and mites can cause plant diseases or transport and inoculate viruses and microorganisms such as bacteria and fungi that cause plant disease. The direct damage to plants caused by insect feeding (herbivory) generally is considered in a separate category from plant disease.

ozone causes considerable damage to plants around the world, including agricultural productions and plants in natural ecosystems. Ozone damages plants by entering leaf openings called stomata and oxidizing (burning) plant tissue during respiration. This damages the plant leaves and causes reduced survival.

Excess fertilizer alters the soil by creating too high a salt concentration, and this can hurt beneficial soil microorganisms. Overfertilization can lead to sudden plant growth with an insufficient root system to supply adequate water and nutrients to the plant.

Herbicide damage is any adverse, undesired effect on a plant that is caused by exposure of that plant to a pesticide designed for weed control. Any plant can be subject to this problem.

These are just a few examples of the adverse effects of environmental conditions on plant growth.

Based on the above, we know that assessing plant physiology, which includes soil quality, water, temperature, nutrition, etc. is of great importance in understanding plant status and plant damage. Mastering the physical database of plants, especially agricultural and horticultural products, allows us to come up with more precise guidelines on how to improve the growth process and identify plant damage. There are several ways to do this. In the traditional way, quality control supervisors and specialist human resources with Observing the plant closely and according to their experience and expertise determine the status of the plant. Doing closely so through this method is very time-consuming, and there is a possibility of human error.

Another disadvantage is that it requires many specialists and botanists to decouple the vast expanses of farmland and gardens. Our purpose is to provide a way based upon the combined application of ICT solutions such as precision equipment, the Internet of Things (IoT), sensors and actuators, geo-positioning systems, Big Data, Unmanned Aerial Vehicles (UAVs, drones), robotics, etc. that assessment the physical properties of plants faster and more efficiently and Provide value-added to farmers in the form of better decision-making or more efficient operation and management.

The method designed to assess the physical indices of plants, especially agricultural and horticultural products, is based on machine learning, deep learning, and machine vision algorithms. In this way, with the development of technology and automation, the evaluation of plant physical indicators and agricultural processes, including more accurate estimation, plant efficiency, and cost reduction, will be improved.

Product evaluation can be divided into two types of field assessment (pre-harvest) and post-harvest evaluation.

As mentioned, evaluation in the field and in the field under cultivation can help in the detection of the disease, the distribution of infections by insects, etc. in the early stages of growth. Our goal in this approach is to optimize the appearance of the plant by eliminating manpower and to diagnose it in the event of any disease or problem. Remote sensing data provides more useful information than the physical indices of the plant because of its significant advantages over other compilation methods.

Remote sensing can be divided into three categories: aerial, satellite, and short-range sensing.

When evaluating the remote sensing platform, a spatial and spectral resolution should be considered. The benefits of remote sensing data in satellite imagery are wide coverage of the area, high-resolution aerial imagery, and the ability to collect data at arbitrary intervals, and in short-range sensing, it is possible to assess altitude.

Drones are usually lighter, less expensive, which are good for gathering information. There are currently two platforms for drones, namely "fixed-wing" and "rotary."

UAVs have many benefits. They can fly fast and frequently. In terms of flight altitude and time, missions are flexible and can receive high-resolution images. Winds, difficult navigation, data overlap, and flight time constraints on UAVs are limitations.

The most important applications of satellite data are the detection and differentiation of different plant species, the calculation of agricultural and horticultural products cultivation levels, the study of irrigated areas affected by water shortages or the attack of various pests. Other uses of such information are the preparation of comprehensive vegetation cover of each area, mapping of waterways and their relationship with susceptible areas, and estimation of agricultural and horticultural product yield.

These UAVs fly in the aisles of agricultural and horticultural products up to the height of the plant and receive different images of leaves, stems, and bodies of plants using machine-like visual techniques of the human eye.

images are transmitted to the operator, and the signals received by the cameras are processed by systems provided in the way that software solutions rely on machine vision, machine learning, deep learning, and image processing. Image processing is divided into two categories of machine vision and image processing.

Image processing is concerned with improving images, but machine vision involves ways of understanding images. It is commonly used in agriculture and horticulture in two ways. One is in processing high altitude images, and the other is near-ground images of plant height, which is our second approach, short-range imaging. Each machine vision system consists of two parts: hardware and software.

In agriculture and gardening, this system consists of five steps: imaging, initial processing to improve original images, segmenting images into separate areas, measuring the properties of plants, and classifying plants into different groups.

After initial processing, to improve the original images using machine learning techniques and deep learning models of 2D and 3D pixel matrices are created.

These matrices each contain data on the physical properties of plants, such as leaves, stems, and plant bodies.

This data is taught by machine learning algorithms. Machine learning is divided into two parts Supervised Learning and Unsupervised Learning. In supervised learning, algorithms are practiced in accordance with specific inputs, such as machine-defined inputs, and human-labeled outputs. But Unsupervised learning will play no role in labeling outputs and will allow the system to find commonalities between the data and its outputs.

The method developed for evaluating plant physics is based on observational learning, in which the computer or robot is introduced to examples of different inputs to the properties of plants and their appropriate outputs.

In other words, examples of plant-specific inputs and outputs suitable for them are first defined for the device.

The purpose of this machine learning approach is to make the computational algorithm able to gradually compare the outputs of the inputs to learn which approach and algorithm work best for a particular input, thereby enabling the system to detect and adjust the inputs to be more precise the model.

This will create a library containing multiple images of the plant's physical properties such as different leaf, stem, body and visual physiological modes, each of which has provided a concept for the machine that the machine can determine the plant's condition recognize. All of this data is categorized by machine learning and deep learning techniques, and the machine will program and relate to the data and outputs given to it, and analyze the other data that is given later after data analyzing the final status of the plant is determined.

This method helps us to identify the plant's physical properties indices in the least time and with high accuracy if there is a problem or disease in the plant.

This is a major development in the agricultural and horticultural industry, resulting in the optimal management of agricultural and horticultural products.

Advantageous effects invention

The method designed to assess the physical indices of plants, is based on machine learning, deep learning, and machine vision algorithms.

improved the evaluation of plant physical indicators and agricultural processes

improved including more accurate estimation, plant efficiency, and cost reduction

Product evaluation can be divided into two types of field assessment (pre-harvest) and post-harvest.

help to the detection of the disease, the distribution of infections by insects, etc. in the early stages of growth

optimize the appearance of the plant

Remote sensing data provides more useful information than the physical indices of the plant because of its significant advantages over other compilation methods.

Diagram block of an artificial-intelligence-based method of agricultural and horticultural plants' physical characteristics and health diagnosing and development assessment

in this patent These UAVs fly in the aisles of agricultural and horticultural products up to the height of the plant and receive different images of leaves, stems, and bodies of plants using machine-like visual techniques of the human eye. images are transmitted to the operator, and the signals received by the cameras are processed by systems provided in the way that software solutions rely on machine vision, machine learning, deep learning, and image processing.

Image processing is divided into two categories of machine vision and image processing. In agriculture and gardening, this system consists of five steps: imaging, initial processing to improve original images, segmenting images into separate areas, measuring the properties of plants, and classifying plants into different groups. After initial processing, to improve the original images using machine learning techniques and deep learning models of 2D and 3D pixel matrices are created.These matrices each contain data on the physical properties of plants, such as leaves, stems, and plant bodies and helps us to identify the plant's physical properties indices in the least time and with high accuracy if there is a problem or disease in the plant.

Examples

A method of the plant's health diagnosing having a plant assessing physical characteristics parameters using an aerial and terrestrial plant health diagnosing system, the aerial and terrestrial plant health diagnosing comprising an unmanned vehicle, capable of being controlled by embedded smart algorithms including machine learning and machine vision with pre-defined maps or real-time pathfinder, a gimbal(s), attached to the unmanned vehicle and a camera(s) attached to the gimbal(s), gimbals attached to the robotic arms or vehicle, the method comprising:

Flying or moving the unmanned vehicle in the agricultural and horticultural products corridors,

Positioning the unmanned vehicle (with or without robotic arms) in the height of plants to capture whole or part of plant body;

Capturing images via the cameras, the images of whole or part of plant body agricultural and horticultural plants, and processing the images to diagnosing plant physical characteristics parameters.

This method can be used to control gardens and agricultural fields. and for developing plant physical characteristics data by imaging methods for specific agricultural and horticultural products to be used in conjunction with any machine learning, machine vision, and deep learning methods (all of the deep learning algorithms) to diagnose plants physical characteristics status.

Claims

what is claimed that a method of developing plant's physical characteristics data collecting by imaging methods for specific agricultural and horticultural products to be utilized in conjunction with any using machine learning, machine vision, and deep learning methods (all of the deep learning algorithms) to diagnose plants physical characteristics status.
According to claim 1, a method of generating pixel matrixes from multiple images collected from within the plant form specific agricultural and horticultural products and extraction of particular plant physical characteristics and extracting image features to diagnosing plant's health and development accuracy using machine learning, machine vision, and deep learning techniques.
According to claim 2, a method of generating pixel matrixes from multiple images collected from within the plant physical characteristics including generate of models from the 2D and 3D pixel matrixes, extraction of specific plant physical details, and convert surface features from images to pixel raw data to extract and clarify image features to diagnosing plant's health and development accuracy using machine learning, machine vision, and deep learning techniques.
According to claim 3, a method of assessing physical characteristics parameters using a learning algorithm capable of identifying plant's physical characteristics status, including generating an image library comprising a plurality of images including images of plant leaves, stems, and the entire plant. the different plant parts images classified in the images according to diagnosing poor plant health status; training the learning algorithm using the sample image library by applying an up sampling to provide approximately the same number of images of each part type and health status in the image library; extracting relevant features from an image by machine vision, machine learning, and deep learning methods.
According to claim 4, further including gathering images or any other of assessing physical characteristics parameters data of plants at a discrete sample farm or garden, generating a pixel models of the images of plants at the discrete sampling; extracting image future by deep learning methods; and matching features of the samples with images in an image library to diagnosing plant's health and development accuracy at the discrete sampling site.
According to claim 5, wherein the pixel matrix in the image is defined by deep learning techniques to pre-learned models of pixel values and the image library comprises images of agricultural and horticultural product's physical characteristics wherein the problems are any physical problems.
According to claim 6, wherein the diseases include any disease identifiable by visual inspection of plant leaves, stems, and entire plant or any physical changes and wherein the pixel model results from multi images from a plurality of cameras received from an image capture device within the agricultural and horticultural products.
An agricultural and horticultural plant's health diagnosing system image capturing and data gathering tools and ways for use with the method according to any of the claims above, comprising: a motorized vehicle including cameras attached (cameras attached to any robots, cameras attached any drones, cameras attached any flying gadgets, etc.), and any storage device for storing images captured by the cameras for later processing and including any network communication platform for transferring images to a remote location.
according to claim 8, An agricultural and horticultural plant's health diagnosing system wherein the unmanned vehicle is a land rover robot or is a rotary-wing drone or is an unmanned aerial vehicle.
According to claim 9, method of the plant's health diagnosing having a plant assessing physical characteristics parameters using an aerial and terrestrial plant health diagnosing system, the aerial and terrestrial plant health diagnosing comprising an unmanned vehicle, capable of being controlled by embedded smart algorithms including machine learning and machine vision with pre-defined maps or real-time pathfinder, a gimbal(s), attached to the unmanned vehicle and a camera(s) attached to the gimbal(s), gimbals attached to the robotic arms or vehicle, the method comprising:
Flying or moving the unmanned vehicle in the agricultural and horticultural products corridors,

Positioning the unmanned vehicle (with or without robotic arms) in the height of plants to capture whole or part of plant body;

Capturing images via the cameras, the images of whole or part of plant body agricultural and horticultural plants, and processing the images to diagnosing plant physical characteristics parameters.