RO138315A0 - Early identification of grape wine leaf integrity by using spectral sensors - Google Patents

Abstract

The invention relates to a system and a method for the early identification of grape vine diseases by using spectral analysis in the visible and near infrared range and the Internet of Things technology. According to the invention, the system comprises: pairs of spectral photoelectric sensors (1, 3), one of which is a calibration sensor and the other one is a measure sensor, mounted on a system of height-adjustable rods (2, 6) and a sensor of photosynthetically active radiation PAR with a reference role within the measurements, where the sensors have a capacity of remote data transmission to a node (10) which, in its turn, transmits them, through radio waves, to a gateway device (12), connected to a server (13), and from the server the data are transmitted into the cloud (14) where they are processed by algorithms based on artificial intelligence. The claimed method consists in identifying the healthy grape vine leaves and the sick ones by using the ratios between the values read by the measure sensors and the calibration sensors for the same wavelength, for instance for green and red colours, while having as reference initial processed values for healthy leaves.

Description

Early identification of grapevine leaf integrity using spectral sensors

b. Specification of the technical field of application of the invention

The invention presents a method for the early identification of vine diseases in order to prevent and combat them using spectral analysis in the visible and near infrared (VISNIR) and the Internet of Things (loT) technology. It is found at the confluence of the fields of agricultural engineering, electronics and computer science.

c. Specifying the known state of the art

Typical vine diseases can be devastating when the degree of infection is high and fungicide treatments are not carried out in time. The occurrence of plant diseases depends on specific environmental factors. Thus, sensor-based detection techniques are useful in identifying disease outbreaks and disease severities. in combination with advanced data analysis methods, these techniques are found in sustainable agricultural production, as they facilitate the estimation of the incidence and severity of diseases and their negative effects on the quantity and quality of agricultural production, leading to the reduction of expenses and the ecological impact caused by the use of pesticides . The unwanted spots that develop on the leaves are specific to each type of disease.

in [1] the system called "guardian of weeds" is presented which uses a multispectral sensor AS7265x, mounted under the agricultural machine with the role of identifying. This system could trigger an identified weed removal system such as a sprayer or mechanical tillage. Weeds are identified by evaluating the Normalized Differentiation of Vegetation Index (NDVI), Enhanced Normalized Differentiation of Vegetation Index (ENDVI) and Enhanced Vegetation Index (EVI), comparing the result to a calibrated threshold that will indicate whether there is a weed under the sensor . The "weed guard" system uses ENDVI, the most relevant system in detection, having the ability to discriminate 7.6 x 7.6 cm vegetation samples from bare soil at sensor heights of 30 and 41 cm from the ground. The system is an alternative to robotic field weeders that could help reduce costs and improve environmental outcomes in agriculture. The disadvantage of this system is the need to save the data locally, on a _microSD card, so the data cannot be processed in real time remotely using complex algorithms, the consequence of which is that weeds can be confused with cultivated plants.

RO 138315 AO

Also, the system can only be used for certain types of plants because it detects vegetation with maximum heights of 41 cm and 30 cm, respectively.

[W02007/041755] describes methods for evaluating sample characteristics from NIR or VIS-NIR spectroscopic images. By processing the samples, the presence of infestation agents or microorganisms such as fungi or other matter is identified. The invention refers to measurement methods carried out in crop transport vehicles, using different types of lighting systems such as xenon, mercury or halogen lamps or using natural light. The purpose is to detect the state of the fruit for the application of certain treatments before transport. Also used are cameras that include the last part of the VIS-NIR spectrum 3502500nm and other types of image acquisition systems. Samples representing plant reflectivity are processed using statistical algorithms and machine learning. Different crop diseases are identified on leaves, fruit, trunk, petals, etc.

The present study has the disadvantage of continuous white light illumination of the target, and the processing is also done in the laboratory using chemometric analyses. Another disadvantage is that the identification of diseases, microorganisms, etc. it is done on the harvest, so the problems that may arise before the harvest cannot be prevented.

the patent [CN115511838A] presents an automatic software method for plant disease identification using leaf images processed with artificial intelligence algorithms. The steps for image processing are:

Preprocessing; 2) Defining and extracting features; 3) Screening of the most valuable feature based on a feature selection method through an algorithm inspired by the displacement of groups of ascidians; 4) Classification and identification of the plant disease image. The algorithm is based on converting a leaf image into 5 color spaces, followed by extracting color and texture features from these spaces. The chromatic characteristics are expressed by the chromatic moments of order 1, 2 and 3. The disadvantages of this invention are the lack of interpretability of the depth characteristics, the number of samples for training is insufficient so that the optimization of the classifier parameters becomes very difficult and the clarity of the collected images is not always good .

The patent [US 8,249,308 B2] presents a chlorophyll fluorescence imaging and computer processing system that measures, detects and quantifies plants under environmental and pathogen stress and uses transmission spectroscopy to obtain spectral signatures (data) of plants . Based on an expert system, the system compares images and data to a plant disease library to instantly diagnose plant stress and disease in the field.

RO 138315 AO

A portable fluorescence spectroscopic imaging system for diagnosing plant health captures the information. The light source is provided by an LED array controlled to turn certain LEDs on or off depending on the desired spectral wavelengths. Thus the primary LED array controllably emits light of wavelength in the desired range, and a digital imaging device captures both spatially and temporally a fluorescent image comprising chlorophyll fluorescence emitted by the plant due to the light emitted by the LED array . A leaf holder is located near the exit of the focus cone to maintain consistent positioning and distance between the digital imaging device, the LED light source, and the leaf. A secondary light source is used to provide broadband light through the leaf, a lens is used to focus the image of light emitted from the secondary light source and one or more memory devices that store data onto the device. A data library contains plant fluorescence intensity data indicating both healthy plants and stressed or diseased plants and data on specific plant conditions. The disadvantage is the use of many light sources, the device can be installed in one place, the data is not transmitted remotely to the cloud.

utility model [CN204389379U] presents a chlorophyll content measuring device for diseased crop leaves, which adopts surface measurement instead of point measurement. This device reduces the problem of low accuracy caused by point measurement and reducing measurement point selection by the operator. The device has the advantages of portability, non-contact and non-destructive operation. The chlorophyll content measuring device consists of a measuring probe, a spectrum signal processing device and an adjustable rod; a measuring probe comprising a laser, a spectral receiver and a blade carrier, a clamp between the laser and the blade holder, the blade holder for positioning and fixing the entire blade of the test crop. The light emitted by the laser illuminates the entire surface of the test crop blade, the spectral receiver detects a spectral signal of the entire test crop blade and transmits the spectral signal to the spectral signal processing device. The spectral signal processing device is additionally equipped with a GPS module. The GPS module is used for real-time positioning of the current test location. A liquid crystal display module is added to the spectral signal processing device. Output data include: relative chlorophyll value, latitude and longitude, division result of disease degree, test location and other information common to the whole leaf. The laser is composed of an LED excitation light source, and the emission spectrum of the LED has two bands, at 650 nm and 940 nm.

RO 138315 AO

The LED laser emits two types of light, one red (maximum wavelength 650 nm) and the other infrared (940 nm). Two types of light enter the blade and are received by the spectral sensor. Preferably, the apparatus additionally includes a power module for powering other modules in the device. The disadvantage of the system is that the data is processed locally, there is no alarm in case of detection of infection on the leaves by changing the chlorophyll content. Only the hardware part of the system is presented, the data processing algorithms are not described.

d. The technical problem that the invention solves

Spectral photoelectric sensors evaluate the local reflectivity properties of plants in different regions of the electromagnetic spectrum and are capable of obtaining information beyond the visible range. Thus, it is possible to detect early changes in plant physiology, such as changes in tissue color, leaf shape, and variations in interaction with solar radiation.

The novelty introduced by the present invention compared to the others described above is the fact that spectral measurements can be made throughout the day without the need for additional illumination. Continuous measurement is possible using a standard sensor throughout the measurement period.

The technical problem that the invention solves is to identify the early appearance of vine diseases in order to combat them in the early stage, before they spread to the neighboring plants and to the level of the entire plant. The amount of chemicals used is thus reduced to a minimum. The degree of drought can also be identified from the color of the leaf in order to perform drip irrigation, thus reducing water consumption.

The data collected from the spectral photoelectric sensors is transmitted remotely to a cloud server where it is correlated and analyzed using intelligent (adaptive) algorithms to identify leaf spots. An alarm can be set on the result display device if the processed values are not within normal limits.

For example Plasmopara viticola settles in 6-10 hours, during this interval the mycelium and spores are formed, if there are favorable atmospheric conditions, namely atmospheric humidity in the range [95, 100]% and temperature [18, 24]°C. Spores are projected from the ground onto the underside of leaves, producing infesting filaments that penetrate host plant tissues through stomata. After the initial appearance of the infestation, there is a period of feeding of the fungus on the leaves, which is observed by the appearance of yellow oily spots.

RO 138315 AO

QD? ^b

e. Disclosure of the invention

The present invention presents a system consisting of pairs of spectral photoelectric sensors, one as a reference (standard - SE) and one that is placed on the leaf, called the measurement sensor (SM). The sensors transmit the data remotely to be processed. An example of a spectral photoelectric sensor is the AS7262. It has a low price and easily connects to Arduino acquisition boards on the I2C port. The measurements of the two sensors are carried out under the same conditions of temperature, humidity and brightness.

The reason for creating such a pair is that the leaves are semi-transparent and the measurements vary with the ambient light, so will refer the measurements to a reference. SE will have a semi-transparent white surface positioned at the same distance that the leaf is placed from its mate. The spectral measuring sensors are fixed on the upper face of the leaf at a maximum distance of 5 cm from it, as in Fig. 1. The measuring distance is given by the height of the window (4). A network of such pairs of sensors placed on several leaves at different heights from the ground will be realized in the monitored vineyard. The network starts the measurements when the leaves are young with a diameter of 2 cm and ends its activity after the grapes are picked. Thus, if there are areas where spores are identified after the fruit has been picked, intervention can be done to remove them. In autumn, after the last secondary contamination, the fungus forms resistant spores, in the form of which the fungus overwinters. A photosynthetic active radiation sensor PAR (S2-442) will also be connected to the same network, which will have a reference role in the measurements, as will be described in the algorithm presented below.

The sensors used as a reference (3) will have a white plexiglass surface (5) placed next to the hole and fixed next to the window (4). The pair of sensors, the reference and the measuring one, will be rigidly mounted on an adjustable rod system described in Fig. 1. The rod (2) will have the same size of maximum 15 cm for the sensors to measure in the same light conditions. The stem (6) will be adjustable to take measurements on leaves at different distances from the ground (7). The measurement sensor (1) will be kept at a distance of approximately 5 cm from the leaf (8) and oriented towards its center. The leaf will be fixed at the same distance from the SM as the Pexiglas surface from the SE. The PAR sensor is placed at a height of 2 m above the ground as recommended in its catalog tab.

The data from the pairs of sensors (9) are collected by means of a device called a node (10), then sent by radio waves with the help of an antenna (11) to a device called a gateway (12), which is connected by cable (16) to a server called FOG (13). From the server they are transmitted via the Internet, via cable (15) to the cloud (14) (Fig. 2). The devices are supplied with electricity using a controlled system based on photovoltaic panels.

RO 138315 AO

Algorithms based on artificial intelligence are used that personalize and automate the system by analyzing the data from the sensors and sounding the alarm if color differences are detected. All the algorithm is processed on the server (14). The software adapts to the measurement conditions depending on how the spectral sensor system is built (9). The computer (14) can be accessed via the Internet remotely. The results obtained from the sensors are compared with environmental reports of temperature and humidity to verify that the system is working correctly.

Each spectral sensor will be inserted in a black plastic box (18) with a 1 cm diameter hole and a maximum height of 5 cm (17) (Fig. 3). The hole will be placed next to the LED and the sensor itself. The LED emits white light with a color temperature of 5700K (it is a Luxeon 3014 LED model L130-5780001400001). The sensor will be inserted in the box with a cover being protected from water (18 and 19). The box has a hole for the power and measurement cable (20). The measuring objects are fixed on the window provided for the measurement (17). The measurements will be made in the visible spectrum at the wavelengths [500, 600]nm and [600, 700]nm which represent the colors green and red. It is known from the literature that when the leaf changes its color the effect is maximum in the spectral range corresponding to the red color. Measurements reach maximum accuracy in dark conditions. The sensors can take measurements within the measurement range indicated in the purchase code. The measurement range can be changed remotely.

The sensor network can include a number of 1 to 1024 sensors, as much as the 12 C serial port allows.

Data processing algorithm.

1. The data is recorded in the cloud as described above (Fig.2).

2. Data from the PAR sensor will be compared to a threshold value to identify the times of the day when it is dark. If the reading is less than 1 pmol m ^-2 s ^-1 it is considered dark.

3. Only the red and green spectra of both sensors (SE and SM) will be selected from the sensor data. To remove the noise, an average of the values recorded by three successive measurements will be calculated.

4. Data cleaning procedure: Given that the sensor pairs are placed at different heights and locations, the amount of sunlight incident on the leaves will be different. Some sensors will be shaded by the leaves or stem, others, which are placed on the leaves at the top of the hub, will have the same lighting conditions as the PAR sensor.

RO 138315 AO

a. It has been observed from experiments that the values from the PAR sensor and those from the reference (individual) sensor are correlated. Thus using a linear regression algorithm the correlation function can be identified for each SE for the wavelength [500,600]nm. So there will be a learning period that will use the data collected over a period of 48 or 72 hours.

b. Once the correlation function is known, a prediction will be made of the reference sensor data (SE) according to the value coming from the PAR sensor. The values read from the SE will be compared with the prediction result. If the error between them is greater than one unit, it means that the read data is erroneous and the current line is deleted from the measurement table.

5. Calibration procedure: The sensors will be calibrated in the laboratory so that the prediction system can be prepared. For this, young, mature leaves with spots and dry ones will be used to form a database. Tests can be done with a single pair of spectral sensors and a PAR sensor. Measurements will be made in dark conditions and at various ambient light intensities identified by the values read by the PAR sensor. the data preprocessing part will calculate the ratio between the value read from SM and the value from SE for the same wavelength, for example for green (Rv) and red (Rr) colors.

following the experiments, it was observed that we can identify, using the results of the calculated ratios, a class of healthy leaves and a class of diseased leaves with reference to the initial values processed from the healthy leaves (evaluation index). in this way a signature of the field leaf will be obtained. The inputs to the classification system are Rv and Rr. The measured data from the sensors installed in the vineyard, after preprocessing, can be classified using one of the known classification systems such as K-nearest neighbor. It will be taken into account that these values depend on the box in which the sensors are inserted, the distance of the sensor from the tested leaf, the diameter of the hole through which the measurement is made, the texture of the box and the white surface used as a standard.

6. The system mounted in the vine will have several pairs of sensors placed at different places and at different heights. Data from the sensors will be collected with acquisition boards that will transmit it to the cloud. They will be cleaned and processed in parallel. If the results of at least 2 pairs of sensors will not be within the threshold values of healthy leaves after 9 measurements, then the alarm will be sent.

RO 138315 AO

The alarm will indicate that there is a problem in the field which can be analyzed by observing measurement thresholds from environmental monitoring sensors or locally by on-site observation by a plant health specialist.

f. Presentation of the advantages of the invention in relation to the state of the art

The proposed invention brings a novelty through the local and continuous monitoring of the spectral reflectivity of the leaves, the correlation of data from the network of spectral photoelectric sensors placed on several leaves to signal the appearance of spots on them. It does not require testing the system on several types of vine leaves, the system adapts to the environment where it is placed by comparing the values initially collected from the leaf on which it was installed. The advantages of the invention are:

The system described in this patent is a non-invasive method of monitoring plant leaves;

The use of low-cost sensors that can be deployed throughout the vineyard;

Processing data from remote sensors in real time and alerting the farmer in case of identification of leaf color changes;

- Use of a reference sensor for continuous calibration of the sensor that measures spectral samples as a function of light;

The sensor assembly reduces or eliminates the influence during measurement of light variations due to the sun, meteorological phenomena and different positioning of the sensors on the plant.

Continuous cleaning of the received data according to the ambient light intensity;

The system can be adapted to any type of cultivated plants.

RO 138315 AO

Relevant document

[1] Duncan L., Miller B., Shaw C., Graebner R., Moretti M.L. Walter C., Selker J., Udell C., Weed Warden: A low-cost weed detection device implemented with spectral triad sensor for agricultural applications, HardwareX 11 (2022), 2468-0672

[WO 2007/041755] Hyperspectral Imaging Contaminants in Products and Processes of agriculture PCT/AU2006/000999, April 19, 2007

[CN 115511838A] Plant disease high-precision Identification method based on group intelligent optimization

[US 8,249,308 B2] Portable intelligent fluorescence and transmittance imaging spectroscopy system, Aug. 21, 2012

[CN108445000A] Portable rapeseed leaf disease detecting device equipped with a multi-spectral camera, 2018-08-24

[CN204389379U] Crop disease leaf chlorophyll content determination device, 2015-06-10

Claims (5)

1. The system is characterized by a pair of identical spectral photoelectric sensors used to determine leaf colors, one being the measurement sensor (1) placed directly on the analyzed leaf (8), the second being the reference or standard sensor (3 ) that measures waves on a white plexiglass (semi-transparent) surface (5). 2. The pair of sensors of claim 1 are mounted on a system of rods (2) and (6) adjustable in height (6). 3. The values from the green spectral component of the reference sensor are correlated with the values of a light intensity identification sensor using a linear regression for prediction, the predicted spectrum result is compared with the measured one so that if the error is greater than one unit, all the measured values from the pair of sensors. 4. Calculate the ratio of the values from the green and red spectral components measured with the sensor pair of claim 1 to eliminate the influence of ambient light during the measurements. 5. Placing an array of sensor pairs as set forth in claim 1 at multiple locations and at various heights in the vineyard.