WO2023233563A1 - Sensor system to be mounted on flying body, and data-processing system for sensor system - Google Patents

Abstract

A sensor system according to the present disclosure is to be mounted on a flying body, and comprises: an optical measurement sensor unit (1) that obtains an interferogram of an observed object; an object determination unit (5b) that includes trained artificial intelligence, and determines the observed object on the basis of the interferogram; and a data reduction unit (6) that reduces data of the interferogram on the basis of the determination result of the object determination unit (5b).

Description

Sensor systems mounted on flying objects and data processing systems for sensor systems

The disclosed technology relates to a sensor system mounted on a flying object and a data processing system for the sensor system.

Technology is known in which a sensor system such as an imaging device is mounted on a flying object such as a drone to observe an observation target on the ground. A multispectrum camera is also known as such an imaging device.

For example, Patent Document 1 discloses that a drone is equipped with an imaging device to sense the state of vegetation in a field. Specifically, Patent Document 1 relates to a technique for easily selecting a plurality of images and efficiently generating a mapping image based on the images selected to be used.

JP2020-21437A

When conducting observations using a sensor system mounted on a flying object, there are situations in which it is desirable to transfer data to the ground and process the information there. In particular, when the flying object is an unmanned drone and observations are to be made in real time, a huge amount of information must be transferred to the ground. Data reduction can be considered to reduce the amount of data transferred to the ground. Mechanical data reduction, such as coarser sampling or coarser quantization by lowering the number of bits, has the problem of reducing even useful information that is originally necessary for observation.

The disclosed technology aims to solve the above problems and provide a sensor system that reduces data while retaining useful information originally required for observation.

The sensor system according to the disclosed technology includes an optical measurement sensor unit that is mounted on a flying object and acquires an interferogram of an observation target, an object determination unit that determines an observation target based on the interferogram, and an object determination unit that determines an observation target based on the interferogram. and a data reduction unit that performs data reduction on the interferogram based on the determination result.

Since the sensor system according to the disclosed technology has the above configuration, it is possible to reduce data while leaving useful information originally required for observation.

FIG. 1 is a block diagram showing the functional configurations of a sensor system and a data processing system according to the first embodiment. FIG. 2 is a diagram illustrating the operation or processing of the sensor system according to the first embodiment. FIG. 3 is a diagram illustrating the operation or processing of the data creation unit 7, memory unit 8, and data processing system of the sensor system according to the first embodiment.

Embodiment 1.
FIG. 1 is a block diagram showing the functional configurations of a sensor system and a data processing system according to the first embodiment. In this specification, the sensor system and data processing system according to the disclosed technology will be collectively referred to as a signal processing system.
The sensor system according to the disclosed technology is mounted on a flying object such as a drone, an aircraft, or an artificial satellite, and is intended to observe the ground. A flying object equipped with a sensor system may have the purpose of observing the ground as an observation target from a plurality of observation directions and obtaining observation results using two-dimensional images.
Furthermore, the data processing system according to the disclosed technology is not mounted on a flying object, but is installed in a data center on the ground, and receives and processes data transmitted from a sensor system mounted on a flying object. It is assumed that
As shown in FIG. 1, the sensor system includes an optical measurement sensor section 1, an AD conversion section 2, an interference image acquisition section 3, an interferogram storage section 4, a signal analysis section 5, a data reduction section 6, It includes a data creation section 7 and a memory section 8. Each component in the sensor system is connected as shown in FIG.
As shown in FIG. 1, the data processing system includes an interferogram holding section 9, a spectrum restoration section 10, an imaging processing section 11, a label creation section 12, a feature extraction section 13, and a teaching data creation section 14. , a machine learning section 15 , and a main analysis section 16 . The interferogram holding unit 9, spectrum restoration unit 10, imaging processing unit 11, label creation unit 12, feature extraction unit 13, teacher data creation unit 14, and machine learning unit 15 are functional blocks related to pre-learning processing. group (hereinafter referred to as "pre-learning processing section"). Each component in the data processing system is connected as shown in FIG.

<<Optical measurement sensor section 1 in sensor system>>
The optical measurement sensor section 1 is a component for acquiring an interferogram. Specifically, the optical measurement sensor section 1 is a Fourier spectroscopy type optical interferometer. An interferogram is an interference wave obtained from an interferometer. The waveform of the interferogram has, for example, the appearance shown above the text "interferogram acquisition" in FIG. 2, which will be described later.

《AD conversion unit 2 in sensor system》
The AD conversion section 2 is a component for converting an analog signal sent from the optical measurement sensor section 1 into a digital signal.

<<Interference image acquisition unit 3 in sensor system>>
The interference image acquisition section 3 is a component for generating and acquiring an interference image from the digital signal sent from the AD conversion section 2. The term "interference image" used in this specification is an image in which the absolute value of the difference between the maximum and minimum amplitudes of an interferogram is expressed as a contrast for each observation point. Simply put, an interference image is an image representing interference fringes. The interference image has, for example, the appearance shown above the words "interference pattern image acquisition" in FIG. 2, which will be described later.

《Interferogram storage section 4 in sensor system》
The interferogram storage unit 4 stores interferogram digital data (hereinafter referred to as “interferogram data”) for pixels corresponding to a certain observation point from the interference image sent from the interference image acquisition unit 3. It is a component for

《Signal analysis section 5 in sensor system》
The signal analysis unit 5 is a component for performing signal analysis on the interferogram data sent from the interferogram storage unit 4. As shown in FIG. 1, the signal analysis section 5 includes a spectrum analysis section 5a, an object determination section 5b, and an importance analysis section 5c. The spectrum analysis section 5a, the object determination section 5b, and the importance analysis section 5c are connected as shown in FIG.

<<Spectrum analysis section 5a in signal analysis section 5>>
The spectrum analysis unit 5a is a component for performing spectrum analysis on the interferogram data sent from the interferogram storage unit 4. Spectral analysis of interferogram data yields the optical spectrum of the optical signal. Specifically, the optical spectrum is obtained by Fourier transforming the interferogram. The optical spectrum is represented by a complex number for each frequency.
The interferogram data and optical spectrum data are sent by the spectrum analysis section 5a to the object determination section 5b.

<<Object determination unit 5b in signal analysis unit 5>>
The object determination unit 5b is a component for determining an object from an interference image based on interferogram data, optical spectrum data, or both.
The object of determination may be one-dimensional data corresponding to individual pixels on the interference image (i.e., interferogram data), or three-dimensional data (i.e., spectral image data) corresponding to a specific region of the interference image. There may be. Moreover, the object of determination may be both of these.
The target determination performed by the target determining unit 5b may be, for example, determining the presence of clouds in the observation direction of each pixel of the interference image. Clouds pose a nuisance to sensor systems that observe the ground.
The target determination performed by the target determination unit 5b is realized by having trained artificial intelligence. Artificial intelligence is realized by a mathematical model called a learning model. The learning model is, for example, an artificial neural network. Artificial intelligence learning is performed in a machine learning unit 15 of the data processing system, which will be described later. The broken line arrow extending from the machine learning section 15 in FIG. 1 represents that the object determination section 5b is equipped with artificial intelligence that has been trained in the machine learning section 15. When the object determination unit 5b performs determination for each pixel, artificial intelligence may be realized using a semantic segmentation algorithm.

<<Importance analysis unit 5c in the signal analysis unit 5>>
The importance analysis section 5c is a component for analyzing the importance of each observation point based on the determination result sent from the object determination section 5b.
The definition of importance analyzed by the importance analysis unit 5c may be set as appropriate depending on the purpose of use of the sensor system and data processing system.
The importance analysis result obtained by the importance analysis section 5c is sent to the data reduction section 6.

《Data reduction part 6 in sensor system》
The data reduction unit 6 is a component for reducing data in locations determined to be unnecessary from the target determination and importance analysis information sent from the target determination unit 5b and importance analysis unit 5c. Specifically, the data reduction section 6 includes at least one component among an area reduction section 6a, a band reduction section 6b, and a bit reduction section 6c.
The criteria for determining that the data reduction unit 6 is unnecessary may be determined as appropriate depending on the purpose of use of the sensor system and data processing system.
The data reduced by the data reduction unit 6 is sent to the data creation unit 7.

<<Area reduction unit 6a in data reduction unit 6>>
When the data reduction unit 6 includes an area reduction unit 6a, the area reduction unit 6a selects observation areas of low importance from the target determination and importance analysis information sent from the target determination unit 5b and importance analysis unit 5c. In contrast, data reduction is performed.
As mentioned above, clouds are a nuisance for sensor systems that observe the ground. When clouds exist between the sensor system and the observation point, scattered light generated by sunlight reflecting off the clouds becomes an interference signal for the sensor system. The observation area in which the area reduction unit 6a performs data reduction corresponds to an area where clouds exist between the sensor system and the observation area.
It is known that the light spectrum of scattered light generated by reflection from clouds is white. Therefore, if the light spectrum obtained by the spectrum analysis unit 5a has many white components, it can be determined that clouds are occurring between the sensor system and the observation point observed by the sensor system.
Data reduction for areas performed by the area reduction unit 6a is realized by providing trained artificial intelligence. Artificial intelligence learning is performed in a machine learning unit 15 of the data processing system, which will be described later. The broken line arrow extending from the machine learning unit 15 in FIG. 1 represents that the area reduction unit 6a is equipped with artificial intelligence that has been trained in the machine learning unit 15.

<<Band reduction unit 6b in data reduction unit 6>>
When the data reduction unit 6 includes a band reduction unit 6b, the band reduction unit 6b can estimate that the degree of importance is low based on the target determination and importance analysis information sent from the target determination unit 5b and the importance analysis unit 5c. This is a component for deleting frequency band data. The term "band" used in the name of the band reduction unit 6b is derived from the English word "Frequency Band" which represents a frequency band.
Specifically, the band reduction unit 6b is a component intended for the case where the observation target of the sensor system is a specific gas existing on the earth's surface. Gases are gaseous atoms, and absorption lines with wavelengths unique to the atoms appear in a continuous spectrum. Therefore, if the characteristics of absorption lines do not appear in the optical spectrum obtained by the spectrum analysis unit 5a, it can be assumed that the specific gas does not exist in the environment of the observation point and can be considered as a candidate for data reduction.

<<Bit reduction unit 6c in data reduction unit 6>>
When the data reduction unit 6 includes a bit reduction unit 6c, the bit reduction unit 6c can estimate that the degree of importance is low based on the target determination and importance analysis information sent from the target determination unit 5b and the importance analysis unit 5c. , is a component for deleting data in the time domain or frequency domain of an interferogram in units of bits or by reducing the number of bits. In this specification, deleting data in units of bits is expressed as "bit reduction."
The bit reduction unit 6c may, for example, reduce bits of data in a wavelength domain where the signal level is excessively high in frequency domain data of the interferogram, that is, optical spectrum data. Further, the bit reduction unit 6c may, for example, reduce bits of data in a time period that does not contribute to measurement accuracy in the time domain data of the interferogram. In the time domain data of the interferogram, the time periods that do not contribute to measurement accuracy are generally the time periods at both ends, between the start and end times (see the interferogram waveform shown in Figure 2 below). ).
The details of bit reduction will become clear from the explanation along with FIG. 2, which will be described later.

《Data creation section 7 in sensor system》
The data creation unit 7 is a component for reconstructing three-dimensional data from the data sent from the data reduction unit 6. The three-dimensional data reconstructed in the data creation unit 7 may be an interferogram for each pixel or an interference image for each sampling period. For example, when an optical spectrum, that is, frequency domain information is sent from the data reduction unit 6 as data reduced, the data creation unit 7 performs an inverse Fourier transform on the frequency domain information and creates an interferogram for each pixel. May be reconfigured.
The three-dimensional data reconstructed by the data creation section 7 is sent to the memory section 8.

FIG. 1 shows that, in addition to the route from the data reduction unit 6, there is a route from the interferogram storage unit 4 as a route for the data creation unit 7 to acquire information. The data creation unit 7 may perform a process of sending the raw interferogram data sent from the interferogram storage unit 4 to the memory unit 8. The raw interferogram data can be used as training data for learning performed by the machine learning unit 15, which will be described later. Note that the raw interferogram data to be used as teacher data may be transmitted intermittently depending on the required amount, that is, depending on the purpose of use of the sensor system and the data processing system.

《Memory part 8 in sensor system》
The memory unit 8 is a component for storing the three-dimensional data sent from the data creation unit 7. The three-dimensional data stored in the memory section 8 is sent to a data processing system.
The memory unit 8 also stores raw interferogram data sent from the data creation unit 7 as appropriate. The raw interferogram data stored in memory portion 8 is also sent to the data processing system.

<<Interferogram holding unit 9 in data processing system>>
The interferogram holding unit 9 is a component for temporarily holding interferogram data. The interferogram data held by the interferogram holding unit 9 can be classified into two types, focusing on their roles.
One type is used in the learning phase of artificial intelligence, and is interferogram data as training data of a learning data set. The interferogram data as the teacher data may be, for example, actual observation data actually observed by a sensor system in the past. If sufficient past actual observation data by the sensor system cannot be obtained, teacher data may be created using actual observation data observed by another system or by performing data expansion. A part of the learning data set may be used as a verification data set for verification performed in the final stage of learning.
The other type is interferogram data, which is used in the inference phase of artificial intelligence and is observation data of the sensor system transmitted from the sensor system.
The interferogram data used in the learning phase is sent to the spectrum restoration section 10 and feature amount extraction section 13 of the pre-learning processing section. Interferogram data used in the inference phase is sent to the main analysis section 16.

<<Spectral restoration unit 10 in data processing system>>
The spectrum restoration section 10 is a component for performing spectrum restoration on the interferogram data sent from the interferogram holding section 9. Spectral restoration refers to the problem or process of estimating and restoring the signal spectrum that a signal originally has. The waveform of the signal spectrum obtained by spectrum restoration has, for example, the appearance shown on the right side of the words "spectrum restoration" in FIG. 3, which will be described later. As a technique applying spectrum restoration, for example, a noise suppression method such as the MMSE-STSA (Minimum Mean-Square-Error Short-Time Spectral Amplitude Estimator) method is known.

<<Imaging processing unit 11 in the data processing system>>
The imaging processing unit 11 is a component for generating image data based on the signal spectrum sent from the spectrum restoration unit 10. The image data generated by the imaging processing unit 11 will be referred to as "spectral image data" in this specification. The spectral image data can be expressed, for example, in the appearance shown above the characters "imaging processing" in FIG. 3, which will be described later. Spectral image data may be thought of as a data cube of three or more dimensions.
When generating image data, the imaging processing unit 11 may correct the data by processing such as interpolation for locations where data has been reduced (see FIG. 3, which will be described later).
The spectral image data generated by the imaging processing section 11 is sent to the label creation section 12.

《Label creation unit 12 in data processing system》
The label creation unit 12 is a component for supporting the user of the sensor system and the data processing system to create correct labels constituting the learning data set. The label creation unit 12 uses, for example, artificial intelligence that is in the process of learning to display correct label candidates on the display screen, overlapping them with the spectral image data. The user may create a correct label by modifying the correct label candidates displayed by the label creation unit 12 as necessary.

<<Feature extraction unit 13, teacher data creation unit 14, and machine learning unit 15 in the data processing system>>
The feature extraction unit 13, the teacher data creation unit 14, and the machine learning unit 15 may be considered to represent the learning process of artificial intelligence as a whole.

The feature amount extraction unit 13 represents a learning process in which the artificial intelligence in the learning phase extracts feature amounts from interferogram data sent and input from the interferogram holding unit 9 according to the purpose of learning. ing. A feature is a term used in the field of learning, and is a variable in data to be analyzed that serves as a clue for prediction. Further, feature extraction is converting data to be analyzed into a feature amount.
The feature quantities extracted for each input interferogram data are multidimensional quantities (vectors), and are shown on the feature quantity map shown below the text "Feature quantity extraction" in Figure 3, which will be described later. can be represented as a plot of A feature map is sometimes referred to as a feature space. Note that as learning progresses, the definition of the feature may change, and the dimensions of the feature may increase or decrease.
The feature amount may artificially include a feature of the envelope for the attenuation of the interferogram, a time width of one vibration in the interferogram, and other amounts having a correlation with the optical spectrum.

The teacher data creation unit 14 represents the preparation process of preparing a sufficient number of learning data sets to perform artificial intelligence learning. Once the number of training datasets is complete, a plot distribution will appear for each label on the feature map.

The machine learning unit 15 represents a core process in the learning process of artificial intelligence. For example, when an artificial intelligence advances its learning to solve a classification problem, the artificial intelligence constructs a classification surface that separates classes, as shown in the feature map at the bottom right of FIG. Artificial intelligence may apply, for example, a support vector machine algorithm when constructing the classification surface.

《Main analysis unit 16 in data processing system》
The analysis unit 16 is a component for analyzing interferogram data acquired by the data processing system via the interferogram holding unit 9. The user of the data processing system can utilize the results analyzed by the analysis unit 16.

<<About the operation of the sensor system according to Embodiment 1>>
FIG. 2 is a diagram illustrating the operation or processing of the sensor system according to the first embodiment.

The upper left part of Figure 2 shows the sensor system mounted on the flying object observing the observation target. As described above, the flying object is, for example, a drone, an aircraft, or an artificial satellite.
The optical measurement sensor unit 1 is a Fourier spectroscopy type optical interferometer as described above, and the signals detected here are based on the detection time (t) and the two-dimensional coordinates (x, y) of the observation point. , are associated. The two-dimensional coordinates (x, y) of the observation point are determined comprehensively, taking into consideration processing details such as the position of the flying object, the direction of light irradiation, TOF (Time of Flight), and beam forming. The two-dimensional coordinates (x, y) of the observation point may be expressed, for example, in a geographic coordinate system.
When emphasizing that the variables x, y, and t are discrete variables, they are expressed with subscripts, such as x _n , y _n , and t _n .
Time t _n represents the sampling time when the nth sampling was performed. Sampling generally starts at time t ₀ =0. That is, the sampling time is a time based on the time when sampling started. The subscript for sampling time (t) starts at 0.
The two-dimensional coordinates (x _n , y _n ) mean the coordinates for the n-th observation point. Subscripts used for two-dimensional coordinates for observation points start at 1. Here, n used as a subscript is simply a variable and has a meaning as a counter variable in a for statement. n does not mean the total number of something. The n used for the two-dimensional coordinates (x _n , y _n ) and the n used for the time t _n do not need to be the same value. Although the same n is used in this specification, different variables may be used for two-dimensional coordinates and time in a program that implements the disclosed technology.

The upper center part of FIG. 2 shows how the interference image acquisition unit 3 acquires an interference image. As a first process, the interference image acquisition section 3 acquires a digital signal from the AD conversion section 2. The digital signal is represented by the symbol F(), as shown in FIG. The digital signal (F(x, y, t)) can also be said to be interferogram array data.

The upper right part of FIG. 2 shows how the interferogram storage unit 4 acquires interferogram data for pixels corresponding to a certain observation point from the interference image sent from the interference image acquisition unit 3. There is. For example, the graph shown with the characters "F (x ₁ , y ₁ , t)" in the upper right part of Figure 2 is the first observation point (two-dimensional coordinates are (x ₁ , y ₁ )) 3 is a graph showing interferogram data for pixels corresponding to . Similarly, the graph shown with the characters "F (x ₂ , y ₂ , t)" in the upper right part of Figure 2 is the second observation point (the two-dimensional coordinates are (x ₂ , y ₂ ) ) is a graph showing interferogram data for pixels corresponding to .
In this way, a certain observation point on the ground corresponds to a pixel at a specific position in the interference image.
It is assumed that the interferogram data (F(x _n , y _n , t)) has a data length of observation time necessary to obtain the wave number resolution (also referred to as spectral resolution) required by the specifications. The sensor system according to the present disclosure acquires interferogram data (F(x _n , _yn , t)) for each pixel corresponding to all observation points.

In the lower left part of FIG. 2, the signal analysis unit 5 performs object determination and importance analysis on the interferogram data (F(x _n , y _n , t)) sent from the interferogram storage unit 4. This shows how signal analysis is performed.

The spectrum analysis unit 5a of the signal analysis unit 5 performs Fourier transform in the time direction on the interferogram data (F(x _n , _yn , t)) sent from the interferogram storage unit 4 . Optical spectrum data obtained by Fourier transforming interferogram data (F(x _n , y _n , t)) in the time direction is an array, and is expressed as B(x _n , y _n ). The optical spectrum data (B(x _n , y _n )) obtained by the spectrum analysis section 5a is sent to the object determination section 5b.

The object determination unit 5b of the signal analysis unit 5 uses interferogram data (F(x _n , y _n , t)), optical spectrum data (B(x _n , y _n )), or both to determine whether all For observation points (observation points at all n), determine what the object was like. The object determination unit 5b specifically determines whether there are any clouds between the sensor system and the observation point. In order to confirm the presence or absence of clouds, the object determination unit 5b prepares a reference interferogram for the case where clouds are present, and compares it with the interferogram data (F(x _n , y _n , t)) of the observation target. A correlation coefficient with a reference interferogram may be determined and compared with a preset threshold.
The target determination performed by the target determination unit 5b is realized by trained artificial intelligence, as described above.

The importance analysis unit 5c of the signal analysis unit 5 uses interferogram data (F(x _n , _yn , t)), optical spectrum data (B(x _n , _yn )), and the judgment of the target judgment unit 5b. Based on the results, analyze the importance of each observation point.
In analyzing the importance of each observation point, the importance analysis unit 5c analyzes the area of pixels corresponding to the area of the observation point, the unique frequency characteristics (continuous spectrum (absorption lines that appear) and the amount of data in the received signal.

The lower center portion of FIG. 2 shows how the data reduction section 6 includes at least one of an area reduction section 6a, a band reduction section 6b, and a bit reduction section 6c.

The bit reduction carried out by the bit reduction unit 6c can be carried out in two ways: a method of reducing the upper bits and a method of reducing the lower bits.
As described above, in the time domain data of the interferogram, the time periods that do not contribute to measurement accuracy are generally the time periods at both ends of the start and end times. Since the interferogram signal is not large in this time period, only 0 is included in the upper bits of the signal data. Therefore, the bit reduction unit 6c can reduce the upper bits of the signal data in the time periods at both ends of the interferogram.
On the contrary, in the time domain data of the interferogram, the signal amplitude is large and the S/N ratio is high in the time zone between the start and end times. During this time period, the information in the lower bits of the signal data of the interferogram is not important. Therefore, the bit reduction unit 6c can also reduce the lower bits of the signal data in a time period intermediate between the start and end of the interferogram.
The bit reduction unit 6c may perform upper bit reduction, lower bit reduction, or a combination of upper bit reduction and lower bit reduction.

In this way, the data reduction unit 6 performs data reduction on the interferogram data (F(x _n , _yn , t)) and generates interferogram data with data reduction performed. The interferogram data subjected to data reduction is expressed as F'(x _n , _yn , t). F'(x _n , y _n , t) is similar to the original signal F(x _n , y _n , t), so it is expressed as a prime ('', usually used in British English), which is used to express similarity in mathematical notation. dash) is used.

The lower right part of FIG. 2 shows how the interference image at time _tn is affected by data reduction. The interference image affected by data reduction is denoted as F'(x, y, t _n ), as shown in FIG. 2 . An interferogram affected by data reduction (F′(x, y, t _n )) can be constructed from interferogram data (F′(x _n , y _n , t)) subjected to data reduction. can.

FIG. 3 is a diagram illustrating the operation or processing of the data creation unit 7, memory unit 8, and data processing system of the sensor system according to the first embodiment.

The upper part of FIG. 3 shows how the three-dimensional data reconstructed by the data creation section 7 is stored in the memory section 8.
The middle part of FIG. 3 shows how the three-dimensional data stored in the memory section 8 is transferred to the data processing system.
The lower portion of FIG. 3 generally describes the operation or processing of the data processing system.

In the lower part of FIG. 3, a signal waveform of F''(x _n , _yn , t) is shown as a diagram for explaining the operation or processing of the interferogram holding unit 9. Considering the influence of data transfer, F''(x _n , y _n , t) is more prime than F' (x _n , _yn , t) stored in the memory unit 8 on the sensor system side. It is expressed by increasing the number.

In the lower part of FIG. 3, as a diagram explaining the operation or processing of the spectrum restoration unit 10, B''(x ₁ , y ₁ , λ), B ''(x ₂ , y ₂ , λ), and two The spectral waveform is shown. B''(x ₁ , y ₁ , λ) is spectrum data whose spectrum is restored from F'' (x ₁ , y ₁ , t). For n=1, 2,..., B''(x _n , y _n , λ) is specifically obtained by Fourier transforming F'' (x _n , y _n , t) in the time direction. . Note that λ in the argument of B''(x _n , _yn , λ) is the wavelength. In other words, including λ in the argument of B''(x _n , y _n , λ) indicates that the spectral data is a quantity expressed in the wavelength domain (or frequency domain) (frequency is the speed of light divided by the wavelength). This is contrasted with the fact that interferogram data (F(x _n , y _n , t)) is a quantity that includes t as an argument and is expressed in the time domain.

In the lower part of FIG. 3, spectral image data for a group of B''(x, y, λ) is shown as a diagram for explaining the operation or processing of the imaging processing unit 11. The spectral image data created by the imaging processing unit 11 can be said to be a three-dimensional data cube consisting of a two-dimensional coordinate plane and a spectrum at the coordinates.
The lower part of FIG. 3 shows how the imaging processing unit 11 corrects data by processing such as interpolation for the portions where data has been reduced when generating image data.

In the lower part of FIG. 3, the white arrow above the text "label creation" represents the operation or processing of the label creation section 12.

At the bottom of FIG. 3, the white arrow to the left of the words "feature extraction" represents the operation or processing of the feature extraction unit 13.

At the bottom of FIG. 3, the feature map above the text "Teacher data creation" represents the operation or processing of the teacher data creation unit 14. In the feature amount map shown in FIG. 3, the horizontal axis represents the first feature amount (feature amount 1), and the vertical axis represents the second feature amount (feature amount 2). A sample with a first label (label 1) and a sample with a second label (label 2) are plotted on the feature map.

At the bottom of FIG. 3, the white arrow above the words "machine learning model creation" represents the operation or processing of the teacher data creation unit 14.

At the bottom of FIG. 3, the feature map with the symbol “15” represents the operation or processing of the machine learning unit 15. In the feature map with the code "15", the classification plane divides the sample into "object 1" with the first label and "object 2" with the second label. It is classified as The feature amount map shown in FIG. 3 shows an example in which "object 1" and "object 2" are classified into classes based on differences in importance. In FIG. 3, the importance of "object 1" is α ₁ , and the importance of "object 2" is α ₂ .

The sensor system according to the disclosed technology has industrial applicability because it can be mounted on a drone and applied to sensing the vegetation state of a field, for example.

1 Optical measurement sensor section, 2 AD conversion section, 3 Interference image acquisition section, 4 Interferogram storage section, 5 Signal analysis section, 5a Spectrum analysis section, 5b Target determination section, 5c Importance analysis section, 6 Data reduction section, 6a area reduction unit, 6b band reduction unit, 6c bit reduction unit, 7 data creation unit, 8 memory unit, 9 interferogram storage unit, 10 spectrum restoration unit, 11 image processing unit, 12 label creation unit, 13 feature amount Extraction section, 14. Teacher data creation section, 15. Machine learning section, 16. Main analysis section.

Claims (7)

an optical measurement sensor unit that is mounted on a flying object and acquires an interferogram of an observation target;
an object determination unit that determines the observation target based on the interferogram;
a data reduction unit that performs data reduction on the interferogram based on the determination result of the target determination unit;
sensor system. The target determination unit includes:
Determine the observation target using a learning model that has been subjected to machine learning in advance,
calculating a correlation coefficient between the interferogram and a reference interferogram when clouds are present, and determining the presence or absence of clouds;
The sensor system according to claim 1. The learning model in the target determination unit uses a semantic segmentation algorithm,
The sensor system according to claim 2. further comprising an importance analysis unit that analyzes the importance of the observation target based on the determination result of the target determination unit,
The data reduction unit refers to the analysis result of the importance analysis unit and performs data reduction on the interferogram.
The sensor system according to claim 1. The data reduction unit includes any one of an area reduction unit, a band reduction unit, or a bit reduction unit,
The area reduction unit performs data reduction in area units based on the determination result and analysis result,
The band reduction unit performs data reduction in frequency band units based on the determination result and analysis result,
The bit reduction unit performs data reduction in units of bits based on the determination result and the analysis result.
The sensor system according to claim 4. The optical measurement sensor section is a Fourier spectroscopy type optical interferometer,
The sensor system according to claim 1. A data processing system for a sensor system according to claim 2, comprising:
comprising a pre-learning processing unit that performs the machine learning on the learning model used by the target determination unit;
Data processing system for sensor systems.

Images