JP6571027B2 - Data storage device and data storage program - Google Patents

Description

  The present invention relates to a technique for storing data acquired from a sensor in a storage.

  In some cases, data acquired from one or more sensors is stored in a storage, and later analysis is performed using the data stored in the storage. In many cases, in order to increase the accuracy of the analysis, all data acquired from the sensor is stored in the storage as it is.

JP 2015-28742 A

If all the data acquired from the sensor is stored as it is in the storage, the accuracy of the analysis may increase. However, the amount of data stored in the storage increases.
An object of the present invention is to reduce the amount of data stored in a storage while maintaining the accuracy of analysis.

A data storage device according to the present invention includes:
A receiver for receiving data from the sensor;
A required quantity identifying unit that identifies the required quantity, which is the amount of data necessary for analysis,
Specify the data extraction interval from the required amount specified by the required amount specifying unit and the received amount which is the data amount of data received in the reference period by the receiving unit, and from the data received by the receiving unit, A storage unit that extracts data at the specified extraction interval and stores the data in a storage;

  The required amount specifying unit specifies, as the required amount, a data amount that provides a specified analysis accuracy when an analysis is performed using data stored in the storage as an input.

The required amount specifying unit specifies a data amount for which the specified analysis accuracy is obtained for each analysis method, and specifies the largest data amount as the required amount among the data amounts specified for each analysis method.

  The required amount specifying unit calculates the required amount when an update period has elapsed, when an analysis method is added, when an analysis method is deleted, and when a specified analysis accuracy is changed. Re-identify.

  When the required amount increases during the update period, the storage unit is configured to increase the required amount for a past period from the start of the update period to the time when the required amount is specified again. Data obtained by interpolating the data stored in the storage so as to be the number of cases extracted by the specified extraction interval is stored.

The data storage device further includes:
A format determination unit for determining whether the format of the data is correct;
The storage unit extracts data from the data determined to be correct by the format determination unit at an extraction interval and stores the data in the storage.

The data includes the sensor ID of the source sensor,
The data storage device further includes:
A sensor storage unit that stores one or more sensor IDs that are sensor identification information;
A sensor determination unit that determines whether or not the sensor ID stored in the sensor storage unit includes a sensor ID included in the data received by the reception unit;
The storage unit extracts data from the data determined to be in the sensor determination unit at an extraction interval and stores the data in the storage.

  The storage unit stores data received by the receiving unit in a temporary storage unit, and when a reference amount of data is accumulated in the temporary storage unit, the data is extracted from the temporary storage unit at the extraction interval. Store in storage.

A data storage program according to the present invention includes:
A receiving process for receiving data from the sensor;
A required amount specifying process for specifying a required amount, which is the amount of data necessary for analysis,
Specify the data extraction interval from the required amount specified by the required amount specifying process and the received amount that is the data amount of data received in the reference period by the receiving process, and from the data received by the receiving process, Data is extracted at the specified extraction interval and stored in the storage.

  In this invention, the data extraction interval is specified from the necessary amount necessary for data analysis, and the data is extracted at the extraction interval and stored in the storage. Therefore, the amount of data stored in the storage can be reduced while maintaining the accuracy of analysis.

1 is a configuration diagram of a data storage device 10 according to Embodiment 1. FIG. Explanatory drawing of the data which the collection condition memory | storage part 121 which concerns on Embodiment 1 memorize | stores. Explanatory drawing of the data which the analysis precision memory | storage part 122 which concerns on Embodiment 1 memorize | stores. Explanatory drawing of the data which the sensor memory | storage part 123 which concerns on Embodiment 1 memorize | stores. FIG. 3 is an explanatory diagram of data stored in a storage 32 according to the first embodiment. 3 is a flowchart showing an overall operation of the data storage device 10 according to the first embodiment. Explanatory drawing of the specific timing of the required amount which concerns on Embodiment 1. FIG. 5 is a flowchart showing a necessary amount specifying process in step S1 according to the first embodiment. Explanatory drawing of the result precision which concerns on Embodiment 1. FIG. 5 is a flowchart showing data storage processing in step S8 according to the first embodiment. Explanatory drawing when a required amount increases in the middle of the update period which concerns on Embodiment 1. FIG. Explanatory drawing of the data interpolation which concerns on the modification 1. FIG. 10 is a flowchart showing a target amount specifying process according to Modification 4; The block diagram of the data storage apparatus 10 which concerns on the modification 5. FIG.

Embodiment 1 FIG.
*** Explanation of configuration ***
The configuration of the data storage device 10 according to the first embodiment will be described with reference to FIGS. 1 to 5.
As shown in FIG. 1, the data storage device 10 is a computer.
The data storage device 10 includes hardware of a processor 11, a storage device 12, an input interface 13, and an output interface 14. The processor 11 is connected to other hardware via a signal line, and controls these other hardware.

  The processor 11 is an IC (Integrated Circuit) that performs processing. Specifically, the processor 11 is a CPU (Central Processing Unit), a DSP (Digital Signal Processor), or a GPU (Graphics Processing Unit).

  The storage device 12 is a RAM (Random Access Memory) or an HDD (Hard Disk Drive). The storage device 12 may be a portable storage medium such as an SD (Secure Digital) memory card, a CF (Compact Flash), a NAND flash, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD.

  The input interface 13 is a device that is connected to one or more sensors 31 and receives data from the sensors 31. As a specific example, the input interface 13 is a USB (Universal Serial Bus), a connection terminal of IEEE1394, or a NIC (Network Interface Card).

  The output interface 14 is a device that is connected to the storage 32 and transmits data to the storage 32. As a specific example, the output interface 14 is a connection terminal of USB or IEEE1394, or a NIC.

The data storage device 10 includes a necessary amount specifying unit 21, a receiving unit 22, a format determining unit 23, a sensor determining unit 24, and a storing unit 25 as functional components. The functions of the necessary amount specifying unit 21, the receiving unit 22, the format determining unit 23, the sensor determining unit 24, and the storage unit 25 are realized by software.
The storage device 12 stores a program that realizes the functions of the respective units of the data storage device 10. This program is read and executed by the processor 11. Thereby, the function of each part of the data storage device 10 is realized.

  Information, data, signal values, and variable values indicating the processing results of the functions of the respective units realized by the processor 11 are stored in the storage device 12 or in a register or cache memory in the processor 11. In the following description, it is assumed that information, data, signal values, and variable values indicating the processing results of the functions of the respective units realized by the processor 11 are stored in the storage device 12.

  It is assumed that a program for realizing each function realized by the processor 11 is stored in the storage device 12. However, this program may be stored in a portable storage medium such as a magnetic disk, a flexible disk, an optical disk, a compact disk, a Blu-ray (registered trademark) disk, or a DVD.

  In FIG. 1, only one processor 11 is shown. However, a plurality of processors 11 may be provided, and a plurality of processors 11 may execute programs that realize each function in cooperation with each other.

  The storage device 12 implements a collection condition storage unit 121, an analysis accuracy storage unit 122, a sensor storage unit 123, and a temporary storage unit 124.

The collection condition storage unit 121 stores conditions for collecting data.
As shown in FIG. 2, the collection condition storage unit 121 stores a data collection interval and the number of normal columns. The data collection interval is an interval for receiving data from the sensor 31. The normal column number is the number of normal items of data received from the sensor 31. The number of normal rows may be determined for each sensor 31.

The analysis accuracy storage unit 122 stores the accuracy of analysis using the data collected from the sensor 31.
As shown in FIG. 3, the analysis accuracy storage unit 122 inputs, for each analysis method, analysis accuracy, tolerance, data amount, total convergence time, number of trials, accuracy transition, data amount transition, and input. Data is stored. The analysis accuracy is the accuracy required by the corresponding analysis method. The allowable range indicates what percentage is acceptable before and after the accuracy indicated by the analysis accuracy. The amount of data is the amount of data necessary for analyzing the accuracy indicated by the analysis accuracy. The total convergence time is the time taken to calculate the amount of data. The number of trials is the number of times analysis processing has been performed when calculating the data amount. The precision transition is a precision transition obtained by the analysis process. The data amount transition is a transition of the data amount used in the analysis process. The input data is an identifier of the learning data 33 used in the analysis process.

The sensor storage unit 123 stores information about the sensor 31.
As illustrated in FIG. 4, the sensor storage unit 123 stores a sensor group, a priority order, a sensor ID, a facility name, a device name, and a model number for each sensor 31 connected to the data storage device 10. The sensor group is a name given to a set of one or more sensors 31. The priority order is a priority order within the sensor group. The sensor ID is an identifier of the sensor 31. The facility name is the name of the facility where the sensor 31 is installed. The device name is a name indicating the type of the sensor 31, and specific examples include a temperature sensor, a humidity sensor, and a pressure sensor. The model number is the model number of the sensor 31.

  The temporary storage unit 124 temporarily stores data being processed by the data storage device 10.

  The sensor 31 detects the detection value and transmits data including the detection value to the data storage device 10 at intervals stored in the collection condition storage unit 121. Specific examples of the detected value include temperature, humidity, and pressure.

The storage 32 stores at least a part of the data extracted by the data storage device 10 among the data detected by the sensor 31.
As shown in FIG. 5, the storage 32 stores data separately for each update period. A set of data divided for each update period is referred to as learning data 33. In Embodiment 1, it is assumed that the update period is one month from the beginning of the month to the end of the month. Therefore, for example, the learning data 33 for each month is stored in the storage 32 such as the learning data 33 for December 2015, the learning data 33 for January 2016, and the learning data 33 for February 2016. The

*** Explanation of operation ***
The operation of the data storage device 10 according to the first embodiment will be described with reference to FIGS.
The operation of the data storage device 10 according to the first embodiment corresponds to the data storage method according to the first embodiment. The operation of the data storage device 10 according to the first embodiment corresponds to the processing of the data storage program according to the first embodiment.

With reference to FIG. 6, the overall operation of the data storage apparatus 10 according to the first embodiment will be described.
In the required amount specifying process of step S1, the required amount specifying unit 21 specifies a required amount that is a data amount necessary for analysis. The amount of data necessary for analysis means the amount of data that provides the specified analysis accuracy.
Specifically, the necessary amount specifying unit 21 uses each analysis method stored in the analysis accuracy storage unit 122 as a target method, and analyzes the data stored in the storage by using the target method. As a result, the necessary amount specifying unit 21 specifies the amount of data from which the analysis accuracy stored in the analysis accuracy storage unit 122 for the target method can be obtained. And the required amount specific | specification part 21 specifies the largest data amount as a required amount among the data amounts specified for every analysis method.

  In the data reception process of step S <b> 2, the reception unit 22 receives data from the sensor 31 at the data collection interval stored in the collection condition storage unit 121.

In the format determination process in step S3, the format determination unit 23 determines whether the format of the data received in step S2 is correct.
Specifically, the format determination unit 23 decomposes the data for each item and specifies the number of items included in the data. Then, the format determination unit 23 determines whether or not the specified number of items matches the number of normal columns stored in the collection condition storage unit 121. The format determination unit 23 determines that the data format is correct if they match, and determines that the data format is incorrect if they do not match. That is, the format determination unit 23 determines that the data format is not correct when a part of the data is lost during communication.
If the data format is correct, the format determination unit 23 proceeds to step S4. If not, the format determination unit 23 proceeds to step S6.

In the sensor determination process in step S4, the sensor determination unit 24 determines whether the data received in step S2 is data transmitted from the designated sensor 31.
Specifically, the sensor determination unit 24 stores the sensor ID included in the data received in step S2 in the sensor storage unit 123 and the sensor ID included in the data received in step S2. It is determined whether or not the priority order is 1. When the sensor ID is stored and the priority is 1, the sensor determination unit 24 determines that the data is transmitted from the specified sensor 31. Otherwise, the sensor determination unit 24 is other than the sensor 31 for which the data is specified. It is determined that it was transmitted from.
The sensor determination unit 24 advances the process to step S5 when data is transmitted from the designated sensor 31, and advances the process to step S6 if not.

  In the temporary storage process of step S5, the storage unit 25 stores the data received in step S2 in the temporary storage unit 124. On the other hand, in the data discarding process in step S6, the storage unit 25 discards the data received in step S1.

In the capacity determination process in step S7, the storage unit 25 determines whether or not the amount of data stored in the temporary storage unit 124 is greater than or equal to the reference amount. The reference amount is a data amount set in advance, and is an arbitrary data amount.
If the data amount is greater than or equal to the reference amount, the storage unit 25 advances the process to step S8, and otherwise returns the process to step S2.

In the data storage process in step S8, the storage unit 25 uses the necessary amount specified in step S1 and the received amount that is the data amount of the data received in the reference period in step S2, to store the necessary amount of data in the update period. The extraction interval of the data from which is extracted is specified.
Then, the storage unit 25 reads the data from the temporary storage unit 124 one by one in the order of the reception time, extracts the data at the extraction interval, and stores it in the storage 32. At this time, the storage unit 25 stores the data in the storage 32 separately for each update period. As a specific example, the storage unit 25 stores data in a separate table or file for each update period, or stores data with a different identifier for each update period. The storage unit 25 discards data that has not been extracted from the read data.

In the necessary amount determination process in step S9, the necessary amount specifying unit 21 determines whether or not re-specification of the necessary amount is necessary.
Specifically, the necessary amount specifying unit 21 (1) when the update period has elapsed, (2) when a record is added to the analysis accuracy storage unit 122 and an analysis method is added, (3) analysis accuracy storage When the record is deleted from the part 122 and the analysis technique is deleted, it is determined whether or not (4) any of the cases where the designated analysis accuracy is changed. Then, the necessary amount specifying unit 21 determines that re-specification of the necessary amount is necessary if any of (1) to (4) is applicable, and if not, re-identifies the necessary amount. Is determined to be unnecessary.
That is, as shown in FIG. 7, (0) after the required amount is specified at the start of the process shown in FIG. 6, the required amount is re-specified for each update period by (1), and from (2) Due to (4), the required amount may be re-specified during the update period.
The necessary amount specifying unit 21 returns the process to step S1 if re-specification of the required amount is necessary, and returns the process to step S2 otherwise.

With reference to FIG. 8, the required amount specifying process in step S1 will be described.
The required amount specifying process will be described for each of the five cases (0) to (4) shown in FIG.

(0) At the start of the process shown in FIG. 6 As a premise, it is assumed that one learning data 33 is stored in the storage 32 when the process shown in FIG. 6 is disclosed. The learning data 33 is, as a specific example, all data transmitted from the sensor 31 during the most recent update period. Alternatively, the learning data 33 is determined to be correct in step S3 among all data transmitted from the sensor 31 during the most recent update period, and transmitted from the sensor 31 designated in step S4. Data to be determined may be used. Alternatively, the learning data 33 may be test data generated separately.

  In step S <b> 11, the necessary amount specifying unit 21 reads the learning data 33 stored in the storage 32. Then, the necessary amount specifying unit 21 extracts target amount data from the read learning data 33. It is assumed that the initial value of the target amount is set in advance.

  In step S <b> 12, the necessary amount specifying unit 21 uses one extracted analysis method stored in the analysis accuracy storage unit 122 as a target method and analyzes the extracted data using the target method.

In step S <b> 13, the necessary amount specifying unit 21 compares the result accuracy, which is the analysis accuracy of the analysis result, with the analysis accuracy stored in the analysis accuracy storage unit 122 for the target method.
If the result accuracy is within the allowable range based on the analysis accuracy stored in the analysis accuracy storage unit 122, the required amount specifying unit 21 stores the current target amount as the data amount for the target method in the analysis accuracy storage unit 122. The writing and processing are advanced to step S14.
On the other hand, if the result accuracy is not within the allowable range, the required amount specifying unit 21 changes the target amount and returns the process to step S11. At this time, the required amount specifying unit 21 increases the target amount when the result accuracy is lower than the analysis accuracy stored in the analysis accuracy storage unit 122, and the result accuracy is higher than the analysis accuracy stored in the analysis accuracy storage unit 122. If is high, reduce the target amount.

In step S <b> 14, the necessary amount specifying unit 21 determines whether an unprocessed analysis method is stored in the analysis accuracy storage unit 122.
If the unprocessed analysis method is stored, the required amount specifying unit 21 returns the process to step S12. If the unprocessed analysis method is not stored, the necessary amount specifying unit 21 proceeds with the process to step S15.

  In step S15, the required amount specifying unit 21 specifies the largest data amount as the required amount among the data amounts written in the analysis accuracy storage unit 122 for each analysis method in step S13.

A specific example will be described with reference to FIG.
The initial value of the target amount is 15 MB (megabytes) (see the data amount transition column in FIG. 3). In step S <b> 11, the necessary amount specifying unit 21 extracts 15 MB of data from the learning data 33. In step S12, the necessary amount specifying unit 21 analyzes the 15 MB data as an input using the analysis method “neural network” as a target method. As a result, the result accuracy is assumed to be 20% (see the accuracy transition column in FIG. 3). Then, the analysis technique “neural network” does not fall within the allowable range of ± 5% based on the analysis accuracy of 80%. Therefore, the necessary amount specifying unit 21 increases the target amount to 30 MB, and returns the process to step S11 (see the data amount transition column in FIG. 3).

  Next, the required amount specifying unit 21 extracts 30 MB of data from the learning data 33 in step S11. In step S12, the necessary amount specifying unit 21 analyzes the 30-MB data as an input using the analysis method “neural network” as a target method. As a result, it is assumed that the result accuracy is 40% (see the accuracy transition column in FIG. 3). Then, it is not within the allowable range ± 5% based on the analysis accuracy of 80%. Therefore, the necessary amount specifying unit 21 increases the target amount to double 60 MB and returns the process to step S11 (see the data amount transition column in FIG. 3).

When the same processing is executed, the result accuracy is 60% when the target amount is 60 MB, and the target amount is increased to 120 MB because it is not within the allowable range ± 5% with reference to the analysis accuracy of 80%. When the target amount is 120 MB, the result accuracy is 90%. In this case, the analysis accuracy is not within the allowable range of ± 5% based on the analysis accuracy of 80%, but the result accuracy is higher than the analysis accuracy of 80% for the first time. Therefore, the necessary amount specifying unit 21 reduces the target amount to 100 MB and returns the process to step S11 (see the data amount transition column in FIG. 3). Here, when the target amount was increased from 60 MB to 120 MB, the result accuracy changed from 60% to 90%. Therefore, the target amount of 100 MB is a data amount when the result accuracy is 80%, which is calculated on the assumption that the change in the target amount is directly proportional to the change in the result accuracy. Specifically, 100 MB = 60 MB + ((120 MB−60 MB) / (90% −60%) × (80% −60%)).
When the target amount is 100 MB, the accuracy of the result is 80%, which is within the allowable range ± 5% based on the analysis accuracy of 80%. Therefore, in step S13, the necessary amount specifying unit 21 writes the target amount of 100 MB in the analysis accuracy storage unit 122 as the data amount for the analysis method “neural network”.

  Next, since an unprocessed analysis method is stored in the analysis accuracy storage unit 122 in step S14, the process returns to step S12. Then, in step S12, the necessary amount specifying unit 21 uses the analysis method “Bayesian network” as an object method, and analyzes 100 MB of data that is the object amount as input. As a result, the result accuracy is assumed to be 75% (see the accuracy transition column in FIG. 3). Then, the analysis accuracy of the Bayesian network is 70% within the allowable range ± 5% with reference to the analysis accuracy of 70%. Therefore, in step S <b> 13, the necessary amount specifying unit 21 writes the target amount of 100 MB in the analysis accuracy storage unit 122 as the data amount for the analysis technique “Bayesian network”.

  Similarly, processing is executed for the analysis technique “random forest”, and 200 MB is written as the data amount.

  Then, in step S15, the required amount specifying unit 21 specifies 200 MB, which is the largest data amount among the data amounts written in the analysis accuracy storage unit 122 for each analysis method, as the required amount.

(1) When the update period has passed (1) When the update period has passed, the point that the latest learning data 33 is used is different from (0) when the process shown in FIG. 6 is started. That is, in step S <b> 11, the necessary amount specifying unit 21 reads the latest learning data 33 among the learning data 33 stored in the storage 32. The subsequent processing is the same as (0) at the start of the processing shown in FIG.

(2) When a record is added to the analysis accuracy storage unit 122 The point that steps S11 to S14 are executed only for the analysis method of the added record is the processing target. (1) When the update period has passed Different. That is, only the analysis method of the added record is set as the target method in step S12. In step S15, the largest amount of data is specified as the required amount for all analysis methods stored in the analysis accuracy storage unit 122.

(3) When a record is deleted from the analysis accuracy storage unit 122 Steps S11 to S14 are not executed, and the largest amount of data is specified as the required amount for the analysis method of the remaining record in step S15. .

(4) When the designated analysis accuracy is changed The point that steps S11 to S14 are executed only for the analysis method of the record whose analysis accuracy has been changed is the processing target. And different. That is, only the analysis method of the record whose analysis accuracy is changed is set as the target method in step S12. In step S15, the largest amount of data is specified as the required amount for all analysis methods stored in the analysis accuracy storage unit 122.

  In step S <b> 11, when the amount of data included in the learning data 33 is smaller than the target amount, the necessary amount specifying unit 21 also uses the old learning data 33.

It supplements about the result precision used by step S13.
In order to calculate the result accuracy, the necessary amount specifying unit 21 prepares correct data in advance, and calculates the result accuracy by comparing the analyzed result with the correct data.
As a specific example, (0) a part of the learning data 33 used at the start of the process shown in FIG. For example, (0) When the learning data 33 used at the start of the process shown in FIG. 6 is all data transmitted from the sensor 31 in one month, the data for one day of the learning data 33 is correct. Data 34 is assumed.
In step S12, the necessary amount specifying unit 21 performs an analysis and calculates an analysis value for one day. Then, in step S13, the necessary amount specifying unit 21 compares the analysis value for the same time with the value of the correct answer data 34, and calculates the coincidence rate. The necessary amount specifying unit 21 uses the calculated coincidence rate as the result accuracy.
For example, as shown in FIG. 9, it is assumed that the values a1 to a15 are given as the correct answer data 34 and the values b1 to b15 are obtained as analysis values. At this time, the necessary amount specifying unit 21 compares values at the same time such as a1 and b1 and a2 and b2. Then, if the value of the analysis value is within the reference range based on the value of the correct answer data 34 to be compared, the necessary amount specifying unit 21 determines that the value of the analysis value matches the value of the correct answer data 34. , Calculate the match rate. For example, if a2− (0.1 × a2) ≦ b2 ≦ a2 + (0.1 × a2), it is determined that the analysis value b2 matches the value of the correct answer data 34.
When the value of the learning data 33 is interpolated by the analysis, the value included in the learning data 33 may be excluded from the objects to be compared with the correct answer data 34. In FIG. 9, it is assumed that the points shown in black among the analysis values indicate values included in the learning data 33. In this case, b2, b4, b6,. . . Only for, the matching rate is calculated by comparing with the value of the correct answer data 34 at the same time.

With reference to FIG. 10, the data storage processing in step S8 will be described.
In step S81, the storage unit 25 determines whether or not the necessary amount has been specified in step S1, after specifying the extraction interval in the process of step S82 described later.
The storage unit 25 advances the process to step S82 when the required amount is specified, and advances the process to step S83 when the required amount is not specified.

In step S <b> 82, the storage unit 25 specifies a data extraction interval so that a necessary amount of data is evenly extracted during the update period. The reason why the data is evenly extracted is that there is a case where the analysis accuracy is low for data biased at a certain time period or a certain time zone, and the data can be used only for a certain purpose.
Specifically, the storage unit 25 calculates the necessary amount per reference period by dividing the necessary amount specified in step S1 by a value obtained by dividing the update period by the reference period. In addition, the storage unit 25 calculates an average amount of data received by the receiving unit 22 per reference period. Here, the storage unit 25 stores the number of designated sensors 31, the number of data collected from one sensor 31 per reference period specified from the data collection interval, and the average data amount of one piece of data. Multiply to calculate the average amount of data.
Then, the storage unit 25 divides the number of data collected from one sensor 31 by a value obtained by dividing the average data amount by the necessary amount, and determines the number of data extracted from one sensor 31 per reference period. calculate. The storage unit 25 calculates the extraction interval by dividing the number of data collected from one sensor 31 per reference period by the calculated number of data.

As a specific example, the reference period is 1 minute. Also, assume that the required amount per minute is 10 MB. Further, it is assumed that the number of designated sensors 31 is 1000, the number of data collected from one sensor 31 per minute is 61, and the average data amount of one data is 512 B (bytes). . Then, the average amount of data received by the receiving unit 22 per minute is 29.7 MB, and the number of data extracted from one sensor 31 per reference period is 61 cases / (29.7 MB / 10 MB) = 20. 54. The extraction interval is 61 cases / 20.54 cases = 2.97 cases.
In other words, if data collected from each sensor 31 is extracted for every 2.97≈3, a necessary amount of data is stored in the storage 32 during the update period.

  In step S <b> 83, the storage unit 25 reads one piece of data in the order collected from the temporary storage unit 124.

In step S84, the storage unit 25 determines whether or not the counter value is equal to the extraction interval. Note that the extraction interval is a positive integer.
The storage unit 25 advances the process to step S85 if the counter value is equal to the extraction interval, and advances the process to step S86 if the counter value is not equal to the extraction interval. The initial value of the counter is 0.

In step S85, the storage unit 25 sets the value of the counter to 0 and holds the read data. On the other hand, in step S86, the storage unit 25 adds 1 to the counter value and discards the read data.
In step S <b> 87, the storage unit 25 determines whether there is unprocessed data in the temporary storage unit 124. If there is unprocessed data, the storage unit 25 returns the process to step S83; otherwise, the process proceeds to step S88.

  In step S <b> 88, the storage unit 25 divides the held data for each collection source sensor 31, compresses it, and stores it in the storage 32. In addition, the storage unit 25 deletes the data in the temporary storage unit 124.

*** Effects of Embodiment 1 ***
As described above, the data storage device 10 according to the first embodiment specifies the data extraction interval from the necessary amount necessary for data analysis, extracts the data at the specified extraction interval, and stores the data in the storage 32. To do. As a result, the amount of data stored in the storage 32 can be reduced while maintaining the accuracy of analysis.

  In addition, the data storage device 10 according to the first embodiment checks the data format and stores only the data in the correct format in the storage 32. Therefore, it is possible to prevent the storage 32 from storing data that is partially lost due to a communication error or the like and cannot be used for analysis. Thereby, unnecessary data can be prevented from being stored in the storage 32.

  Further, the data storage device 10 according to the first embodiment stores only the data collected from the designated sensor 31 in the storage 32. Therefore, it is possible to prevent duplicate data from being stored in the storage 32 when a plurality of similar sensors 31 are operated for backup.

  *** Other configurations ***

<Modification 1>
The required amount may increase during the update period, and the extraction interval may be re-specified. In this case, if the processing described in the first embodiment is simply continued, the amount of data stored in the storage 32 during the update period is not the re-specified required amount.
This will be specifically described with reference to FIG. Assume that the required amount increases from 50 MB to 100 MB at the midpoint of the update period. In this case, in the first half of the update period, data is extracted at a pace at which 50 MB of data is stored in the storage 32 at the end of the update period. Therefore, 25 MB of data is stored in the storage 32 in the first half of the update period. In the second half of the update period, data is extracted at twice the pace of the first half. Therefore, 50 MB of data is stored in the storage 32 in the second half of the update period. As a result, 75 MB of data is stored in the storage 32 at the end of the update period.

In this case, during the update period, the amount of data necessary for data analysis is not accumulated in the storage 32. Therefore, as a first modification, when the required amount increases in the middle of the update period, the storage unit 25 increases the required amount for the past period from the start of the update period to the time when the required amount is specified again. The data stored in the storage 32 is interpolated and inserted at equal intervals so that the number of cases extracted by the extraction interval specified in step (b) is obtained. As a result, the amount of data stored in the storage 32 for the past period is increased.
Specifically, the storage unit 25 arranges data in the past period stored in the storage 32 in time series for each collection source sensor 31, and specifies values between the data by processing such as spline interpolation. Then, the storage unit 25 extracts data for the shortage and stores it in the storage 32 as separate data.
This will be specifically described with reference to FIG. In FIG. 12, black dots indicate data stored in the storage 32, and white dots indicate points interpolated. Assume that the required amount increases from 50 MB to 100 MB at the midpoint of the update period. In this case, it is necessary to double the amount of data stored in the storage 32 in the first half of the update period. Therefore, the storage unit 25 specifies values between data by arranging the data stored in the storage 32 in the first half of the update period in time series for each collection source sensor 31 and performing interpolation. Here, when i is an integer of 1 or more, the value of the interpolation point i + 0.5 is an average value of two adjacent values i and i + 1. Then, data indicating the specified value is stored in the storage 32. As a result, the data amount of the first half of the update period is changed from 25 MB to 50 MB. Therefore, in the second half of the update period, if data is extracted at a rate twice that of the first half and 50 MB of data is extracted, 100 MB of data is stored in the storage 32 at the end of the update period.

Note that if the required amount is re-specified during the update period, the extraction interval may be specified so that the required amount of data is stored in the storage 32 in the remaining period of the update period. In other words, as described above, if the required amount increases from 50 MB to 100 MB at the midpoint of the update period, only 25 MB of data is stored in the first half of the update period, so 75 MB of data is stored in the second half. It is also conceivable to specify the extraction interval as described above.
However, if this is done, the data in the latter half will increase in the storage 32, and the data will be biased. Therefore, there is a possibility that the analysis accuracy does not increase.

  Here, the case where the required amount has increased during the update period has been described. The required amount may decrease during the renewal period. In this case, if the processing described in the first embodiment is simply continued, the amount of data stored in the storage 32 increases. However, the amount of data required for analysis is ensured only by increasing the amount of data a little. Therefore, the process described in the first embodiment may be simply continued. Further, the amount of data stored in the storage 32 may be reduced by thinning out the data of the update period stored in the storage 32 at regular intervals.

<Modification 2>
In the first embodiment, data having the correct format is stored in the storage 32. However, there are cases where there are errors in the format of many data depending on the communication status. As a result, the required amount of data may not be stored in the storage 32.
Therefore, as a second modification, when there is an error in the data format equal to or higher than the reference rate, the format determination unit 23 performs interpolation using data collected before and after the same sensor 31 to adjust the data format. As a specific example, if the item of the data temperature is missing, the format determination unit 23 is missing from the temperature of the data collected before the data and the temperature of the data collected after the data. Calculate the value of the measured temperature item and arrange the data format. The storage unit 25 stores the data in the data format in the temporary storage unit 124 as a target to be stored in the storage 32.

<Modification 3>
In the first embodiment, data collected from the sensor 31 having the priority order 1 is stored in the storage 32. However, the sensor 31 with the priority order 1 may fail.
Therefore, as a third modification, when the receiving unit 22 does not receive data transmitted from a certain sensor 31 for a certain period, it is considered that the sensor 31 has failed. Then, the priority order of the sensor 31 is excluded, and the priority order of other sensors 31 belonging to the same sensor group as the sensor 31 is increased. As a specific example, if the priority order of the sensor 31 that is regarded as having failed is 3, the priority order of the other sensors 31 whose priority order is 4 or less is increased by one.
As a result, even if the sensor 31 with the priority order 1 fails, data of other sensors 31 belonging to the same sensor group is stored in the storage 32.

<Modification 4>
In the first embodiment, when the result accuracy is not within the allowable range in step S13 of FIG. 8, the target amount is increased if the result accuracy is lower than the analysis accuracy, and the result accuracy is higher than the analysis accuracy. In some cases the target amount was reduced. This is because it was assumed that the more data to be input, the higher the analysis accuracy. However, the analysis accuracy may not increase as the input data increases.
Therefore, as a fourth modification, the target amount is increased or decreased according to the past change in the target amount and the result accuracy.

This will be specifically described with reference to FIG.
In step S131, the required amount specifying unit 21 determines whether the result accuracy is lower than the analysis accuracy. If the result accuracy is lower than the analysis accuracy, the required amount specifying unit 21 proceeds to step S132, and otherwise proceeds to step S135.

In step S132, the required amount specifying unit 21 determines whether or not the target amount has been increased in step S11 executed most recently compared to step S11 executed previously. The necessary amount specifying unit 21 advances the process to step S133 if it is increased, and advances the process to step S134 if not.
In step S133, the required amount specifying unit 21 determines whether or not the result accuracy in the most recently executed step S12 is higher than the result accuracy in the previously executed step S12. The necessary amount specifying unit 21 increases the target amount when it is high, and decreases the target amount otherwise.
In step S134, the necessary amount specifying unit 21 determines whether or not the result accuracy in the most recently executed step S12 is higher than the result accuracy in the previously executed step S12. The required amount specifying unit 21 decreases the target amount when it is high, and increases the target amount otherwise.

In step S135, the necessary amount specifying unit 21 determines whether or not the target amount has been increased in step S11 executed most recently compared to step S11 executed previously. The necessary amount specifying unit 21 advances the process to step S136 if it is increased, and advances the process to step S137 if not.
In step S136, the necessary amount specifying unit 21 determines whether or not the result accuracy in the most recently executed step S12 is higher than the result accuracy in the previously executed step S12. The required amount specifying unit 21 decreases the target amount when it is high, and increases the target amount otherwise.
In step S137, the necessary amount specifying unit 21 determines whether or not the result accuracy in the most recently executed step S12 is higher than the result accuracy in the previously executed step S12. The necessary amount specifying unit 21 increases the target amount when it is high, and decreases the target amount otherwise.

<Modification 5>
In the first embodiment, the function of each unit of the data storage device 10 is realized by software. However, as a fifth modification, the function of each unit of the data storage device 10 may be realized by hardware. The fifth modification will be described with respect to differences from the first embodiment.

With reference to FIG. 14, the structure of the data storage apparatus 10 which concerns on the modification 5 is demonstrated.
When the functions of the respective units are realized by hardware, the data storage device 10 includes a processing circuit 15 instead of the processor 11 and the storage device 12. The processing circuit 15 is a dedicated electronic circuit that realizes the functions of the respective units of the data storage device 10 and the functions of the storage device 12.

The processing circuit 15 is assumed to be a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a logic IC, a GA (Gate Array), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field-Programmable Gate Array). Is done.
The function of each part may be realized by one processing circuit 15, or the function of each part may be realized by being distributed to a plurality of processing circuits 15.

<Modification 6>
As a sixth modification, some functions may be realized by hardware and other functions may be realized by software. That is, some of the functions of the data storage device 10 may be realized by hardware, and other functions may be realized by software.

  The processor 11, the storage device 12, and the processing circuit 15 are collectively referred to as “processing circuitry”. That is, the function of each part is realized by a processing circuit.

  10 data storage device, 11 processor, 12 storage device, 121 collection condition storage unit, 122 analysis accuracy storage unit, 123 sensor storage unit, 124 temporary storage unit, 13 input interface, 14 output interface, 15 processing circuit, 21 required amount specification Unit, 22 receiving unit, 23 format determining unit, 24 sensor determining unit, 25 storage unit, 31 sensor, 32 storage, 33 learning data.

Claims (9)

A receiver for receiving data from the sensor;
A required quantity identifying unit that identifies the required quantity, which is the amount of data necessary for analysis,
Specify the data extraction interval from the required amount specified by the required amount specifying unit and the received amount which is the data amount of data received in the reference period by the receiving unit, and from the data received by the receiving unit, A data storage device comprising a storage unit that extracts data at a specified extraction interval and stores the data in a storage. The data storage device according to claim 1, wherein the required amount specifying unit specifies, as the required amount, a data amount that provides a specified analysis accuracy when an analysis is performed using data stored in the storage as an input. . The required amount specifying unit specifies a data amount for which a specified analysis accuracy is obtained for each analysis method, and specifies the largest data amount as the required amount among the data amounts specified for each analysis method. 2. The data storage device according to 2. The required amount specifying unit calculates the required amount when an update period has elapsed, when an analysis method is added, when an analysis method is deleted, and when a specified analysis accuracy is changed. The data storage device according to claim 3, wherein the data storage device is specified again. When the required amount increases during the update period, the storage unit is configured to increase the required amount for a past period from the start of the update period to the time when the required amount is specified again. The data storage device according to claim 4, wherein data obtained by interpolating data stored in a storage so as to be the number of cases extracted by the specified extraction interval is stored. The data storage device further includes:
A format determination unit for determining whether the format of the data is correct;
The data storage device according to any one of claims 1 to 5, wherein the storage unit extracts data at an extraction interval from data determined to be correct by the format determination unit and stores the extracted data in a storage. The data includes the sensor ID of the source sensor,
The data storage device further includes:
A sensor storage unit that stores one or more sensor IDs that are sensor identification information;
A sensor determination unit that determines whether or not the sensor ID stored in the sensor storage unit includes a sensor ID included in the data received by the reception unit;
The data storage device according to any one of claims 1 to 6, wherein the storage unit extracts data at an extraction interval from data determined to be in the sensor determination unit and stores the data in a storage. The storage unit stores data received by the receiving unit in a temporary storage unit, and when a reference amount of data is accumulated in the temporary storage unit, the data is extracted from the temporary storage unit at the extraction interval. The data storage device according to claim 1, wherein the data storage device is stored in a storage. A receiving process for receiving data from the sensor;
A required amount specifying process for specifying a required amount, which is the amount of data necessary for analysis,
Specify the data extraction interval from the required amount specified by the required amount specifying process and the received amount that is the data amount of data received in the reference period by the receiving process, and from the data received by the receiving process, A data storage program for causing a computer to execute a storage process of extracting data at a specified extraction interval and storing it in a storage.

Images