WO2024014035A1 - Data prediction support method and data prediction system - Google Patents

Abstract

This system extracts a plurality of data set candidates from a plurality of pieces of observation data having periodicity. The data set candidate is a set of two or more pieces of observation data for a target period. The system calculates vector data for each data set candidate, the vector data including, as elements, the magnitudes of a plurality of different frequency components of the observation data for each piece of the observation data in the dataset candidate, the system classifies two or more pieces of the vector data into one or more data groups on the basis of a distance between pieces of the vector data, and the system calculates the goodness of fit to a prediction model with the observation data serving as the input and the prediction data serving as the output, on the basis of the number of data pieces for each data group. The system outputs a data set candidate to be used for prediction using the prediction model, on the basis of the goodness of fit that is calculated for each data set candidate.

Description

Data prediction support method and data prediction system

The present invention generally relates to data prediction.

In the energy business field such as electric power business and gas business, the communication business field, and the transportation business field such as taxi and delivery businesses, the value of demand is calculated in order to operate equipment and allocate resources in accordance with consumer demand. Make predictions.

For example, in the field of electric power business, there is a physical constraint that the amount of electricity generated must always match the amount of electricity demanded. Since it is necessary to have sufficient generators on standby in advance, it is necessary to accurately predict the demand for electricity.

Power consumption trends are diversifying due to the spread of renewable energy and distributed power sources such as electric vehicles, and retail contract switching is becoming more frequent due to electricity liberalization. Therefore, changes in power consumption trends occur that cannot be captured by a collective demand forecast for the jurisdiction area, making highly accurate demand forecasting difficult. Techniques related to predicting power demand are disclosed in Patent Documents 1 and 2, for example.

The power demand forecasting system disclosed in Patent Document 1 classifies consumers into a plurality of groups based on performance data indicating the amount of power consumed by a plurality of consumers at each predetermined time, and predicts a portion of the demand from each of the groups. Houses are selectively extracted, and the total power demand, which is the sum of the power demands of the consumers, is predicted based on the actual data of the consumers.

The power demand prediction device disclosed in Patent Document 2 classifies power measurement data from a smart meter into groups with similar trends, and constructs a prediction model and calculates a predicted value for each group.

Japanese Patent Application Publication No. 2016-181060 Japanese Patent Application Publication No. 2017-182266

A dataset consisting of two or more observational data is extracted from multiple observational data, the two or more observational data in the dataset are classified into multiple groups, and the observational data (observed values) are classified into multiple groups. It is conceivable to construct a prediction model that uses time-series data (time-series data) as input and prediction data (time-series data of predicted values) as output.

According to Patent Document 1, a group can be a group of consumers having the same attributes such as location, contract type, contract power, and industry. However, even among consumers with the same attributes, electricity consumption trends may differ depending on whether renewable energy or electric vehicles have been introduced or differences in the behavioral characteristics of the consumers. When observed data with different trends are mixed in the same group, the accuracy of the prediction model decreases.

According to Patent Document 2, a group can be a group in which the observed data itself has a similar tendency. However, in this case, depending on the group, the number of observation data is insufficient, and therefore prediction cannot be made with sufficient accuracy.

The purpose of the present invention was made in consideration of the above points, and is to extract a data set useful for constructing a prediction model with high prediction accuracy.

A data prediction system extracts multiple dataset candidates from a data source that includes multiple observational data, each with periodicity. Each of the plurality of data set candidates is a set of two or more observation data for the target period. In each data set candidate, the observation data is all or part of one of the plurality of observation data. For each of a plurality of dataset candidates, the data prediction system calculates, for each observational data in the dataset candidate, vector data whose elements are the magnitudes of multiple different frequency components of the observational data, and Vector data is classified into one or more data groups based on the distance between vector data, and based on the number of data in each data group, the goodness of fit to a prediction model that uses observed data as input and predicted data as output is calculated. calculate. The data prediction system outputs a dataset candidate used for prediction processing using a prediction model as a compatible dataset based on the degree of fitness calculated for each of the plurality of dataset candidates.

According to the present invention, it is possible to extract a dataset useful for constructing a prediction model with high prediction accuracy.

1 is a diagram showing a device configuration according to a first embodiment of a data prediction system. FIG. 2 is a diagram showing an apparatus configuration when the embodiment of FIG. 1 is implemented in an electric power supply and demand management system. 1 is a diagram showing the internal configuration of a data prediction system according to a first embodiment; FIG. FIG. 2 is a diagram showing a data flow according to the first embodiment of the data prediction system. FIG. 3 is a diagram showing a processing flow according to the first embodiment of the data prediction system. FIG. 3 is a diagram showing details of a matching extraction section. It is a figure which shows the processing flow of a suitability extraction part. FIG. 3 is a diagram illustrating an overview of a part of the processing of the compatibility extraction unit. FIG. 6 is a diagram illustrating the remaining outline of the process of the matching extraction unit. FIG. 3 is a diagram showing details of a trend classification section. FIG. 3 is a diagram showing details of a group-by-group prediction unit. FIG. 3 is a diagram showing details of the overall prediction unit. It is a figure showing an outline of an example of an effect. It is a figure which shows the data flow by the fourth embodiment of a trend classification part. FIG. 12 is a diagram showing a data flow according to a sixth embodiment of a matching extraction unit. FIG. 11 is a diagram showing a data flow according to a seventh embodiment of a matching extraction unit. FIG. 7 is a diagram showing a data flow according to an eighth embodiment of a matching extraction unit.

In the following description, an "interface device" may be one or more interface devices. The one or more interface devices may be at least one of the following:
- One or more I/O (Input/Output) interface devices. The I/O (Input/Output) interface device is an interface device for at least one of an I/O device and a remote display computer. The I/O interface device for the display computer may be a communication interface device. The at least one I/O device may be a user interface device, eg, an input device such as a keyboard and pointing device, or an output device such as a display device.
- One or more communication interface devices. The one or more communication interface devices may be one or more communication interface devices of the same type (for example, one or more NICs (Network Interface Cards)), or two or more communication interface devices of different types (for example, one or more NICs). It may also be an HBA (Host Bus Adapter).

Additionally, in the following description, "memory" refers to one or more memory devices, typically a main storage device. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

Also, in the following description, "persistent storage" refers to one or more persistent storage devices. The persistent storage device is typically a nonvolatile storage device (for example, an auxiliary storage device), and specifically, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

Furthermore, in the following description, a "storage device" may be at least a memory or a permanent storage device.

Also, in the following description, a "processor" refers to one or more processor devices. The at least one processor device is typically a microprocessor device such as a CPU (Central Processing Unit), but may be another type of processor device such as a GPU (Graphics Processing Unit). At least one processor device may be single-core or multi-core. The at least one processor device may be a processor core. At least one processor device may be a broadly defined processor device such as a hardware circuit (for example, a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC)) that performs part or all of the processing.

In addition, in the following explanation, functions may be explained using the expression "yyy part", but functions may be realized by one or more computer programs being executed by a processor, or one or more computer programs may be executed by a processor. It may be realized by the above hardware circuits (for example, FPGA or ASIC), or by a combination thereof. When a function is realized by a program being executed by a processor, the specified processing is performed using a storage device and/or an interface device as appropriate, so the function may be implemented as at least a part of the processor. good. A process described using a function as a subject may be a process performed by a processor or a device having the processor. The program may be installed from program source. The program source may be, for example, a program distribution computer or a computer-readable recording medium (for example, a non-temporary recording medium). The description of each function is an example, and a plurality of functions may be combined into one function, or one function may be divided into a plurality of functions.

Hereinafter, some embodiments of the present invention will be described in detail with reference to the drawings.
(1) First embodiment (1-1) Configuration of data prediction system according to this embodiment

FIG. 1 shows the device configuration of the entire data prediction system according to this embodiment.

For example, when applied to the electric power business field, the data processing system 1 analyzes observed values of past electric power demand and calculates predicted values of electric power demand and transaction prices for any target period. Based on the predicted values, it becomes possible to manage the supply and demand of electricity, including the formulation and execution of generator operation plans, the formulation and execution of electricity procurement transaction plans from other electric utilities, and the formation planning of power transmission and distribution facilities, etc. .

The data processing system 1 includes a user 2, a data prediction system 3, an observation data provider 4, an observation data storage device 5, an external data provider 6, an external data storage device 7, a supply and demand management facility 8, a control device 9, and a communication device. It consists of a route 10. The communication path 10 may be, for example, a communication network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and is a communication path that connects various devices and terminals that make up the data processing system 1 so that they can communicate with each other. The control device 9 may use the prediction data calculated by the data prediction system 3 to create and execute plans regarding the operation and control of equipment such as generators and communication stations, market transactions, equipment formation, and the like.

As a specific example, when the data processing system 1 is implemented in a system that performs power supply and demand management, the device configuration shown in FIG. 2 can be considered.

The user 2 corresponds to the operator of the supply and demand management equipment 8. The observation data provider 4 corresponds to a consumer, and the observation data storage device 5 corresponds to a power measuring device. The external data provider 6 corresponds to a public data provider, and the external data storage device 7 corresponds to a public data storage device. In addition, the supply and demand management equipment 8 includes a generator, power storage equipment, a switch, etc., and the control device 9 includes, for example, a market transaction management device, a generator control device, a power storage equipment control device, and a switch. at least one of the controllers.

Note that the public data may be, for example, data including at least one of the following.
- Weather data such as temperature, humidity, solar radiation, wind speed, and atmospheric pressure.
・Calendar date data such as year, month, day, day of the week, and flag values indicating the type of arbitrarily set day.
・Data indicating the occurrence of sudden events such as typhoons and events.
- Data on industrial dynamics (for example, the number of energy consumers, attributes indicating the type of energy consumers such as factories, offices, and households, the industry of energy consumers, and the number of production and sales amount for each industry and company).
・Data showing the topography or climate characteristics of each region.
-Data such as the number of communication terminals connected to a communication base station.
・Past observation data itself.

The observation data storage device 5 is an example of a data source and stores observation data groups. The observation data group is composed of a plurality of observation data each having periodicity. The observed data is data that serves as input data for performing data prediction, and may be time-series data of past observed values.

The observation data may be, for example, data including at least one of the following. Furthermore, observation data may be data for each measuring instrument (for example, data for each smart meter), or data as a total of multiple measuring instruments (for example, observation data for all smart meters belonging to a predetermined area). (observed data as an average value) may also be used.
- Energy consumption data such as electricity, gas, water, etc.
- Energy production data such as solar power generation and wind power generation.
・Data on energy transaction prices traded on wholesale exchanges.
・Communication amount data measured at communication base stations, etc.
・Historical data of location information of moving objects such as cars.

The observation data storage device 5 searches for and/or transmits observation data in response to data acquisition requests from other devices.

The external data storage device 7 is input data for performing data prediction, and stores external data linked to one or more observation data. The external data may be linked in advance, or may be linked dynamically by a processing unit such as the group classification unit 3522. Furthermore, the external data may be data representing past values or data representing future values.

The external data may be, for example, data including at least one of the following. The external data may be time-series data of values, similar to observation data.
- Weather data such as temperature, humidity, solar radiation, wind speed, and atmospheric pressure.
・Calendar date data such as year, month, day, day of the week, and flag values indicating the type of arbitrarily set day.
・Data indicating the occurrence of sudden events such as typhoons and events.
- Data on industrial dynamics (for example, the number of energy consumers, attributes indicating the type of energy consumers such as factories, offices, and households, the industry of energy consumers, and the number of production and sales amount for each industry and company).
・Data showing location information and regional topographic or climate characteristics.
-Data such as the number of communication terminals connected to a communication base station.
・Past observation data itself.

The external data storage device 7 searches for and/or transmits external data in response to data acquisition requests from other devices.

As described above, the data processing system 1 includes the data prediction system 3 and the power control system (for example, the supply and demand management equipment 8) that controls at least one of the power generation equipment and the power storage equipment. The data prediction system 3 outputs predicted data by performing a prediction process using a compatible data set, which will be described later. The power control system receives the prediction data, uses the prediction data to create at least one plan for power generation and storage, and controls at least one of the power generation equipment and the power storage equipment based on the created plan. . As described later, the prediction accuracy of the data prediction system 3 is high, so the created plan is a plan suitable for control. Therefore, suitable power generation and/or storage is realized, and therefore, suitable power supply and demand control is achieved. can be expected.
(1-2) Device internal configuration

FIG. 3 shows the device configuration of the data prediction system 3, observed data storage device 5, and external data storage device 7 included in the data processing system 1.

The data prediction system 3 includes an input device 32, an output device 33, a communication device 34, a storage device 35, and a CPU (Central Processing Unit) 31 connected thereto. The data prediction system 3 is an information processing system such as a personal computer, a server computer, or a handheld computer. The data prediction system 3 may be such a physical computer system (one or more physical computers) or a logical computer system based on a physical computer system (for example, a system as a cloud computing service). ) but that's fine. The input device 32 and the output device 33 may be omitted. The communication device 34 is an example of an interface device, and the CPU 31 is an example of a processor.

The input device 32 is composed of, for example, a keyboard or a mouse, and the output device 33 is composed of, for example, a display or a printer. Further, the communication device 34 is configured to include, for example, a NIC (Network Interface Card) for connecting to a wireless LAN or a wired LAN. Further, the storage device 35 is a storage medium such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The output results and intermediate results of each processing unit may be outputted as appropriate via the output device 33.

The storage device 35 stores one or more computer programs for realizing functions such as a matching extraction section 351, a trend classification section 352, a group prediction section 353, and an overall prediction section 354. When these computer programs are executed by the CPU 31, these functions are realized.

Furthermore, the storage device 35 has a storage area 355 for storing a matching data set 355A, and a storage area 356 for storing overall prediction data 356A. Note that the storage areas 355 and 356 may be a single storage area.

The compatible data set 355A may be database information or text information used to calculate the overall prediction data 356A, and may be a data set (two or more observation data) extracted from the observation data group 521A (a plurality of observation data). ).

The overall prediction data 356A is generated as follows. That is, two or more pieces of observed data included in the compatible data set 355A are classified into one or more groups having similar transition patterns. A prediction model is constructed for each group, and prediction data is generated using the prediction model. The overall prediction data generated from the prediction data of each group is the overall prediction data 356A. The overall prediction data 356A may be database information, text information, or image information that is a graph of calculated values.

The observation data storage device 5 includes at least a communication device 51, a storage device 52, and a CPU 53 connected thereto. The storage device 52 has a storage area 521 for storing observation data group 521A.

The external data storage device 7 includes at least a communication device 71, a storage device 72, and a CPU 73 connected thereto. The storage device 72 has a storage area 721 for storing an external data group 721A. The external data group 721A is composed of a plurality of external data.

The data prediction system 3 performs data prediction using observation data and external data acquired from the observation data storage device 5 and the external data storage device 7.

The matching extraction unit 351 extracts a plurality of data set candidates from the observation data group 521A, which is a collection of periodic observation data. A dataset candidate is a collection of two or more observed data. For each extracted data set candidate, the matching extraction unit 351 converts each observed data included in the data set candidate into data representing the magnitudes of a plurality of different frequency components, and then performs one-on-one matching with each frequency component. map the data to a multidimensional feature space with corresponding dimensions. The matching extraction unit 351 classifies data into a plurality of groups by grouping data that are close to each other in the feature space. The suitability extraction unit 351 calculates, for each dataset candidate, an index representing variation such as variance for the number of data included in each group, and uses the candidate with the minimum index from among the plurality of dataset candidates for data prediction. Output as a conforming data set.
(1-3) Processing and data flow of data prediction system according to this embodiment

The data flow and processing flow of the data prediction system 3 in this embodiment will be explained using FIGS. 4 and 5.

The data prediction system 3 in this embodiment acquires an observation data group 521A and an external data group 721A from the observation data storage device 5 and the external data storage device 7, respectively.

The observed data group 521A is input to the matching extraction unit 351. The compatible extraction unit 351 extracts a plurality of data set candidates from the input observation data group 521A, and outputs one of the plurality of data set candidates as the compatible data set 355A (S301).

The matching data set 355A is input to the trend classification unit 352 together with the external data group 721A. The trend classification unit 352 links the observed data included in the input compatible data set 355A with the external data included in the external data group 721A. The trend classification unit 352 converts each observed data (typically time series data) included in the adapted data set 355A into a multidimensional vector whose elements are the magnitudes of frequency components, and classifies data with close distances between the vectors. By grouping them together, the data included in the compatible data set 355A and the external data group 721A are classified into one or more groups (S302). A group includes observation data, and external data is linked to the observation data. The "distance" may be a general distance measure that satisfies the axiom of distance, such as Euclidean distance, Mahalanobis distance, Manhattan distance, Chebyshev distance, or Minkowski distance, or a degree of similarity such as cosine similarity. In addition, the grouping process can be performed using hierarchical clustering methods such as the Ward method, single link method, complete link method, or centroid method, or clustering methods as neighborhood optimal methods such as k-means, EM algorithm, or spectral clustering. Alternatively, processing may be performed using a clustering method for optimal identification boundary such as unsupervised SVM (Support Vector Machine), VQ algorithm, or SOM (Self-Organizing Maps).

The plurality of groups obtained by the trend classification section 352 are input to the group-by-group prediction section 353. The group prediction unit 353 constructs a prediction model for each group created by the trend classification unit 352, and calculates prediction data for each group (for example, time series data of predicted values) using each prediction model. (S303).

The group-specific prediction data calculated by the group-specific prediction unit 353 is input to the overall prediction unit 354. The overall prediction unit 354 calculates and outputs the overall prediction data using the group-specific prediction data calculated by the group-specific prediction unit 353 and the residual observed data not extracted by the adaptive extraction unit 351 ( S304). When the observed data is data for each customer, the overall prediction data is prediction data for the entire area to which the customer belongs.

The overall prediction data 356A calculated by the overall prediction unit 354 is input to the supply and demand management equipment 8. The supply and demand management equipment 8 uses the input overall forecast data 356A to control the generator, power storage equipment, switch, etc.

The details of each part will be explained below.
(1-4) Details of each component (1-4-1) Compatibility extraction unit 351

An embodiment of the matching extraction unit 351 will be described using FIGS. 6, 7, and 8.

FIG. 6 shows the data flow inside the matching extraction unit 351.

The suitability extraction unit 351 includes a candidate generation unit 3511, an index calculation unit 3513, a group classification unit 3514, and a candidate selection unit 3515. The storage area 3512 is an area in the storage device 35, and the data set candidate 3512A is stored as an intermediate output.

The candidate generation unit 3511 extracts a plurality of dataset candidates 3512A from the observed data group 521A, and stores each dataset candidate 3512A in the storage area 3512. Data set candidates may be extracted at random, or by extracting some observed data from each of a plurality of data classifications. Data classification may be any of the following.
- A collection of observation data with the same label (the label may depend on the attribute (for example, factory, commercial facility, home, etc.) to which the measuring instrument from which the observation data was obtained belongs).
- A collection of observed data with the same or similar statistics (statistics may be the average value, maximum value, minimum value, variance, etc. of the data transition represented by the observed data).

Furthermore, the number of observation data included in each dataset candidate may be arbitrarily determined by the user 2.

For each data set candidate 3512A, the index calculation unit 3513 converts the observed data into multidimensional vector data in which each element has the magnitude of a frequency component. The conversion process may be, for example, a process of normalizing a value representing a data transition, a Fourier transform or a wavelet transform process of a value representing a data transition, or both. Any frequency component may be designated in advance as the frequency component to be calculated by the conversion process. The specified frequency component may be a plurality of different periodic components, and the periodic component may be selected from yearly, monthly, weekly, daily, 1 hour, and 0.5 hour period components. As examples, characteristic components tend to appear in the electric power field: one year (365 days), half a year (180 days), March (90 days), January (30 days), one week (7 days), one year Two or more periodic components of a day, half a day (12 hours), 6 hours, 1 hour and 30 minutes (0.5 hours) may be employed. In addition to the method of determining frequency components in advance, components whose values vary widely among vector data may be mechanically adopted as frequency components.

For each data set candidate 3512A, the group classification unit 3514 collects data whose vector data distances are close after being converted by the index calculation unit 3513, so that the group classification unit 3514 can perform calculations for two or more observed data in the compatible data set 355A. The two or more vector data obtained are classified into one or more groups. The number of groups may be arbitrarily determined in advance, or may be a number that minimizes an information amount criterion such as AIC (Akaike Information Criterion) or BIC (Bayesian Information Criterion).

The candidate selection unit 3515 calculates, for each dataset candidate 3512A, an index representing variation such as the variance of the number of data in a group. The candidate selection unit 3515 outputs the dataset candidate 3512A with the smallest index among the plurality of dataset candidates 3512A as the compatible dataset 355A used for data prediction (prediction processing using a prediction model).

FIG. 7 shows the processing flow of the matching extraction unit 351. This processing flow corresponds to the internal processing of S301 shown in FIG.

The matching extraction unit 351 repeats the processing from S3012 to S3014 N times (S3011). Note that instead of repeating the processes from S3012 to S3014 N times, the matching extraction unit 351 may repeat each process of S3012, S3013, and S3014 N times. N is an integer of 2 or more.

First, the matching extraction unit 351 extracts the dataset candidate 3512A from the observed data group 521A (S3012).

Next, the index calculation unit 3513 converts each observation data included in the dataset candidate 3512A extracted in S3012 into multidimensional vector data (periodicity index) having each element as a magnitude of a frequency component (S3013 ).

Next, the group classification unit 3514 classifies two or more vector data regarding two or more observed data in the dataset candidate 3512A into a plurality of groups by grouping together vector data that are close to each other (S3014). A group (data group) is a group of vector data in which the distance between vector data is equal to or less than a threshold value. The threshold may be a distance threshold from representative vector data determined based on two or more vector data, or a relative distance threshold between calculated vector data.

Finally, for each of the plurality of (N) dataset candidates, the candidate selection unit 3515 calculates an index representing variation, such as variance, with respect to the number of data in the group for the dataset candidate, and selects the dataset candidate with the smallest index. , and output as a compatible data set 355A (S3015).

With the above, the extraction process by the matching extraction unit 351 is completed.

The processing contents of the matching extraction unit 351 will be explained in more detail using FIGS. 8A and 8B. As an example, the observed data in the observed data group 521A is time series data of power consumption.

First, the candidate generation unit 3511 extracts N data set candidates from the observed data group 521A. Each data set candidate includes two or more observation data 521B1 (power consumption transition data). Among the N dataset candidates, the number of observed data in the dataset candidates may be the same or different. Further, each of the N data set candidates is composed of two or more pieces of observation data for the target period. The "target period" may be a past period such as the past year or half a year from the present, and the past period may be a period corresponding to the forecast target period (future period) (for example, the forecast target period is from January to December In the case of months, the relevant past period can also be from January to December).

Next, the index calculation unit 3513 converts each observation data 521B1 into vector data 521B2 whose elements are the magnitudes of frequency components by performing Fourier transform on each observation data 521B1. Transforms other than Fourier transform, such as wavelet transform, may be employed. In FIG. 8A, for convenience, observation data 521B1 is converted into two-dimensional vector data of a daily periodic component and a semi-daily periodic component, but instead, long periodic components such as a yearly period and a half-yearly period, etc. , 1-hour period, 30-minute period, etc., and may be converted into more multidimensional vector data.

Next, the group classification unit 3514 groups the vector data converted into the magnitude of the frequency component into data with a small distance measure. As an example, groups 1 to 3 classified by group boundaries 521B3 indicated by dotted lines are created. Note that, to simplify the explanation, the number of groups is the same for dataset candidates 1 and 2, but the number of groups may be different for each dataset candidate.

Next, the candidate selection unit 3515 totals the number of data for each group for each dataset candidate (see reference numeral 521B4). The candidate selection unit 3515 calculates the variance of the number of data for each group for each dataset candidate from the total number of data for each group (see reference numeral 521B5). Finally, the candidate selection unit 3515 outputs the dataset candidate 2 with the minimum variance as the compatible dataset 355A.
(1-4-2) Trend classification unit 352

The trend classification unit 352 receives the compatible data set 355A and the external data group 721A as input, and links the observed data included in the compatible data set 355A with the external data included in the external data group 721A. The trend classification unit 352 classifies two or more pieces of observation data included in the fit data set 355A into one or more types based on the degree of similarity between the feature data (vector data) of the observation data included in the fit data set 355A. Categorize into groups.

The processing contents of the trend classification unit 352 will be explained in more detail using FIG. 9.

The trend classification section 352 includes a feature amount conversion section 3521 and a group classification section 3522. The storage area 3523 is one area of the storage device 35, and the intermediate output observation data with group information 3523A and the external data with group information 3523B are stored.

The feature value conversion unit 3521 receives the compatible data set 355A as input, and converts each observed data included in the compatible data set 355A into feature data. The feature amount data may be the multidimensional vector data (periodicity index) described above.

The group classification unit 3522 receives vector data for each observed data in the compatible data set 355A (feature data output from the feature conversion unit 3521) and the external data group 721A as input. The group classification unit 3522 links vector data with external data that is linked to observation data corresponding to the vector data. Next, the group classification unit 3522 groups the combinations in which the distance between each vector data is close, so that two or more vector data regarding two or more observed data in the compatible data set 355A are grouped into one or more vector data. Classify into groups. In a group (data group), the distance, classification into groups (grouping process), and number of groups may be as described above (as described for the matching extraction unit 351).

Finally, the group classification unit 3522 stores the observation data with group information 3523A and the external data with group information 3523B in the storage area 3523, and outputs them to the group-by-group prediction unit 353. The observation data with group information 3523A includes each observation data in the compatible data set 355A, and each observation data is associated with group information indicating the group to which the feature amount data of the observation data is classified. The external data with group information 3523B includes external data linked to the observed data in the compatible data set 355A, and represents the group to which the feature data of the observed data to which the external data is linked is classified. Group information is linked.
(1-4-3) Group-specific prediction unit 353

The group-by-group prediction unit 353 receives as input the observed data with group information 3523A, the external data with group information 3523B, and the external data group 721A output from the trend classification unit 352, and constructs a prediction model and makes predictions for each group. Calculate forecast data for the target period.

The processing contents of the group-by-group prediction unit 353 will be explained in more detail using FIG. 10.

The group prediction unit 353 is composed of a construction unit 3531 and a group calculation unit 3532. The storage area 3533 is an area of the storage device 35, and the group-specific prediction data 3533A is stored in the storage area 3533.

The construction unit 3531 receives the observation data with group information 3523A and the external data with group information 3523B as input, and constructs a prediction model for each group. For each group, the prediction model uses regression, classification, clustering, etc., using data belonging to the group with group information attached observation data 3523A as an objective variable, and using data belonging to the group in the group information attached external data 3523B as an explanatory variable. A model or a model that combines them is fine. In constructing the prediction model, a value obtained by taking an average value or a median value from data belonging to a certain group of the observation data with group information 3523A may be adopted. Further, different models may be adopted for each group.

The group calculation unit 3532 receives the group-by-group prediction model output from the construction unit 3531 and the external data group 721A as input, and calculates prediction data for the prediction target period of each group. The external data group 721A may include actual values and forecast values in the prediction target period of data associated with observation data used to construct the prediction model of the group for which prediction data is to be calculated.

Finally, the group calculation unit 3532 stores the prediction data calculated for each group in the storage area 3533 as group-specific prediction data 3533A, and outputs it to the overall prediction unit 354.
(1-4-4) Overall prediction unit 354

The overall prediction unit 354 receives as input the group-specific prediction data 3533A, the observed data group 521A, the adaptive data set 355A, and the external data group 721A output from the group-specific prediction unit 353, and obtains the overall prediction data (e.g., adaptive Forecast data for the entire area to which the customer corresponding to the observation data belonging to the data set 355A belongs is calculated.

The processing contents of the overall prediction unit 354 will be explained in more detail using FIG. 11.

The overall prediction unit 354 includes a residual prediction unit 3541 and an overall calculation unit 3542.

The residual prediction unit 3541 first creates residual data by taking the sum of observed data other than the compatible data set 355A among the observed data group 521A. The residual data may be a set of observed data other than the compatible data set 355A among the observed data group 521A. Next, the residual prediction unit 3541 constructs a prediction model using the residual data as an objective variable and the data associated with the residual data among the data included in the external data group 721A as an explanatory variable. The predictive model may be a regression, classification, clustering, or other model, or a combination of these models. In constructing the prediction model, a value obtained by taking an average value or a median value from the residual data may be adopted. Finally, the residual prediction unit 3541 inputs the actual values and predicted values for the prediction period of the external data group 721A used as explanatory variables into the prediction model, calculates the predicted value of the residual data, and outputs it to the overall calculation unit 3542. do.

The overall calculation unit 3542 uses the group-specific prediction data 3533A and the predicted value of the residual data output from the residual prediction unit 3541 to calculate the predicted value for the entire customer. Specifically, for example, the overall calculation unit 3542 takes the sum of each value of the group-by-group prediction data 3533A and the predicted value of the residual data, and outputs the sum as the overall prediction data 356A.

With the above processing, the first calculation process in this embodiment is completed, and the calculation process of the data prediction system 3 in this embodiment is completed.
(1-5) Description of an example of the effect of this embodiment

With reference to FIG. 12, the effects of the data prediction system 3 according to this embodiment will be explained.

FIG. 12 shows an overview of an example of the effect when prediction data for the prediction target period (for example, observed data to be predicted in the prediction target period) is calculated using the compatible data set 355A extracted by the compatible extraction unit 351. It is a diagram. As an example, we will use observation data indicating power consumption collected by smart meters. As shown in legend 24, in the graph, the dotted line represents observed data (the time series of observed values in the past period corresponding to the forecast period), and the solid line represents the predicted data (time series of observed values predicted for the forecast period). series), and the broken lines represent the observed data to be predicted (the time series of observed values actually observed for the prediction target period). The observed data indicated by the dotted line is the observed data included in the compatible data set 355A. The prediction data indicated by the solid line is data output as the group-based prediction data 3533A or the overall prediction data 356A. The black dots 25 simulate vector data (data converted from observed data) calculated by the index calculation unit 3513 and plotted on the feature space. The feature space is a coordinate system space with axes each having a plurality of different periodic components. Vector data is plotted as coordinates in the feature space in a simulated manner.

Case 21 is a case in which observed data is extracted so that the breakdown of attributes representing the type of factory, office, general household, etc. is uniform, and predicted data is calculated for each group classified by the same attribute. In this case, there are two types of attributes: factory and household, and data is classified into groups by attribute. In this case, data with different power consumption trends are mixed in group A that has the factory attribute. Differences in trends are caused by, for example, differences in the amount of solar power generation installed and differences in business types. As a result of data with different trends being mixed in group A, it is not possible to accurately predict demand in group A, and as a result, there is a discrepancy between the overall forecast data that is finally output and the observed data to be predicted.

In case 22, observation data is extracted so that the breakdown of attributes representing types such as factories, offices, and general households is uniform, and predicted data is calculated in groups of data with small distance scales in the feature space. It is a case. In this case, data with the same attributes but different trends are classified into different groups, but groups such as group B for which it is not possible to secure sufficient training data to build a predictive model (in other words, the number of data is extremely small) are classified into different groups. group) may occur. As a result, it becomes difficult to predict demand for this group, resulting in errors remaining in the overall forecast data.

Case 23 is a case in which observed data is extracted using the extraction method according to the present embodiment, and predicted data is calculated for each group of data with a small distance scale in the feature space. In this case, a data set is selected in which the variation (for example, variance) in the number of data for each group is small, so the number of data is likely to be equal between the groups. As a result, it is possible to prevent groups with a small amount of data from occurring and to make accurate demand predictions possible.

In other words, in this embodiment, we will address the technical issue of classifying data into groups with the same attribute and making predictions for each group. Specifically, we will address the technical issue of classifying data into groups with the same attribute and making predictions for each group. Specifically, we will address the technical issue of classifying data into groups with the same attribute and making predictions for each group. Therefore, it is possible to solve the problem that the prediction accuracy may decrease due to the above. This is because, for each data set candidate, vector data whose elements are the magnitudes of different frequency components is calculated for each observed data, and the data is classified into groups based on the distance between the vector data. This is because it is possible to avoid mixing data with different trends in observed values in a group.

In addition, in this embodiment, the technical problem of classifying data into groups with similar trend trends of observed values and making predictions for each group, specifically, due to the insufficient amount of data required for prediction. Any of the problems that could lead to a decrease in prediction accuracy can be solved in this embodiment. This is because multiple dataset candidates are extracted, and the compatible dataset 355A is the dataset with the smallest variation in the number of data in groups among the multiple dataset candidates. This is because it is a data set that is expected not to have an extremely insufficient amount of data.

Note that for each of the plurality of periodic components, the period may be any period (for example, a period longer than one year, one year, half a year, March, January, one week, one day, half a day, six hours, one hour, and 0.5 hours) is sufficient. Since this period is a period in which periodicity of observed values is expected, it is more expected that the adapted data set 355A is appropriate. Further, the frequency component and the time component may be used together. For example, when only frequency components are used, Fourier transform may be employed, and when frequency components and time components are used together, wavelet transform may be employed.
(2) Second embodiment (changing the processing frequency of the matching extraction unit 351)

In the first embodiment, the compatibility extraction unit 351 is executed every time prediction processing is performed to extract a new compatibility data set 355A, but the present invention is not limited to this. may be optional.

Specifically, the execution of the compatibility extraction unit 351 is triggered, for example, after a certain period of time arbitrarily set by the user 2, after a certain number of executions, or when a consumer stored in the observation data storage device 5 This may be considered as a case where an increase or decrease of more than a certain value occurs.

By making it optional whether or not the matching extraction unit 351 executes the extraction process, the effect of reducing the processing load can be obtained.
(3) Third embodiment (change in processing content of candidate selection unit 3515)

In the first embodiment, in the matching extraction unit 351, the candidate selection unit 3515 selects the matching data set 355A from the plurality of data set candidates 3512A. ) was used as the evaluation index, but the evaluation index is not limited to this, and an expanded evaluation index may be used. ).

Specifically, when evaluating one dataset candidate 3512A, the candidate selection unit 3515 uses the variance of the candidate as an evaluation index, and also uses the residual data (observed data group) of the dataset candidate 3512A as an evaluation index. 521A other than the relevant data set candidate 3512A) (that is, each observation data in the residual data may also be converted to the above-mentioned vector data and the vector data may be classified into one or more groups. ). The candidate selection unit 3515 sets the linear sum of the variance of the dataset candidate 3512A and the evaluation index of the residual data as the overall evaluation index, and sets the overall evaluation index as the evaluation index of the dataset candidate 3512A. good.

The residual data may be the sum of observed data other than the relevant data set candidate 3512A among the observed data group 521A, and may be data obtained by the same method as the residual prediction unit 3541. The candidate selection unit 3515 constructs a prediction model that predicts the residual data (a prediction model using the residual data as an objective variable) in the same manner as the residual prediction unit 3541, and calculates the training error and likelihood of the prediction model. , the training error or likelihood may be used as an evaluation index for the residual data.

As described above, the overall evaluation index may be the linear sum of the index representing the variation of the data set candidate 3512A and the evaluation index of the residual data. As the weight of the linear sum, an arbitrary value set in advance or a ratio of the number of data in the extracted observation data group and the unextracted observation data group can be used. Further, when using the likelihood of the prediction model as an evaluation index of residual data, the above-mentioned linear sum may be the reciprocal of the likelihood or the linear sum of the training error and the dispersion index.

Finally, the candidate selection unit 3515 outputs the dataset candidate with the minimum overall evaluation index value as the compatible dataset 355A.

By using the expanded evaluation index, it becomes possible to extract a suitable data set 355A that allows calculation of more accurate prediction data.
(4) Fourth embodiment (change in configuration of trend classification unit 352)

In the first embodiment, the number of groups in the trend classification unit 352 is arbitrarily determined in advance, but is not limited to this. The number of groups that maximizes the accuracy verified by

This will be explained in detail using FIG. 13. The trend classification section 352 includes a group structure determination section 3524 that is an extension of the group classification section 3522.

The group structure determining unit 3524 includes a group number candidate setting unit 35241, a group classification unit 35242, an accuracy verification unit 35243, and a group number determining unit 35244.

The group number candidate setting unit 35241 receives as input a value obtained by converting each observed data included in the compatible data set 355A into vector data having elements of different sizes of frequency components from the feature amount converting unit 3521, and Generate a value for the number of groups when classifying data into groups. The group number candidates are a set of natural numbers arbitrarily determined in advance, or a set of values that are increased by an arbitrarily determined number from the minimum value to the maximum group number determined based on the number of input data. Good as. Finally, the group number candidate setting unit 35241 outputs the vector data and the set of group number candidates to the group classification unit 35242.

The group classification unit 35242 receives as input the vector data and the set of group number candidates output from the group number candidate setting unit 35241, and the external data group 721A. Next, the group classification unit 35242 groups each vector data and the external data group 721A linked thereto into a predetermined number of groups based on the group number candidates by grouping combinations in which the distance between each vector data is close. Classify. Finally, the group classification unit 35242 outputs the group number candidates, the corresponding observation data with group information, and the external data with group information to the accuracy verification unit 35243.

The accuracy verification unit 35243 receives as input the group number candidates output from the group classification unit 35242, the corresponding observed data with group information, and the external data with group information, and constructs a prediction model and predicts values for each group number candidate. Perform calculations. The construction of the prediction model and the calculation of the prediction value may be performed in the same manner as the group prediction unit 353 and the overall prediction unit 354. Here, as the prediction target period, it is preferable to select a period as close as possible to the target period to be predicted by the data prediction system 3 and for which observation data is obtained as actual measured values. Next, the accuracy verification unit 35243 compares the predicted value calculated for each group number candidate with the observed data obtained as the actual value, and evaluates the prediction accuracy. The accuracy index may be calculated using an error measure such as absolute error. Finally, the observed data with group information, the external data with group information, and the accuracy index value for each group number candidate are output to the group number determining unit 35244.

The group number determining unit 35244 receives as input observation data with group information, external data with group information, and accuracy index value for each group number candidate from the accuracy evaluation unit 35234, and determines the number of groups that provides the best accuracy. Specifically, the error scale for each group number candidate is compared, and observation data with group information and external data with group information for the group number candidate with the minimum error are output.

By using the group structure determination unit 3524, it becomes possible to more accurately calculate the predicted value for the target period at the cost of increasing the processing load.
(5) Fifth embodiment (change in configuration of overall prediction unit 354)

In the first embodiment, the overall prediction unit 354 is configured to calculate the overall prediction data 356A from the group-specific prediction data 3533A and the calculation results of the residual prediction unit 3541, but the present invention is not limited to this. The overall prediction data 356A may be calculated using only the group prediction data 3533A without using the group prediction data 3533A.

In that case, the overall calculation unit 3542 calculates a coefficient obtained by dividing the total number of consumers by the number of consumers from which the observed data included in the compatible data set 355A is obtained, and adds the coefficient to the sum of the group-specific forecast data 3533A. By multiplying, the overall prediction data 356A is calculated.

By calculating the overall prediction data 356A without using the residual prediction unit 3541, the overall prediction data 356A can be obtained with a low processing load, although the prediction error increases to a certain extent.
(6) Sixth embodiment (change in configuration of matching extraction unit 351)

In the first embodiment, the number of data included in the compatible data set 355A can be arbitrarily determined by the user 2 in the compatible extraction unit 351, but this is not limited to this. The data size may be determined by considering the relationship between an index of accuracy obtained when using a compatible data set, a processing load when using a compatible data set of the data size, or both.

This will be explained in detail using FIG. 14. The suitability extraction section 351 may include a data size determination section 3516. When selecting a data set candidate to be output as a suitable data set, the suitability extraction unit 351 uses the data determined by the data size determination unit 3516 in addition to the variation index of the data set candidates or the evaluation index described in the third embodiment. Use the specified data size.

The data size determination unit 3516 determines the data size and prediction accuracy based on the data size of the compatible data set, the actual accuracy of the prediction result obtained as the overall prediction data 356A, and the actual processing load when calculating the predicted value. Model the relationship between processing load and/or both. Information on the target prediction accuracy, processing load, or both is received from the input device 32, and a data size that satisfies the prediction accuracy, processing load, or both is specified from the model (an example of a relational model). The data size determination unit 3516 outputs the specified data size to the candidate selection unit 3515, and the candidate selection unit 3515 narrows down the dataset candidates having the input data size and outputs the dataset candidates as a suitable dataset. Select.

Not limited to the above, the data size determination unit 3516 uses the evaluation index of each dataset candidate calculated by the candidate selection unit 3515 instead of the prediction accuracy record, and the data size determination unit 3516 uses the evaluation index of each dataset candidate calculated by the candidate selection unit 3515, and It may also be a process of narrowing down to size.

Furthermore, the data size determined by the data size determination unit 3516 may be output to the candidate generation unit 3511 instead of being output to the candidate selection unit 3515. In that case, the candidate generation unit 3511 generates a dataset candidate having the data size of the value received as input.

As described above, the data size determination unit 3516 constructs a relational model that takes the following (A) as an output and at least one of the following (B) and (C) as an input, and uses the relational model to , (B) and (C) may be estimated.
(A) Data size of a dataset composed of two or more observational data.
(B) Prediction accuracy when a prediction process is performed using a dataset of the data size, or the degree of compatibility of the dataset with a prediction model.
(C) Processing load when prediction processing is performed using a data set of the data size.

The data size determination unit 3516 may use a relational model to estimate a data size that satisfies the conditions regarding at least one of (B) and (C). The suitable data set may be a data set having the estimated data size.

By using the data size determination unit 3516, by performing prediction processing for a certain period of time in the past, trends in prediction accuracy deterioration or improvement can be grasped, and it is determined whether or not it is necessary to update the compatible data set 355A. be able to.
(7) Seventh embodiment (addition of input of external data to matching extraction unit 351)

In the first embodiment, the matching extraction unit 351 performs group classification using only the index indicating the periodicity of observed data, but the classification is not limited to this, and classification may also be performed using external data information. good.

This will be explained in detail using FIG. 15. The group classification unit 3514 also receives the external data group 721A as input, and performs group classification using the external data information as well. As the external data, for example, position information of the subject who observed the observation data may be used. Taking electricity demand forecasting as an example, by assigning data with similar location information of electricity consumers to the same group, it is possible to accurately reflect location-dependent influences such as weather and topography in the prediction model. can be expected to improve prediction accuracy.
(8) Eighth embodiment (addition of observation data complementation unit 3517 to matching extraction unit 351)

In the first embodiment, the observation data group 521A input to the matching extraction unit 351 is used as is, but the present invention is not limited to this. You can substitute the value.

This will be explained in detail using FIG. 16. The matching extraction section 351 includes an observed data complementation section 3517. The observation data group 521A is processed by the observation data complementation unit 3517 and then input to the candidate generation unit 3511.

Specifically, for observation data series for which a sufficient amount of past performance values have not been accumulated, one or more similar observation data series are identified based on recent trends, and similar observation data series are identified. The past performance value of the target observation data series may be supplemented with a past performance value or a statistic such as an average of past performance values of a plurality of similar observation data. In addition, when determining observation data to be used for interpolation, external data such as location information and attribute information linked to the observation data to be used for interpolation and the observation data to be interpolated are compared, and decisions are made based on the degree of similarity of the external data. You may do so.

Through the above processing, it becomes possible to use observation data that could not be used in the processing of the data prediction system due to a lack of past performance values, and an improvement in accuracy can be expected. An example of a case where past performance values are insufficient is a case where the number of entities whose observation data, such as electricity consumers, are observed increases.

Although several embodiments of the present invention have been described above, these are illustrative examples for explaining the present invention, and are not intended to limit the scope of the present invention to these embodiments. The present invention can also be implemented in various other forms. For example, an embodiment may be adopted in which two or more of the embodiments listed in the above-mentioned embodiments are used together.

1...Data processing system, 3...Data prediction system

Claims (7)

an interface device connected to a data source containing a plurality of observation data each having periodicity;
a storage device;
a processor connected to the interface device and the storage device;
the processor extracts a plurality of dataset candidates from the plurality of observation data and stores them in the storage device;
Each of the plurality of dataset candidates is two or more observational data for the target period,
In each dataset candidate, the observed data is all or part of one of the plurality of observed data,
The processor,
For each of the plurality of data set candidates, for each observation data in the data set candidate, vector data having as elements the magnitudes of a plurality of different frequency components of the observation data,
For each of the plurality of data set candidates, classifying two or more vector data into one or more data groups based on the distance between the vector data,
For each of the plurality of dataset candidates, based on the number of data for each data group, calculate the degree of fit to a prediction model that uses observed data as input and predicted data as output,
outputting a dataset candidate used for prediction processing using the prediction model as a compatible dataset based on the fitness degree calculated for each of the plurality of dataset candidates;
Data prediction system. For each observation data, the vector data is data at coordinates to which the observation data corresponds in a coordinate system with axes each having a plurality of different periodic components;
The data prediction system according to claim 1. For each of the plurality of dataset candidates, the degree of fit to the prediction model is one of the following:
・Value representing the variation in the number of data in the data group,
・The linearity between the training error or the reciprocal of the likelihood and the value representing the variation when building a prediction model using residual data that is observation data other than the data set candidate among the plurality of observation data. sum,
The data prediction system according to claim 1. In the data group, the distance between vector data is less than or equal to a threshold;
The data prediction system according to claim 1. The processor constructs a relational model that is a model that has the following (A) as an output and at least one of the following (B) and (C) as an input,
(A) Data size of a dataset composed of two or more observational data,
(B) the prediction accuracy when the prediction process is performed using a dataset of the data size, or the degree of conformity of the dataset to the prediction model;
(C) processing load when performing the prediction process using a data set of the data size;
The processor uses the relational model to estimate a data size that satisfies conditions regarding at least one of (B) and (C);
The adapted data set is a data set of the estimated data size,
The data prediction system according to claim 1. The data prediction system according to claim 1,
and a power control system that controls at least one of the power generation equipment and the power storage equipment,
The data prediction system outputs the prediction data by performing the prediction process using the adapted data set,
The power control system receives the prediction data, uses the prediction data to create at least one plan for power generation and power storage, and uses the created plan to control at least one of the power generation equipment and the power storage equipment. control one,
Data processing system. A computer extracts multiple dataset candidates from a data source that includes multiple observational data, each with periodicity,
Each of the plurality of dataset candidates is a collection of two or more observational data for the target period,
In each dataset candidate, the observed data is all or part of one of the plurality of observed data,
for each of the plurality of data set candidates, the computer calculates, for each observation data in the data set candidate, vector data having as elements the magnitudes of a plurality of different frequency components of the observation data,
for each of the plurality of data set candidates, the computer classifies two or more vector data into one or more data groups based on the distance between the vector data;
the computer calculates, for each of the plurality of data set candidates, the degree of fit to a prediction model that uses observed data as input and predicted data as output, based on the number of data for each data group;
a computer outputs a dataset candidate to be used in a prediction process using the prediction model as a compatible dataset, based on the fitness degree calculated for each of the plurality of dataset candidates;
Data prediction support method.

Images