RU2848144C1 - Method and computer system for predicting catalyst activity - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: stages at which: a set of characteristics related to catalytic activity is selected based on domain knowledge, whereby, for the practical value of subgroups as predictive models, the selected characteristic values must be easier to determine for catalysts not included in the training data set than catalytic activity; a training data set is formed and characteristic values are calculated using quantum chemistry methods for catalysts in this set; calculate catalytic activity values using quantum chemistry methods for catalysts in the training data set; apply a subgroup detection method (DM) with built-in cross-validation to search for subgroup selectors that contain and predict catalysts with high catalytic activity; calculate characteristics for a large number of candidate materials not included in the training set; use the found subgroup selectors to predict and search for the most active catalysts.

EFFECT: improved accuracy of prediction and search for the most active catalysts.

1 cl, 2 dwg, 2 tbl

Description

AREA OF TECHNOLOGY

This technical solution relates to the field of information technology, in particular to a method for predicting the activity of catalysts.

The work was carried out within the framework of the implementation of the Decree of the Government of the Russian Federation No. 220 of April 9, 2010 and Agreement No. 075-10-2022-119 of September 30, 2022 on the provision of grants from the federal budget in the form of subsidies in accordance with paragraph 4 of Article 78.1 of the Budget Code of the Russian Federation, concluded between the Ministry of Science and Higher Education of the Russian Federation and the Autonomous Non-Commercial Educational Organization of Higher Education "Skolkovo Institute of Science and Technology" (government contract identifier 000000S707521QJX0002).

LEVEL OF TECHNOLOGY

A prior art solution describes approaches to learning classification rules for subgroup detection. The goal of subgroup detection is to find rules that describe population subsets that are sufficiently large and statistically unusual (article "Subgroup Discovery with CN2-SD", Journal of Machine Learning Research 5 (2004) 153-188, https://sci2s.ugr.es/keel/pdf/algorithm/articulo/2004-Lavrac-JMLR.pdf). This article presents a subgroup detection algorithm, CN2-SD, developed by modifying parts of the CN2 classification rule learner: its coverage algorithm, search heuristics, probabilistic classification of instances, and evaluation measures.

The drawback of this solution is that it adapts the standard classification rule learning method to the problem of finding special subgroups. This necessitates additional parameters that define the tradeoff between the generality and accuracy of the resulting rules. Furthermore, the property of subgroup locality (independence from other subgroups) is violated due to the global nature of standard classification rule learning methods.

In addition, a subgroup discovery approach for defect prediction is known in the prior art (Article “A study of subgroup discovery approaches for defect prediction”, Information and Software Technology Volume 55, Issue 10, October 2013, Pages 1810–1822, https://doi.org/10.1016/j.infsof.2013.05.002). This paper describes two subgroup discovery algorithms, the SD algorithm and the CN2-SD algorithm, for obtaining rules that identify defect-prone modules. The empirical work is performed with publicly available datasets from the Promise repository and object-oriented metrics from the Eclipse repository related to defect prediction. Subgroup discovery algorithms mitigate dataset characteristics that hinder the applicability of classification algorithms and thus eliminate the need for preprocessing methods.

A drawback of this well-known solution is that it uses a standard subgroup detection method, and it lacks cross-validation to improve prediction accuracy. Furthermore, the CN2-SD method did not perform better than the standard subgroup detection method.

The prior art also includes the article "A New Method to Evaluate Subgroup Discovery Algorithms" (Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications (CIARP 2019), https://link.springer.com/chapter/10.1007/978-3-030-33904-3 39). This article discloses a method for evaluating and comparing subgroup detection algorithms. This method begins with redundancy removal using a novel procedure based on the examples covered by the patterns and the statistical redundancy between them. Then, novel similarity and quality methods are used to compare the algorithms based on their ability to detect patterns and the quality of the mined patterns, respectively. The experimental results obtained reveal some interesting findings that would have gone unnoticed using the traditional approach.

The disadvantage of this solution is that it does not allow subgroup selection based on cross-validation, although it does offer an interesting way to assess the quality of subgroups in addition to standard quality functions.

ESSENCE OF THE INVENTION

The technical problem that the proposed solution aims to solve is the creation of a computer-implemented method for predicting the activity of catalysts.

The technical result of the claimed technical object consists in the implementation of the purpose, as well as in ensuring effective forecasting and search for the most active catalysts.

The claimed technical result is achieved through the implementation of a computer-implemented method for predicting the activity of catalysts, comprising the following stages:

select a set of characteristics based on domain knowledge related to catalytic activity, whereby for the subsets to be of practical value as predictive models, the selected characteristic values should be more easily determinable for catalysts not included in the training data set than catalytic activity;

a training data set is formed and the characteristic values are calculated using quantum chemistry methods for the catalysts in this set;

calculate catalytic activity values using quantum chemistry methods for catalysts in the training data set;

The subgroup detection (SD) method with built-in cross-validation is used to find subgroup selectors that contain and predict catalysts with high catalytic activity;

calculate characteristics for a large number of candidate materials not included in the training set;

use the found subgroup selectors to predict and search for the most active catalysts.

DESCRIPTION OF DRAWINGS

The invention will be further described in accordance with the accompanying drawings, which are provided to illustrate the invention and in no way limit its scope. The following drawings are attached to the application:

Fig. 1 illustrates a scheme for searching for highly active catalysts using a modified data mining method called “subgroup detection” (SD).

Fig. 2 illustrates the general appearance of a programmable multifunctional device.

DETAILED DESCRIPTION OF THE INVENTION

The following detailed description of the invention provides numerous implementation details intended to provide a clear understanding of the present invention. These details disclose preferred embodiments of the technical solution and should not be construed as limiting other, specific embodiments within the scope of the claimed legal protection that are obvious to those skilled in the art. However, it will be obvious to those skilled in the art how the present invention can be used, both with and without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the features of the present invention.

Furthermore, it will be clear from the foregoing description that the invention is not limited to the embodiment described. Numerous possible modifications, changes, variations, and substitutions, while preserving the spirit and form of the present invention, will be apparent to those skilled in the art.

The claimed technical solution addresses an existing technical problem. In particular, the claimed method finds combinations of catalyst characteristics and a range of their quantitative values that lead to increased catalytic activity. The models obtained by the proposed method are physically interpretable. For example, in the presented paper "Interpretable Data-Driven Descriptors for Establishing the Structure-Activity Relationship of Metal-Organic Frameworks Toward Oxygen Evolution Reaction" (Angewandte Chemie International Edition, https://onlinelibrary.wiley.com/doi/epdf/10.1002/anie.202409449), a subgroup of effective catalysts for the oxygen evolution reaction during water electrolysis to produce hydrogen was identified. The found subgroup has the form (first ionization potential of impurity >7.91 eV) AND (number of electrons in e _g orbitals of impurity in the catalyst volume ≥2.44) AND (number of missing electrons in e _g orbitals of nickel in the catalyst volume ≥0.42) AND (-2.90 eV ≤ center of d-band of Ni in the catalyst volume ≤-2.13 eV). As shown in the article, this description suggests a mechanism for catalyst optimization by tuning the electronic structure of the catalyst.

The paper "Artificial-intelligence-driven discovery of catalyst genes with application to CO2 activation on semiconductor oxides" (Nature Communications, https://www.nature.com/articles/s41467-022-28042-z) identified a subset that allows rapid prediction of whether a given surface will activate _CO2 for subsequent chemical conversion of this greenhouse gas into useful products such as fuels and reagents for the chemical industry. The subset has the form where U is the energy of the 2p state of surface oxygen with the least negative energy, _d2 is the distance between the adsorption site and the second-nearest cation, _qmin and _qmax are the minimum and minimum charges of cations on the surface, and kurt is the excess of the 2p band of surface oxygen. Since all these properties are characteristics of a clean surface, without an adsorbed _CO2 molecule, their calculation requires far fewer computational resources than calculating the properties of an adsorbed molecule. Furthermore, the models found by the new method described below allow for more reliable predictions of catalytic activity.

This technical solution utilizes a data mining method called subgroup detection (SD) (Fig. 1), which enables the identification of combinations of catalyst characteristics and ranges of their values that lead to increased catalytic activity. For example, in the examples described above, subgroups were identified that contain materials with high catalytic activity, but are defined by characteristics that are much easier to calculate than the catalytic activity itself.

In the OP method, the key parameter is the quality factor (QF). The QF quantifies the "uniqueness" or "usefulness" of a subgroup. The higher or lower the QF, the more interesting the subgroup is for a specific application. A classic example of a QF is the following function:

where Q is the value of the functional complex, N _s , N _a are the number of data in the subgroup and in the entire data set, mean _s , mean _a are the average value of the target property (attribute) in the subgroup and in the entire data set, max _a is the maximum value of the target property in the entire data set [Webb GI (2001) Discovering associations with numeric variables. In: Proceedings of the 7th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 383–388]. In subgroups that maximize such a functional complex, the average value of the target property is closer to its maximum value in the entire data set, and the subgroups contain a sufficient amount of data, which increases their statistical significance.

Subgroups are defined by combinations of conditions (selectors) of the form (characteristic 1>a) AND (characteristic 2=b) AND …, where characteristic1, characteristic2, … are some characteristics of each data element, a, b are numerical parameters or class identifiers (categories, for example, "good catalyst"/"bad catalyst"). Thus, in the above example of a subgroup of effective water electrolysis catalysts, the conditions are the following inequalities: (first ionization potential of impurity >7.91 eV), (number of electrons in e _g orbitals of impurity in catalyst volume ≥2.44), (number of missing electrons in e _g orbitals of nickel in catalyst volume ≥0.42), (-2.90 eV ≤ center of d-band of Ni in catalyst volume ≤ -2.13 eV). The numerical parameters in such inequalities are typically found using machine learning, such as k-means clustering, as described in "Uncovering structure-property relationships of materials by subgroup discovery" (New Journal of Physics, https://iopscience.iop.org/article/10.1088/1367-2630/aa57c2).

Finding subgroups that optimize a functional complex is a so-called non-polynomially difficult (NP-hard) problem, the computational complexity of which grows exponentially with the number of conditions defining the subgroups. Stochastic (Monte Carlo, evolutionary) or deterministic (branch-and-bound) algorithms are used to solve it.

Below is a general algorithm for searching for the most active catalysts using the OP method:

1) Select a set of characteristics that are potentially related to catalytic activity. The user makes the selection based on domain knowledge. For example, the catalytic activity of transition metals can be significantly affected by the center of the d-band. For the catalytic activity of metal oxides, the position of the center of the 2p-band of oxygen is also important. In general, the properties of the atoms that make up the catalyst, such as ionization potential, electron affinity, size, etc., influence catalytic properties. Therefore, these properties must be included in the set of characteristics. For the subsets to be useful as predictive models (i.e., ones that can predict the properties of materials not included in the training dataset), the selected characteristic values should be more easily determinable for catalysts not included in the training dataset than the catalytic activity.

2) A training dataset is generated and the characteristic values for the catalysts in this dataset are calculated using quantum chemistry methods. The training set can be generated by randomly selecting materials. However, a more effective approach is active learning, in which the training set is generated in stages: 1) a small (20-30 materials) random sample, 2) training the model, 3) predicting several new candidates, 4) including the best candidates in the training set, 5) repeating the steps from step 2.

3) Calculate the catalytic activity values using quantum chemistry methods (e.g., within the framework of density functional theory with approximate functionals LDA, PBE, SCAN, or others; in the case of the presence of transition metal ions in the catalyst, as in Ni-containing metal-organic frameworks, it is necessary to use corrections to the standard approximate functionals, for example, the Hubbard correction +U) for the catalysts in the training data set.

4) The subgroup detection (SD) method with built-in cross-validation is used to search for subgroup selectors containing and predicting catalysts with high catalytic activity.

5) Calculation of characteristics for a large number of candidate materials not included in the training set.

6) Use the found subgroup selectors to predict and search for the most active catalysts.

A key feature of applying the OP method to the search for active catalysts is that the dominant goal is to increase catalyst activity; i.e., the primary factor in PK should be an increase in catalytic activity in a subgroup compared to the entire dataset, rather than the subgroup size. However, this may reduce the statistical significance of the subgroup, as the training set contains few materials with high activity, meaning that only a small fraction of the materials in the training dataset will be included in the subgroup. This may reduce the predictive power of the selectors. Therefore, to ensure reliable predictions, a method is needed to find subgroups with the highest possible predictive power for a given range of target property (catalytic activity).

Cross-validation (CV) is widely used to find machine learning (ML) and data mining (DM) parameters (hyperparameters) that yield machine learning models with the highest predictive power, i.e., the lowest prediction error for data not included in the training dataset. For example, in the N-fold CV method (https://scikit-learn.Org/1.5/modules/cross_validation.html), the training dataset is split into N (usually) roughly equal parts. For each of these N parts, the CV model is trained on the dataset from which that part was excluded, and the prediction error of the resulting model is calculated. The prediction errors obtained in this way for all N parts are then averaged. This procedure is repeated for different sets of hyperparameters, ultimately finding the set of hyperparameters that yields the minimum prediction error. Machine learning with this set of hyperparameters on the full dataset yields a final machine learning model with maximum predictive power. In the OP method, the FC can be considered a hyperparameter.

The goal of the proposed solution is to develop a method for improving the reliability of data subset selector predictions designed to quickly predict combinations of characteristics that optimize a target property. Here, characteristics and target properties are defined as any properties of materials or processes that can be measured or calculated using mathematical methods. Optimization is defined as achieving the desired quantitative or qualitative values of a target property.

In the context of OP, PP can be used to select a FC for which the optimizing FC subgroups have the highest predictive power, i.e., the fewest errors in predicting whether new data falling into a subgroup have the desired properties. For example, in formula (1), a set of FCs can be generated by introducing the power 0<γ<1 for the population frequency N _s /N _a . The smaller y, the smaller the size of the optimal subgroup, but the greater the deviation of the average target property in the subgroup from the average for the entire data set. If subgroups with the greatest deviation are of interest, then smaller values of y yield more interesting subgroups. However, for smaller subgroups, the predictive power of OP decreases. PP makes it possible to find the minimum value of y for which the predictive power of the subgroups remains at an acceptable level.

The proposed solution proposes using PP to select subgroups that are least sensitive to variations in the composition of the training dataset. The key difference from traditional approaches to PP is that it uses variation within the subgroups themselves, rather than within the PK, to find predictive subgroups. For a fixed PK shape, there are often many subgroups with PK values close to the maximum. This occurs due to the correlation between the main characteristics included in the conditions defining the subgroups, as well as the correlation between the conditions from which the selectors are constructed. This variation can be exploited to select the most predictive subgroups. Moreover, the deviation of the PK values for such subgroups from the PK values for the subgroups optimal for each data subset tested in PP can be used as a criterion for the predictive ability of the subgroup. This approach can be used both independently and alongside the traditional PP approach to increase the predictive ability of OP.

In other words, the conditions themselves included in the subgroup selectors can be viewed as method parameters that can be optimized using PP to improve the predictive power of the subgroups. Since the number of conditions can be large, the number of parameters to optimize is also large, but the algorithms used to find optimal subgroups are precisely suited for efficient optimization of functions with a large number of variables. For example, the Monte Carlo method (Uncovering structure-property relationships of materials by subgroup discovery, New Journal of Physics, https://iopscience.iop.org/article/10.1088/1367-2630/aa57c2), beam search (https://en.wikipedia.org/wiki/Beam search), and branch and bound methods (A. H. Land and A. G. Doig (1960). "An automatic method of solving discrete programming problems". Econometrica. Vol. 28, no. 3. pp. 497-520. doi: 10.2307/1910129) are used. Thus, the problems of finding optimal subgroups and increasing their predictive ability can be effectively solved simultaneously. The general scheme of the algorithm for finding such predictive subgroups is as follows:

1. From the training dataset, N training sets are created following some PP scheme.

2. For each of the training data sets obtained in the previous step, find the optimal value of the FC using one of the above algorithms.

3. Using one of the above algorithms, find subgroups that minimize the deviation of the FC from the optimal FC values for all training sets of the PP simultaneously.

The deviation measure in step 3 can be chosen differently. For example, it could be the average difference between the FC values for a given subset and the optimal FC values for the PP training datasets. However, this choice can lead to excessive deviations for one or more training sets if the optimal FC values for these sets differ significantly from the other sets. To avoid this, the maximum relative deviation from the optimal FC values across all sets can be used. The most predictive subset should minimize this maximum deviation.

The proposed solution improves the reliability of predictions for data subgroup selectors designed to predict combinations of characteristics that maximize catalytic activity. This method can be applied at any enterprise requiring catalytic process optimization. Furthermore, the resulting subgroup selectors will provide more reliable predictions for new data not included in the training set.

Below is a detailed algorithm of the method.

1) The class of oxygen evolution reaction catalysts MaMb-TDC is studied, where Ma, Mb are transition metals, TDC is thiophenedicarboxylic acid.

2) The data set was obtained using the VASP package (S. Grimme, J. Comput. Chem. 27 (2006) 1787–1799). All calculations were performed within the framework of the density functional theory with the approximate PBE functional ([J. P. Perdew, K. Burke, M. Ernzerhof, Phys. Rev. Lett. 1996, 77, 3865) with the Hubbard correction U implemented in the VASP package. The values of U = 3.4 [Phys. Rev. B 77, 241–201 (2008)], 2.0 [Phys. Chem. Chem. Phys. 13, 978–984 (2010)], 4.0 [Chemistry of Materials 27, no. 17 (2015): 5856–5867], 3.2 [Nano Energy 68, 104–306 (2020)], 4.0 [Acta Mater. 163, 199–207 (2019)], and 3.5 eV [Surface Science 606, no. 17–18 (2012): 1422–1425] were used for Ni, Co, Fe, Mn, Ti, and Cr, respectively. The target property (additional reaction potential) and the characteristics used to define the subgroups were calculated for 100 materials. Optimal catalysts should have as low an additional potential as possible.

3) The data set with additional modifications for the PP is given below in the XARF format used by the OP (property - target property in volts) method algorithm:

4) The data set is modified as follows: column 1 (property) is copied into ten new columns (prop1 - prop10). In each of the copied columns, 10% of the data is replaced with the "?" character, with each row containing only one "?" character. These columns are used in the software implementation of the new OP with PP method. This format is not the only possible one for implementing the new OP with PP method; OP data can be generated automatically. The method by which data for PP is generated is not an essential part of the proposed invention.

5) The characteristics were selected based on physical concepts about the influence of the active site properties Ma (the first element in the first column of the data set), the dopant (the second element in the first column of the data set), and the bridging oxygen atoms connecting Ma and Mb in the metal–organic framework structure on the additional potential. The list of characteristics used is given in Table 1:

6) A new method of OP with PP is applied to the data set. The new method of OP with PP, described above, was implemented by the authors of the invention in a modified version of the Java program realkd (http://www.realkd.org/). The following quality function for PP was used:

where N _s , N _a are the numbers of materials in the subgroup (s) and in the entire data set (a), c is a parameter, N _s<c is the number of materials in the subgroup (s) with an additional potential below c, and θ is the Heaviside function. This form of the quality function is motivated by the concept of information gain in statistics: information is maximized when most of the subgroup is within a given interval. This allows one to find subgroups containing as many materials as possible for which the maximum additional potential is below c. The specific choice of the objective function is not an essential part of this invention.

The resulting subgroup with the maximum FC value for predicting whether a given catalyst has an additional potential below c=0.5 eV has the following form:

To test the new OP with PP method, the above dataset was randomly split into two parts: a) 90% and 10%, b) 80% and 20%. Afterwards, the OP with PP method and the regular method with the above-mentioned PK were applied to the majority of each of the splits a), and b), with the parameter c = 0.5 and 0.6 for the 90%/10% and 80%/20% splits, respectively. The resulting subgroups were then applied to 100% of the data, and the number of mispredictions and their average incremental potential were obtained. The results are presented in Table 2.

It can be seen that even for a very small subgroup size, the new OP with PP method gives better predictions on average, namely: the number of incorrectly classified materials (i.e., those included in the subgroup but having an additional potential above the threshold) and the average value of the additional potential in the subgroup are smaller than in the standard OP approach.

Fig. 2 shows a general view of a programmable multifunctional device (1600). On the basis of the device (1600), any of the devices (110, 120, 130, 140, 210, 230, 240, 260) and another type of device that provides computer processing of data necessary for the implementation of the claimed method can be implemented.

In general, the computing device (1600) comprises one or more processors (1602), memory means such as RAM (1603) and ROM (1604), input/output interfaces (1605), input/output devices (1406), and a device for network interaction with a telecommunications data transmission network (1607), united by a common information exchange bus (1601).

A processor (1602) is an integrated circuit that executes machine instructions (program code), the main component of a computer's hardware. It can contain multiple processor cores in a single package.

RAM (Random Access Memory) (1603) is a temporary storage device that contains data and machine-readable instructions to perform the necessary logical data processing operations.

ROM (read-only memory) (1604) is a type of non-volatile memory used in computers and other electronic devices. ROM (1604) represents one or more permanent storage devices, such as a hard disk drive (HDD), solid-state drive (SSD), etc.

An I/O interface (input/output interface) (1605) is a "conductor" between the user and a program, operating system, technical device, or a way for applications to interact with each other.

Input and output devices (1606) are peripheral equipment designed to input and output data or signals into a computer or other electronic device during its operation. Various I/O devices (1606) are used, such as a keyboard, display (monitor), computer mouse, trackball, touchpad, stylus, graphics tablet, touchscreen, etc.

The network communication means (1607) ensures the transmission of data by the device (1600) via an internal or external computer network, for example, an Intranet, the Internet, a LAN, etc. One or more means (1606) may be, but are not limited to: an Ethernet card, a modem, a Bluetooth module, a Wi-Fi module, etc.

Although the invention has been described with reference to the disclosed embodiments, it will be apparent to those skilled in the art that the specific experiments described in detail are provided merely for the purpose of illustrating the present invention and should not be construed as limiting the scope of the invention in any way. It should be understood that various modifications are possible without departing from the spirit of the present invention.

Claims (7)

A computer-implemented method for predicting catalyst activity, comprising the following steps: select a set of characteristics based on domain knowledge related to catalytic activity, whereby for the subsets to be of practical value as predictive models, the selected characteristic values should be more easily determinable for catalysts not included in the training data set than catalytic activity; a training data set is formed and the characteristic values are calculated using quantum chemistry methods for the catalysts in this set; calculate catalytic activity values using quantum chemistry methods for catalysts in the training data set; The subgroup detection (SD) method with built-in cross-validation is used to find subgroup selectors that contain and predict catalysts with high catalytic activity; calculate characteristics for a large number of candidate materials not included in the training set; use the found subgroup selectors to predict and search for the most active catalysts.