JP7723253B2 - Apparatus for estimating steel material properties, apparatus for generating a steel material property estimation model, method for estimating steel material properties, method for generating a steel material property estimation model, and program - Google Patents

Description

The present invention relates to a steel material property estimation device, a steel material property estimation model generation device, a steel material property estimation method, a steel material property estimation model generation method, and a program.

The strength, elongation, and other properties of steel materials are important when designing products using steel sheets. For example, Patent Document 1 proposes a technique for predicting the properties of steel materials, using image analysis and machine learning to predict tensile strength.

Patent No. 6747391

However, due to constraints such as the size of the steel material and time, it can be difficult to collect test pieces of the steel material and conduct the tests.
An object of the present invention is to provide a steel material property estimation device, a steel material property estimation model generation device, a steel material property estimation method, a steel material property estimation model generation method, and a program that solve the above-mentioned problems.

One aspect of the present invention is a property estimation device comprising: an image acquisition unit that acquires images obtained by imaging a steel material; a hardness acquisition unit that acquires values related to the hardness of the steel material; a feature extraction unit that extracts features from the images acquired by the image acquisition unit; an estimation unit that estimates the mechanical properties of the steel material by inputting the feature values and the values related to hardness into an estimation model whose parameters have been trained so that the image feature values and the values related to hardness are used as input and a value related to the mechanical properties of the steel material is output; and an estimation result output unit that outputs the results of the estimation.

One aspect of the present invention is an estimation model generation device comprising: an image acquisition unit that acquires an image of a steel material; a hardness acquisition unit that acquires a value related to the hardness of the steel material; a property acquisition unit that acquires a value related to a property of the steel material; a feature extraction unit that extracts features from the image acquired by the image acquisition unit; an estimation model update unit that updates parameters of an estimation model that determines a value related to the strength of the steel material from the image of the steel material and the value related to hardness using a dataset in which the feature extracted by the feature extraction unit and the value related to the hardness are used as input samples and the value related to the property is used as output samples; and an estimation model output unit that outputs the estimation model with updated parameters.

The present invention makes it possible to estimate the properties of steel materials.

1 is a diagram illustrating a configuration of a characteristic estimation device according to a first embodiment. 1 is an example of an image of a steel material. 4 is a flowchart showing the operation of the characteristic estimation device 1 according to the first embodiment. FIG. 1 is a diagram illustrating a configuration of an estimation model generating device 2 according to a first embodiment. 4 is a flowchart showing the operation of the estimation model generating device 2 according to the first embodiment. 10 is a table showing a comparison of maximum stresses estimated using an estimation model. 10 is a table showing a comparison of growth rates estimated using an estimation model. 1 is a mapping image of EBSD.

Embodiments of the present invention will be described in detail below with reference to the drawings.

<Configuration of characteristic estimation device>
FIG. 1 is a diagram showing the configuration of a characteristics estimation device 1 according to the first embodiment.
The property estimation device 1 includes an image acquisition unit 10, a hardness acquisition unit 12, a feature extraction unit 14, an estimation model storage unit 16, an estimation unit 18, and an estimation result output unit 20. The property estimation device 1 estimates the mechanical properties of a steel material based on an image and hardness of the steel material.

The image acquisition unit 10 acquires images relating to the structure of a steel material. The image of a steel material is, for example, a photograph of the structure of the steel material taken using an electron microscope such as a scanning electron microscope (SEM) or a transmission electron microscope (TEM). More specifically, the image of a steel material is, for example, an image (photograph) obtained by cutting a test piece of appropriate size from the steel material, polishing and corroding its cross section, and then magnifying and photographing it using an electron microscope. Figure 2 is an example of an image of a steel material. The image shown in Figure 2 was taken using a scanning electron microscope at 1000x magnification. The image of a steel material may also be an image showing information such as the crystal structure, composition, and orientation of the material, such as an EBSD or EPMA mapping image.

The hardness acquisition unit 12 acquires a value related to the hardness of the steel material. The value related to the hardness of the steel material is, for example, the Vickers hardness of the steel material. The value related to the hardness of the steel material is measured, for example, after cutting a test piece from the steel material and polishing its cross section. The test piece is then etched with nital, and an image of the steel material is captured by the image acquisition unit 10. The hardness acquisition unit 12 may acquire the value related to the hardness of the steel material, for example, by automatically calculating it from an image showing an indentation on the test piece.

The feature extraction unit 14 extracts features from the image acquired by the image acquisition unit 10. The feature extraction unit 14 extracts features, for example, by inputting the image into a feature extraction model that takes the image as input and outputs the feature of the image. The feature extraction model may be part of an existing image recognition model such as Inception-v3 (https://tfhub.dev/google/imagenet/inception_v3/feature_vector/) to reduce the training burden. In other words, an image recognition model that has an input layer, an intermediate layer, and an output layer, with only the input layer and intermediate layer extracted, may be used as the feature extraction model. Note that the feature extraction model according to other embodiments may be a model trained by an autoconverter or the like using a dataset, for example.

The estimation model storage unit 16 stores estimation models. The estimation model stored in the estimation model storage unit 16 is a model that receives as input the feature quantities extracted by the feature quantity extraction unit 14 and the values related to the hardness of the steel material acquired by the hardness acquisition unit 12, and outputs estimated values related to the mechanical properties of the steel material. The values related to the mechanical properties of the steel material are values that indicate characteristics such as the tensile strength (TS) or elongation (EL) of the steel material, such as the maximum stress or elongation rate that are the results of a tensile test on the steel material.

The estimation unit 18 inputs the feature values extracted by the feature extraction unit 14 and the values related to the hardness of the steel material acquired by the hardness acquisition unit 12 into the estimation model stored in the estimation model storage unit 16, causing the estimation model to output estimated values related to the mechanical properties of the steel material.

The estimation result output unit 20 outputs the estimation result by the estimation unit 18 to the outside. The estimation result is output to, for example, a display and displayed on the display.

<<Operation of the characteristic estimation device>>
FIG. 3 is a flowchart showing the operation of the characteristic estimation device 1 according to the first embodiment.
Before executing the function of the property estimation device 1, an operator takes an image of the steel material and measures a value related to the hardness of the steel material, so that the property estimation device 1 can acquire the image and the value related to the hardness. The operator inputs an instruction to execute the property estimation function to the property estimation device 1. When the property estimation device 1 executes the property estimation function, the image acquisition unit 10 acquires an image of the steel material (step S101). The hardness acquisition unit 12 acquires a value related to the hardness of the steel material (step S102). The feature extraction unit 14 extracts feature values from the image of the steel material (step S103). The estimation unit 18 estimates a value related to the mechanical property based on the feature values and the value related to the hardness of the steel material (step S104). The estimation result output unit 20 outputs the result estimated by the estimation unit 18 (step S105).

<Configuration of Estimation Model Generation Device>
4 is a diagram showing the configuration of the estimation model generation device 2 according to the first embodiment. The estimation model generation device 2 includes an image acquisition unit 30, a hardness acquisition unit 32, a property acquisition unit 34, a feature extraction unit 36, an estimation model creation unit 38, and an estimation model output unit 40. The estimation model generation device 2 performs learning processing of the estimation model used for calculating mechanical properties by the property estimation device 1.
The image acquisition unit 30, the hardness acquisition unit 32, and the feature extraction unit 36 have the same functions as the image acquisition unit 10, the hardness acquisition unit 12, and the feature extraction unit 14, respectively, provided in the property estimation device 1. The image acquisition unit 10 may acquire multiple images of different steel materials. The hardness acquisition unit 12 may acquire multiple values related to the hardness of different steel materials.

The characteristic acquisition unit 34 acquires values related to the mechanical characteristics of the steel material. The values related to the mechanical characteristics of the steel material are values that indicate characteristics such as the strength and elongation of the steel material, such as the maximum stress and elongation rate that are the results of a tensile test on the steel material. The characteristic acquisition unit 34 may acquire multiple values related to the mechanical characteristics of different steel materials.

The estimation model creation unit 38 updates the parameters of an estimation model that calculates values related to the mechanical properties of a steel material from images and hardness values of the steel material, using a dataset that uses the feature values extracted by the feature extraction unit 36 and the hardness values acquired by the hardness acquisition unit 32 as input samples and the values related to the mechanical properties of the steel material acquired by the property acquisition unit 34 as output samples. The dataset is, for example, a dataset that associates data linking image feature values, hardness values, and mechanical property values corresponding to a single steel material. When the image acquisition unit 30 acquires multiple images, the hardness acquisition unit 32 acquires multiple hardness values, and the property acquisition unit 34 acquires multiple mechanical property values, the estimation model creation unit 38 updates the parameters of the estimation model using the multiple datasets. The estimation model creation unit 38 updates the parameters of the estimation model the same number of times as the number of datasets. For example, the estimation model creation unit 38 creates multiple decision trees from the feature values, hardness values, and mechanical property values using techniques such as bagging and boosting, and creates an estimation model that obtains a solution from the multiple decision trees. Note that in other embodiments, the estimation model may be another machine learning model, such as a neural network model.

The estimation model output unit 40 outputs the estimation model whose parameters have been updated by the estimation model creation unit 38. The estimation model output unit 40 outputs the estimation model to, for example, the characteristic estimation device 1. The estimation model is input to the characteristic estimation device 1 and stored in the estimation model storage unit 16.

<<Operation of the estimation model generation device>>
FIG. 5 is a flowchart showing the operation of the estimation model generating device 2 according to the first embodiment.
The image acquisition unit 30 acquires an image of the steel material (step S201). The hardness acquisition unit 32 acquires a value related to the hardness of the steel material (step S202). The property acquisition unit 34 acquires a value related to the mechanical property of the steel material (step S203). The feature extraction unit 36 extracts feature values from the image of the steel material (step S204). The estimation model creation unit 38 updates the parameters of the estimation model based on the feature values, the value related to hardness, and the value related to the mechanical property (step S205). The estimation model output unit 40 outputs the updated estimation model (step S206).

In this way, the property estimation device 1 estimates the mechanical properties of steel materials by inputting feature quantities extracted from images and hardness-related values into the estimation model generated by the estimation model generation device. Hardness-related values are difficult to obtain from feature quantities extracted from images. On the other hand, hardness-related values can be measured at the same time as the image is captured. That is, as described above, it is possible to obtain images of the steel material's structure and hardness-related values, such as Vickers hardness, using the same test specimen. Mechanical properties such as tensile strength and elongation are known to be correlated with hardness. For example, tensile strength (TS (MPa)) is known to be approximately three times the Vickers hardness value (HV). Mechanical properties are affected by the structure and distribution of the steel material's structure. These structure and distribution appear in images of the steel material (such as structure photographs and EBSD mapping images). Therefore, it can be said that mechanical properties are correlated with hardness and images. The estimation model in this embodiment can reflect hardness and image-related features, enabling more accurate estimation of the mechanical properties of steel materials.

<experiment>
The following describes an evaluation experiment of the estimation model.
The estimation models generated in this experiment were of two types: an estimation model that takes as input the feature values and hardness values of an image of a steel material and outputs an estimated value of maximum stress, and an estimation model that takes as input the feature values and hardness values of an image of a steel material and outputs an estimated value of elongation.
In this experiment, two types of crude steel with different chemical compositions were prepared and processed using known methods (hot rolling, cold rolling, and heat treatment). A 1.0 mm thick steel plate was used as the steel material to be measured. Hereinafter, one of the two types of crude steel will be referred to as the first crude steel, and the other as the second crude steel. The mass percentages of elements excluding iron and impurities in the first crude steel were 0.1% C, 0.005% Si, 2.0% Mn, 0.05% P, and 0.002% S, respectively. The mass percentages of elements excluding iron and impurities in the second crude steel were 0.2% C, 0.005% Si, 2.0% Mn, 0.05% P, and 0.002% S, respectively. In other words, the first crude steel and the second crude steel differ only in their carbon contents. Furthermore, by varying whether or not heat treatment is performed after hot rolling and/or cold rolling in the processing of these first crude steels and second crude steels, as well as the timing or conditions of the heat treatment (heating rate, temperature, holding time, cooling rate, number of times, etc.), the steel plate, which is an iron and steel material, is made to have a variety of structures and a variety of mechanical properties.

The images of the steel materials were obtained by cutting out 15 mm x 7 mm test pieces from each steel plate prepared as described above, polishing and corroding the cross sections of the test pieces, and then photographing them with a scanning electron microscope. In this experiment, the magnification of the electron microscope was set to 1000x. The microstructure distribution of the steel material can be determined by using images of the steel material at magnifications of approximately 500x to 3000x.
The Vickers hardness of the test piece was used as the value relating to the hardness of the steel material obtained. Specifically, the Vickers hardness of the test piece was measured five times and the average value was used.
The obtained values of the mechanical properties were the maximum stress or elongation measured by a tensile test of the steel material. Specifically, test specimens for the tensile test were cut out from each steel plate separately from the test specimens used for the image acquisition and hardness test, and the tensile test was performed.

By using a model of Inception-v3 with the output layer removed, 2048 features were extracted from the image. To increase the number of extracted features, the image from which the features were extracted was a portion of the image acquired by the image acquisition unit (hereinafter referred to as a partial image). Specifically, 10 partial images were randomly selected from the image of one test piece, and features were extracted from the partial images.
The dataset includes 2,048 feature values and average Vickers hardness values extracted from each of the 10 partial images as input samples, and includes maximum stress or elongation as output samples. Images of steel materials, average Vickers hardness values, and maximum stress or elongation were obtained for 131 types of steel plates (test pieces cut from the steel plates) manufactured with two types of chemical compositions and different processing conditions (with or without heat treatment, the timing of heat treatment, or the conditions of heat treatment) as described above. Therefore, the dataset also includes 131 types.

Three estimation models were created for comparison. The first estimation model (estimation model A) was generated using 2,048 feature values extracted from partial images and the average Vickers hardness value, resulting in 2,049 feature values. Estimation model A is the estimation model related to this embodiment. The second estimation model (estimation model B) was generated using 2,048 feature values extracted from partial images, without using the average Vickers hardness value. The third estimation model (estimation model C) was generated using the average Vickers hardness value as a single feature value, without using the 2,048 feature values extracted from partial images. Since there were 131 x 10 partial images, 1,310 combinations of partial images, average Vickers hardness values, and maximum stress or elongation rate were used to generate estimation model A. Similarly, 1,310 combinations of partial images and maximum stress or elongation rate were used to generate estimation model B. There are 131 combinations of average Vickers hardness and maximum stress or elongation used to generate estimated model C.

LightGBM, a gradient boosting algorithm, was used to generate the estimation model. The estimation model shows the correlation between feature values and maximum stress or elongation, and when feature values are input into the estimation model, the maximum stress or elongation is output.

33 types of datasets were used to evaluate the estimation models. For the evaluation of the estimation models, 10 partial images were randomly selected from the image. To evaluate estimation model A, the feature values extracted from the partial images and the average value of Vickers hardness were input into estimation model A, and the average value of the 10 maximum stresses or elongations output from estimation model A was used as the estimated maximum stress or elongation. To evaluate estimation model B, the feature values extracted from the partial images were input into estimation model B, and the average value of the 10 maximum stresses or elongations output from estimation model B was used as the estimated maximum stress or elongation. To evaluate estimation model C, the average value of Vickers hardness was input into estimation model C, and the maximum stress or elongation output from estimation model C was used as the estimated maximum stress or elongation.

The estimation model was evaluated based on two indicators. The first indicator is the accuracy of the estimation model. Accuracy is the slope of a graph with measured values on the horizontal axis and estimated values on the vertical axis; the closer the slope is to 1, the more accurate the estimation model. The second indicator is the precision of the estimation model. Precision is the RMSE (Root Mean Square Error) between the measured values and estimated values; the smaller the RMSE, the more accurate the estimation model.

Figure 6 is a table showing a comparison of maximum stress estimates using estimation models. Since the accuracy of estimation model A is closer to 1 than the accuracy of estimation models B and C, and the precision of estimation model A is less than the accuracy of estimation models B and C, it can be said that estimation model A is a superior estimation model.

Figure 7 is a table showing a comparison of growth rates estimated using estimation models. As with the table shown in Figure 6, the accuracy of estimation model A is closer to 1 than the accuracy of estimation models B and C, and the precision of estimation model A is less than the accuracy of estimation models B and C. Therefore, it can be said that estimation model A is a superior estimation model.

Other Embodiments
One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention.
For example, values relating to the mechanical properties of steel materials may be values indicating properties such as toughness and hole expandability, which are related to the hardness and structure of the steel material. Toughness generally decreases as hardness increases (see, for example, Nobuhiro Nihira, "Latest Heat Treatment Mechanisms and Technologies," pp. 48-49). Furthermore, toughness improves as the structure becomes finer (see, for example, Toshiro Kobayashi, "Materials Strength and Toughness: Material Strength and Toughness," p. 113). Examples of values indicating toughness include the impact absorption energy, fracture toughness, and ductile-brittle transition temperature of steel materials. Hole expandability generally tends to decrease as hardness increases (see, for example, the relationship between strength and hole expandability, http://plast.me.tut.ac.jp/present/090304highten_pres.pdf). Hole expandability is also affected by the structure, and tends to improve as the structure becomes finer (see, for example, Nippon Steel Technical Report, Vol. 378 (2003), p. 7). The value indicating the hole expandability is, for example, the hole expansion ratio.

The image of the steel material is not limited to an image taken by magnifying the steel material with an electron microscope after polishing and corroding it. For example, the image of the steel material may be a crystal orientation analysis image obtained by measuring the composition and crystal orientation of a small area of the steel material and mapping the measurement results. For example, it may be an EBSD mapping image as shown in Figure 8. The image of the steel material may also be an image taken by magnifying the steel material with a microscope after polishing it with a mirror polishing material without corroding it.

The hardness value of steel materials is not limited to Vickers hardness. For example, the hardness value may be any index of the hardness of the test object, such as Rockwell hardness, Knoop hardness, Brinell hardness, Shore hardness, Durometer hardness, Barcol hardness, or ultrafine hardness.

1. Characteristic estimation device, 2. Estimation model generation device, 10. Image acquisition unit, 12. Stiffness acquisition unit, 14. Feature extraction unit, 16. Estimation model storage unit, 18. Estimation unit, 20. Estimation result output unit, 30. Image acquisition unit, 32. Stiffness acquisition unit, 34. Characteristic acquisition unit, 36. Feature extraction unit, 38. Estimation model creation unit, 40. Estimation model output unit

Claims (9)

an image acquisition unit that acquires an image obtained by imaging the steel material;
a hardness acquisition unit that acquires a value related to the hardness of the steel material;
a feature extraction unit that inputs the image acquired by the image acquisition unit into a feature extraction model that outputs feature quantities of the image input as an input, and extracts an output from the feature extraction model as a feature quantity of the image acquired by the image acquisition unit ;
an estimation unit that estimates the mechanical properties of the steel material by inputting the feature amounts of the image acquired by the image acquisition unit and the values related to hardness into an estimation model whose parameters have been trained so as to input the feature amounts of the image and the values related to hardness and output values related to the mechanical properties of the steel material;
an estimation result output unit that outputs the result of the estimation;
Equipped with
The values relating to the mechanical properties do not include the values relating to the hardness.
Steel material property estimation device. The hardness value is Vickers hardness.
The steel material property estimation device according to claim 1 . The value related to the mechanical property of the steel material is the maximum stress of the steel material or the elongation of the steel material.
The steel material property estimation device according to claim 1 or 2. an image acquisition unit that acquires an image of the steel material;
a hardness acquisition unit that acquires a value related to the hardness of the steel material;
a characteristic acquisition unit that acquires values related to the mechanical characteristics of the steel material;
a feature extraction unit that inputs the image acquired by the image acquisition unit into a feature extraction model that outputs feature quantities of the image input as an input, and extracts an output from the feature extraction model as a feature quantity of the image acquired by the image acquisition unit ;
an estimation model update unit that updates parameters of an estimation model that determines values related to the mechanical properties of a steel material from images of the steel material and values related to hardness, using a dataset in which the features of the images acquired by the image acquisition unit extracted by the feature extraction unit and the values related to hardness are used as input samples and the values related to the mechanical properties are used as output samples;
an estimation model output unit that outputs the estimation model with updated parameters;
Equipped with
The values relating to the mechanical properties do not include the values relating to the hardness.
A device for generating a model to estimate the properties of steel materials. The estimation model is an estimation model output by the steel material property estimation model generation device according to claim 4.
The steel material property estimation device according to any one of claims 1 to 3. an image acquisition step of acquiring an image obtained by imaging the steel material;
a hardness acquisition step of acquiring a value related to the hardness of the steel material;
a feature extraction step of inputting the image acquired in the image acquisition step into a feature extraction model that outputs feature quantities of the image input by using the image as an input, and extracting an output from the feature extraction model as a feature quantity of the image acquired in the image acquisition step;
an estimation step of estimating mechanical properties of the steel material by inputting the feature amounts of the image acquired in the image acquisition step and the values related to hardness into an estimation model whose parameters have been trained so as to input the feature amounts of the image and the values related to hardness and output values related to mechanical properties of the steel material;
an estimation result output step of outputting a result of the estimation;
and
The values relating to the mechanical properties do not include the values relating to the hardness.
Methods for estimating the properties of steel materials. an image acquisition step of acquiring an image of the steel material;
a hardness acquisition step of acquiring a value related to the hardness of the steel material;
a characteristic acquisition step of acquiring values related to mechanical characteristics of the steel material;
a feature extraction step of inputting the image acquired in the image acquisition step into a feature extraction model that outputs feature quantities of the image input by using the image as an input, and extracting an output from the feature extraction model as a feature quantity of the image acquired in the image acquisition step;
an estimation model updating step of updating parameters of an estimation model that determines values related to the mechanical properties of a steel material from the image of the steel material and the values related to hardness, using a dataset in which the feature amounts of the image acquired in the image acquiring step extracted in the feature amount extracting step and the values related to hardness are used as input samples and the values related to the mechanical properties are used as output samples;
an estimation model output step of outputting the estimation model with updated parameters;
and
The values relating to the mechanical properties do not include the values relating to the hardness.
A method for generating a model to estimate the properties of steel materials. A program for causing a computer to execute the method described in claim 6. A program for causing a computer to execute the method described in claim 7.