WO2020071559A1 - Vehicle state evaluation device, evaluation program therefor, and evaluation method therefor - Google Patents

Abstract

[PROBLEM] To flexibly, quickly and accurately evaluate a damage state of a vehicle. [SOLUTION] A component evaluation unit 4 evaluates a vehicle component in a captured image by referring to a component learning model 9. The component learning model 9 is constructed by supervised learning using data relating to vehicle components as training data, using a deep learning-based object detection algorithm. The object detection algorithm inputs captured images into a single neural network system, thereby collectively performing region extraction of vehicle components in the captured images with classification of attributes by means of a regression problem approach. A state evaluation unit 5 references a state learning model 10 to evaluate the damage state of a vehicle component evaluated by the component evaluation unit 4. The state learning model 10 is constructed by supervised learning using data relating to the damage states of vehicle components as training data.

Description

VEHICLE STATE DETERMINATION DEVICE, ITS DETERMINATION PROGRAM AND ITS DETERMINATION METHOD

The present invention relates to a vehicle state determination device that determines a damage state of a vehicle from a captured image, a determination program thereof, and a determination method.

For example, Patent Literature 1 discloses an accident vehicle repair cost estimation system capable of accurately and promptly evaluating damage to an accident vehicle and estimating repair. This estimation system includes a capture unit, a storage unit, an input unit, a display unit, a link unit, and an estimation unit. The capture means captures image data of the accident vehicle. The storage means stores vehicle attribute data necessary for estimating an accident vehicle repair cost. The input means inputs estimation data necessary for estimating an accident vehicle repair cost. The display means displays various data including the accident vehicle image data. The linking means determines which portion of the vehicle the image data clearly indicates damage to. The estimating means simultaneously displays the image data and the vehicle attribute data of the part corresponding to the image data on the display means. The estimating means estimates the cost required for repairing the accident vehicle based on the estimated data and the vehicle attribute data.

JP-A-10-197285

However, in the system of Patent Literature 1, since the degree of damage is determined from the image data of the accident vehicle based on the similarity to the image data of the vehicle that has been repaired in the past, if there is no appropriate past image data, Decision accuracy decreases.

Therefore, an object of the present invention is to determine a damaged state of a vehicle with high flexibility and at high speed and with high accuracy.

べ く In order to solve such a problem, a first invention provides a vehicle state determination device that includes a component determination unit and a state determination unit and determines a damage state of a vehicle from a captured image. The component determination unit determines a vehicle component in the captured image with reference to the component learning model. The component learning model is constructed by supervised learning using data on vehicle components as teacher data using an object detection algorithm based on deep learning. The object detection algorithm inputs a captured image to a single neural network system, and collectively performs region extraction of vehicle components in the captured image with attribute classification by a regression problem approach. The state determination unit determines the damage state of each vehicle component determined by the component determination unit with reference to the state learning model. The state learning model is constructed by supervised learning using data on the damage state of the vehicle parts as teacher data.

Here, in the first invention, the object detection algorithm is preferably YOLO or SSD. In the first aspect, a surface determination unit may be further provided. The surface determination unit determines the constituent surface of the vehicle in the captured image with reference to the surface learning model. The surface learning model is constructed by supervised learning using data on the constituent surfaces of the vehicle as teacher data. In this case, it is preferable that the component determination unit filters the vehicle component based on the constituent surface determined by the surface determination unit, and outputs the filtered vehicle component as a determination result. Further, when there is a vehicle component unique to a specific component surface in the captured image, the surface determination unit may include a component surface corresponding to the specific component component in the determination result.

に お い て In the first invention, an estimate calculation unit may be further provided. The estimation calculation unit estimates the repair cost of the vehicle from the damage state of each vehicle component determined by the state determination unit by referring to the estimation table. The estimation table holds a repair cost associated with a damaged state of a vehicle component for each vehicle component.

In the first aspect, at least one of a shift of processing from the surface determination unit to the component determination unit, a shift of processing from the component determination unit to the state determination unit, and a shift of processing from the state determination unit to the estimation calculation unit. One is preferably performed on condition that the user's approval is obtained, while allowing the user to correct the determination result.

The second invention provides a vehicle state determination program that causes a computer to execute a process having a component determination step and a state determination step, and determines a damage state of the vehicle from a captured image. In the component determination step, a vehicle component in the captured image is determined with reference to the component learning model. The component learning model is constructed by supervised learning using data on vehicle components as teacher data using an object detection algorithm based on deep learning. The object detection algorithm inputs a captured image to a single neural network system, and collectively performs region extraction of vehicle components in the captured image with attribute classification by a regression problem approach. In the state determination step, the damage state of each vehicle component determined by the component determination unit is determined with reference to the state learning model. The state learning model is constructed by supervised learning using data on the damage state of the vehicle parts as teacher data.

A third invention provides a vehicle state determination method that includes a component determination step and a state determination step, and determines a damage state of a vehicle from a captured image. In the component determination step, a vehicle component in the captured image is determined with reference to the component learning model. The component learning model is constructed by supervised learning using data on vehicle components as teacher data using an object detection algorithm based on deep learning. The object detection algorithm inputs a captured image to a single neural network system, and collectively performs region extraction of vehicle components in the captured image with attribute classification by a regression problem approach. In the state determination step, the damage state of each vehicle component determined by the component determination unit is determined with reference to the state learning model. The state learning model is constructed by supervised learning using data on the damage state of the vehicle parts as teacher data.

Here, in the second and third inventions, the object detection algorithm is preferably YOLO or SSD. In the second and third inventions, a surface determination step may be further provided. In the surface determination step, the vehicle constituent surface in the captured image is determined with reference to the surface learning model. The surface learning model is constructed by supervised learning using data on the constituent surfaces of the vehicle as teacher data. In this case, it is preferable that the component determining step filters the vehicle component based on the constituent surface determined in the surface determining step, and outputs the filtered vehicle component as a determination result. Further, in the surface determination step, when there is a vehicle component unique to the specific component surface in the captured image, the component surface corresponding to the specific vehicle component may be included in the determination result.

In the second and third inventions, an estimate calculation step may be further provided. In the estimation calculation step, the repair cost of the vehicle is estimated from the damage state of each vehicle component determined in the state determination step by referring to the estimation table. The estimation table holds a repair cost associated with a damaged state of a vehicle component for each vehicle component.

In the second and third inventions, a transition of processing from the surface determination step to the component determination step, a transition of processing from the component determination step to the state determination step, and a transition of processing from the state determination step to the estimation calculation step At least one of them is preferably performed on condition that the user's approval is obtained while allowing the user to correct the determination result.

According to the present invention, it is possible to flexibly deal with various vehicles including an unknown vehicle by determining the damage state of the vehicle based on machine learning. At this time, the processing is separated into the component determination and the damage determination, and the damage determination of each component is performed after the component determination is performed, so that the determination accuracy as a whole can be improved. In addition, as for component determination in which many vehicle components can be detected as an object in a captured image, the captured image is input to a single neural network system as represented by YOLO or SSD, and a regression problem approach is used. By adopting the object detection algorithm that collectively extracts the area of the vehicle component with the attribute classification, the processing speed can be increased.

Block diagram of the vehicle state determination device Diagram showing a screen display example of image reception Figure showing a screen display example of the surface determination result Illustration of the object detection algorithm YOLO network configuration diagram Diagram showing a screen display example of the vehicle part determination result Figure showing a screen display example of the damage state determination result Schematic configuration diagram of quote table The figure which shows the example of the screen display of the estimation result

FIG. 1 is a block diagram of the vehicle state determination device 1 according to the present embodiment. The vehicle state determination device 1 determines a damage state of a vehicle from a captured image specified by a user, and presents an approximate cost required for vehicle repair to the user. In the present embodiment, the vehicle to be determined is assumed to be a privately-owned vehicle, but this is an example, and any vehicle including a truck, a bus, a motorcycle, etc. may be targeted depending on design specifications. Can be. The vehicle state determination device 1 can be equivalently realized by installing a computer program (form layout analysis program) in a computer.

The vehicle state determination device 1 includes an image receiving unit 2, a surface determination unit 3, a component determination unit 4, a state determination unit 5, a estimation calculation unit 6, an input / output interface 7, learning models 8 to 10, The estimation table 11 is mainly configured. Each of the processing units 2 to 6 is connected to a display device 12 via an input / output interface 7, and the input / output interface 7 controls input and output between the display unit 12 and the input / output interface. Basically, each processing in the processing units 3 to 6 is sequentially performed. However, the transition of the processing between adjacent processing units, specifically, the transition from the surface determination unit 3 to the component determination unit 4, the component determination The transition from the unit 4 to the state determination unit 5 and the transition from the state determination unit 5 to the estimation calculation unit 6 are performed on condition that the user's approval is obtained after allowing the user to correct the determination result. Done. This is to improve the overall determination accuracy by appropriately reflecting the user's intention in the process. However, part of the process transition may be performed automatically without the condition of the user's approval. When the display device 12 is connected to a network such as the Internet, the input / output interface 7 has a communication function required for performing network communication.

The image receiving unit 2 receives a captured image to be determined from the display device 12, specifically, an image obtained by capturing the external appearance of a damaged vehicle with a camera. The direction in which the vehicle is imaged may be any of the front, rear, and side, and may be diagonally forward, diagonally rear, or the like.

The captured image to be determined is specified by the user operating the display device 12 while viewing the screen, and is output / uploaded to the vehicle state determination device 1. FIG. 2 is a diagram illustrating an example of a screen display of image reception on the display device 12. The display screen 30 for receiving an image has an image receiving area 31. In the image receiving area 31, a thumbnail of a captured image to be determined is displayed. The user designates a determination target by designating a specific image file through the file reference button or by dropping a specific image file in the image receiving area 31. As can be understood from the existence of a plurality of blank frames in the image receiving area 31, a plurality of determination targets can be specified, and even if the determination target has already been specified, it can be canceled through the cancel button. When the specification of the determination target is completed, the user presses the determination start button. By this action, the determination target is output to the image receiving unit 2. The captured image received by the image receiving unit 2 is output to the surface determination unit 3.

The captured image to be determined may be a color image, but a grayscale image may be received in order to reduce the amount of memory used. The size of the captured image is appropriately set in consideration of the memory usage of the system and the like.

The surface determination unit 3 determines the constituent surface of the vehicle shown in the captured image to be determined. Here, the “configuration surface” refers to an individual surface configuring a three-dimensional vehicle, and includes a front surface, a back surface, a side surface, and the like of the vehicle. For example, in the case of a captured image of a vehicle captured from the front, the determination result is the front, and in the case of a captured image of the vehicle captured from the rear, the determination result is the back. However, the number of constituent surfaces obtained as a result of the determination is not limited to one, but may be plural. For example, in the case of a picked-up image of a vehicle taken diagonally from the front, the determination results are front and side. The reason for performing such a configuration surface determination is to improve the determination accuracy in the post-processing.

面 In the determination of the constituent surface, the surface learning model 8 is referred to. The surface learning model 8 is mainly composed of a neural network system imitating a human brain nerve. The neural network system is formed on a work area of a computer, and has an input layer, a hidden layer, and an output layer. When transmitting the input signal to the hidden layer, the input layer is weighted by the activation function. Then, transmission with weighting according to the number of hidden layers is repeated, and the signal transmitted to the output layer is finally output. The number of ports in the input layer, the number of ports in the output layer, the number of layers in the hidden layer, and the like are arbitrary. The output layer also outputs the classification probabilities of the outputs (front, back, side, etc.).

The construction of the plane learning model 8 is performed by supervised learning using data on the constituent planes of the vehicle as teacher data. The teacher data includes a captured image of the vehicle and classification of the constituent surfaces of the vehicle, and various and large amounts of data are used, including various types of vehicles, various body colors, various shooting directions, and the like. In the supervised learning, the classification probability of the output with respect to the input of the teacher data is verified, and the adjustment of the activation function (weighting) is repeatedly performed based on the verification to set the surface learning model 8 to a desired state.

As the surface learning model 8, a machine learning method such as a support vector machine, a decision tree, a Bayesian network, a linear regression, a multivariate analysis, a logistic regression analysis, and a judgment analysis may be used in addition to the neural network. Also, a convolutional neural network and R-CCN (Regions @ with @ CNN @ features) using the same may be used. This is the same for the state learning model 10 described later.

In the determination of the component surface, in addition to the reference to the surface learning model 8, the component surface may be estimated by focusing on the presence / absence of a unique vehicle component existing on the specific component surface. For example, front lights and front grilles are unique to the front of the vehicle and have a characteristic shape that can be distinguished from other vehicle parts. Can be considered included. Therefore, when a vehicle component unique to a specific component surface exists in a captured image, a component surface corresponding to the specific component component is included in the determination result.

{Furthermore, the surface learning model 8 may be reduced in weight in order to reduce the memory usage of the GPU (Graphics Processing Unit) or to improve the learning efficiency. As an example, reduction of the number of VGG blocks (one block by convolution → convolution → pooling) in a general-purpose model such as VGG-16 and reduction of the size of a captured image can be cited. VGG-16 is a method of using a trained model for a CNN (convolutional neural network), that is, one of transfer learning models, including a total of 16 layers of a convolutional layer and a fully connected layer, and a size of a convolution filter. All layers are 3 × 3, the total bonding layer is composed of two layers of 4096 units, and one unit of 1000 units for class classification.

The determination result of the surface determination unit 3 is output to the display device 12. FIG. 3 is a diagram illustrating a screen display example of a surface determination result displayed on the display device 12. The determination result display screen 40 has an image display area 41, a plurality of determination result display areas 42, and a plurality of check boxes 43. In the image receiving area 41, a captured image of the vehicle related to the determination target is displayed. In each of the determination result display areas 42, candidates (front, side, and back) of the constituent surface of the vehicle are displayed with classification probabilities. The check boxes 43 are provided corresponding to the respective determination result display areas 42, and those satisfying predetermined conditions are marked with check marks. The predetermined condition is that the classification probability of the component surface candidate is equal to or higher than a predetermined threshold value (in this case, a plurality of check marks may be added), and the component surface classification probability is the highest. And the like. When the user determines that the determination result is valid, the user presses the determination continuation button. If the user determines that this is not appropriate, the user corrects the determination result including the change of the tick mark of the check box 43, and then presses the determination continue button. By this action, the determination result of the surface determination unit 3 (including the determination result corrected by the user) is output to the component determination unit 4 together with the captured image.

The determination result corrected by the user may be reflected on the surface learning model 8. Thereby, the learning depth in the determination of the constituent surface can be increased.

(4) The component determination unit 4 individually extracts a vehicle component extracted in a captured image, specifically, a component region in which the vehicle component is extracted and an attribute of the vehicle component. In this component determination, the component learning model 9 is referred to. The learning model 9 is constructed based on an object detection algorithm based on deep learning, such as YOLO or SSD, in consideration of multi-scale performance and operation speed.

FIG. 4 is an explanatory diagram of the object detection algorithm. As shown in FIG. 1A, in a conventional detection method used for face detection or the like, processing for an input is divided into three stages: area search, feature extraction, and machine learning. That is, first, an area search is performed, then feature extraction is performed according to an object to be detected, and finally, an appropriate machine learning method is selected. In this detection method, the object detection is realized by being divided into three algorithms. As for the feature amount, basically, only a specific target can be detected because it is designed exclusively for the detection target. Therefore, in order to eliminate such a restriction, an object detection algorithm based on deep learning as shown in FIGS. As shown in FIG. 2B, in R-CNN and the like, feature amount extraction is automatically realized by using deep learning. This enables flexible classification of various objects by simply designing the network. However, the area search still remains as a separate process. Therefore, the method shown in FIG. 3C, which is represented by YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector), includes the area search in the deep learning. In this method, by inputting an input (captured image) to a single neural network, extraction of an item region and classification of its attribute are collectively performed. The first feature of this method is that it is a regression problem approach. Regression is an approach that predicts numerical values directly from trends in data. Instead of determining an area and then classifying it, the coordinates and size of an object are directly predicted. Second, the processing is completed on a single network. After data is input, it can be said to be “End-to-End” processing, meaning that it goes to the end (output result) only by deep learning. One of the features of the present embodiment is that a method shown in FIG. 3C typified by YOLO or SSD is used for component determination in which many vehicle components can be detected as objects in a captured image. .

For example, the YOLO process is generally as follows. First, the input image is divided into S × S areas. Next, the classification probability of the object in each area is derived. Then, the parameters (x, y, height, width) and the reliability (confidence) of the B (hyperparameter) bounding boxes are calculated. The bounding box is a circumscribed rectangle of the object area, and the reliability is the degree of coincidence between the prediction and the correct bounding box. For the object detection, a product of the classification probability of the object and the reliability of each bounding box is used. FIG. 5 is a network configuration diagram of YOLO. In YOLO, a form image is input to a CNN (Convolutional Neural Network) layer, and the result is output through a plurality of fully connected layers. The output includes the image area divided into S * S pieces, five parameters of a bounding box (BB) including the reliability (classification accuracy), and the number of classes (attributes of items).

The construction of the part learning model 9 is performed by supervised learning using data on vehicle parts for each constituent surface of the vehicle as teacher data. The teacher data has a partial image of the vehicle part and the attribute of the vehicle part, and various and large amounts of data including various types of vehicles, various vehicle parts, various photographing directions, and the like are used. In order to secure a large amount of data, a source image that has been subjected to image processing is also used. However, in order to distinguish the right head lamp, the left head lamp, and the like, the image is not horizontally inverted, which is one of image processing. By using such a part learning model 9, it is possible to determine and extract the vehicle parts included in the constituent surface of the vehicle regardless of the presence or absence of the damage of the vehicle.

(4) The component determination unit 4 filters the vehicle component specified based on the component learning model 9 based on the configuration surface of the vehicle determined by the surface determination unit 3, and outputs the filtered vehicle component as a determination result. For example, when the determination result of the component surface is the front surface and the side surface, it is obvious that even if a tail lamp is extracted as a result of the component determination, this is an erroneous determination. This is because the tail lamp is present at the back and cannot be present at the front and side. Such a correlation is also recognized for a side door unique to a side surface, a headlight unique to a front surface, a front grill, a front window, and the like. Therefore, of the vehicle parts obtained as a result of the component determination, only the components related to the component surface obtained as the component surface determination result are included in the determination result, and other components are excluded, thereby improving the component determination accuracy. Can be planned.

(4) The determination result of the component determination unit 4 is output to the display device 12. FIG. 6 is a diagram illustrating a screen display example of the component determination result. The determination result display screen 50 has an image display area 51, a plurality of determination result display areas 52, and a plurality of check boxes 53. In the image display area 51, a captured image of the vehicle related to the determination target is displayed, and a rectangular frame indicating each vehicle component extracted by the component determination unit 4 is displayed. In the determination result display area 52, respective vehicle component candidates (right headlight, left headlight, front window, etc.) are displayed with classification probabilities. Check boxes 53 are provided corresponding to the respective determination result display areas 52, and those satisfying predetermined conditions are marked with check marks. Typically, the predetermined condition is that the classification probability of the constituent surface candidate is equal to or higher than a predetermined threshold. When the user determines that the determination result is valid, the user presses the determination continuation button. When the user determines that this is not appropriate, the user corrects the determination result including the change of the tick mark of the check box 53, and then presses the determination continue button. With this action, the determination result of the component determination unit 4 (including the determination result corrected by the user) is output to the state determination unit 5 together with the captured image.

The determination result corrected by the user may be reflected on the component learning model 9. This makes it possible to increase the learning depth of component determination.

The state determination unit 5 determines the damage state of each vehicle component extracted by the part determination unit 4. This damage state determination is performed by referring to a state learning model 10 constructed by supervised learning using teacher data on the damaged state of the vehicle component. The configuration of the state learning model 10 is basically the same as that of the surface learning model 8.

{Circle around (2)} The construction of the state learning model 10 is performed by supervised learning using data on the damaged state of the vehicle parts as teacher data. The teacher data includes a partial image of a damaged vehicle part and attributes (eg, “replacement”, “removal”, damage degree “large”, “medium”, “small”) of the damage state. Various and large amounts of data are used, including various types of vehicles, various vehicle parts, various photographing directions, and the like. Here, "replacement" means such damage that the vehicle parts themselves must be replaced without repair. "Removable" means that the vehicle part itself is undamaged, but must be temporarily removed from the vehicle body for replacement or repair of another damaged vehicle part. In addition, in order to secure a large amount of data, a source image that has been subjected to image processing may be used. By using such a state learning model 10 to determine a damaged state for each vehicle component, it is possible to improve the accuracy of determining the damaged state.

In the present embodiment, a network in which only the fully connected layer is replaced based on the above-described general-purpose model such as VGG-16 is used as the state determination unit 5. In addition, since the data amount varies greatly for each vehicle component, fine tuning using weights learned in advance by ImageNet may be performed in order to perform learning with a small data amount. Fine tuning is a method of reusing a part of an existing model to construct a new model. Furthermore, in order to cope with the case where the image of a slight pull and the image of the close-up and a different type of image are mixed for each data, the clean-up by selecting the close-up image and trimming the target portion may be performed (trimming). Some used images are used).

判定 The judgment result of the state judgment unit 5 is output to the display device 12. FIG. 7 is a diagram illustrating an example of a screen display of the result of the determination of the damage state displayed on the display device 12. The determination result display screen 60 includes an image display area 61, a plurality of determination result display areas 62, and a plurality of check boxes 63. In the image display area 61, a captured image of the vehicle as the determination target is displayed, and a round frame indicating the damaged part determined by the state determination unit 5 is displayed. In the determination result display area 62, candidates for the damage state (replacement, detachment, large, medium, small) are displayed with classification probabilities. The check boxes 63 are provided corresponding to the respective determination result display areas 62, and those satisfying predetermined conditions are marked with check marks. The predetermined condition typically includes that the classification probability of the candidate for the degree of damage is equal to or higher than a predetermined threshold. When determining that the determination result is appropriate, the user presses the estimation start continuation button. When the user determines that this is not appropriate, the user corrects the determination result including the change of the tick mark of the check box 63, and then presses the estimation start button. By this action, the determination result of the state determination unit 5 (including the determination result corrected by the user) is output to the estimation calculation unit 6 together with the captured image.

The determination result corrected by the user may be reflected on the state learning model 10. Thereby, the learning depth of the damage state determination can be increased.

The estimate calculation unit 6 estimates the cost required for repairing the vehicle to be determined by referring to the estimate table 11. FIG. 8 is a schematic configuration diagram of the estimation table 11. The estimation table 11 holds a vehicle part name, a degree of damage, and a cost including a labor cost in association with each other. The cost is specified for each vehicle part by searching the estimation table 11 using the vehicle part specified by the part determination unit 4 and the damaged state of the vehicle part determined by the state determination unit 5 as keys. The cost required for vehicle repair is the sum of the costs for each vehicle component. For example, when the component determination unit 4 extracts “front bumper” and “headlight”, and the state determination unit 5 determines that the damage degree of the front vehicle is “medium” and the damage degree of the latter is “small”, The wage is "¥ 200,000", the latter is "¥ 30,000", and the total amount is "¥ 230,000".

Note that if the vehicle type is specified by the analysis of the captured image, or if the vehicle type is known, such as when the vehicle type is specified by the user, the design is calculated assuming that the vehicle type is known. As shown in (1), the model of the estimation table 11 may include a vehicle type (for example, “ABC”). In this way, by setting the cost finely for each vehicle type, the estimated amount of money according to the vehicle type can be accurately calculated.

The estimation result of the estimation calculation unit 6 is output to the display device 12. FIG. 9 is a diagram illustrating a screen display example of the estimation result displayed on the display device 12. The estimation result display screen 70 has an image display area 71 and an estimation result display area 72. In the image display area 71, a captured image of the vehicle related to the determination target is displayed. In the estimation result display area 72, the damaged vehicle parts (parts), the degree of damage, and the cost are displayed. If there are multiple damaged vehicle parts, the total cost is displayed along with the individual costs. When the user presses the end button, a series of processes in the vehicle state determination device 1 ends.

As described above, according to the present embodiment, it is possible to flexibly deal with various vehicles including an unknown vehicle by determining the damage state of the vehicle based on machine learning. At this time, the processing is separated by the component determination unit 4 and the state determination unit 5, and the damage state is determined for each vehicle component after the component determination is performed. In comparison, the determination accuracy as a whole can be improved.

Further, according to the present embodiment, for the component determination in which many vehicle components can be detected as an object in the captured image, the captured image is input to a single neural network system as represented by YOLO or SSD. In addition, an object detection algorithm that collectively extracts vehicle parts with attribute classification by a regression problem approach is adopted. This makes it possible to speed up the processing in the component determination unit 4.

According to the present embodiment, prior to the process of the component determination unit 4, the surface determination unit 2 specifies the constituent surface of the vehicle in the captured image, and then performs the component determination on the captured image. As a result, among the vehicle parts obtained as a result of the determination by the part determination unit 4, those that cannot exist on the constituent surface specified by the surface determination unit 2 can be excluded as erroneous determinations, thereby improving the accuracy of the part determination. Can be achieved.

Furthermore, according to the present embodiment, the cost required for vehicle repair is presented to the user by providing the estimation calculation unit 6 for estimating the repair cost of the vehicle from the damage state of each vehicle component determined by the state determination unit 5. I do. Thereby, the convenience for the user can be further improved.

DESCRIPTION OF SYMBOLS 1 Vehicle state determination apparatus 2 Image receiving part 3 Surface determination part 4 Parts determination part 5 State determination part 6 Estimation calculation part 7 I / O interface 8 Surface learning model 9 Parts learning model 10 State learning model 11 Estimation table 12 Display device

Claims (18)

In a vehicle state determination device that determines a damage state of a vehicle from a captured image,
A component determination unit that determines a vehicle component in a captured image with reference to a component learning model constructed by supervised learning using data on a vehicle component as teacher data using an object detection algorithm based on deep learning,
A state determination unit that determines a damage state of each vehicle component determined by the component determination unit with reference to a state learning model constructed by supervised learning using data on the damage state of the vehicle part as teacher data. And
The object detection algorithm,
A vehicle state determination characterized in that, by inputting the captured image to a single neural network system, region extraction of the vehicle components in the captured image is collectively performed with attribute classification by a regression problem approach. apparatus. The vehicle state determination device according to claim 1, wherein the object detection algorithm is YOLO or SSD. Reference is made to a surface learning model constructed by supervised learning using data on the constituent surfaces of the vehicle as teacher data, further comprising a surface determination unit for determining the constituent surfaces of the vehicle in the captured image,
The said parts determination part filters the said vehicle parts based on the said component surface determined by the said surface determination part, and outputs the filtered vehicle parts as a determination result, The claim 1 characterized by the above-mentioned. Vehicle state determination device. The said surface determination part, When the vehicle component peculiar to a specific component surface exists in the said picked-up image, the component surface corresponding to the said specific vehicle component is included in a determination result, The claim 4 characterized by the above-mentioned. Vehicle state determination device. The repair cost associated with the damage state of the vehicle part is estimated by referring to the estimation table held for each vehicle part, and the vehicle repair cost is estimated from the damage state of each vehicle part determined by the state determination unit. The vehicle state determination device according to any one of claims 1 to 4, further comprising an estimation calculation unit. At least one of a transition of processing from the surface determination unit to the component determination unit, a transition of processing from the component determination unit to the state determination unit, and a transition of processing from the state determination unit to the estimation calculation unit 6. The vehicle state determination device according to claim 1, wherein the determination is performed on condition that the user's approval is obtained after allowing the user to correct the determination result. In the vehicle state determination program for determining the damage state of the vehicle from the captured image,
A component determination step of determining a vehicle component in a captured image by referring to a component learning model constructed by supervised learning using data on the vehicle component as teacher data using an object detection algorithm by deep learning,
A state determination step of determining a damage state of each vehicle component determined by the component determination unit with reference to a state learning model constructed by supervised learning using data on the damage state of the vehicle component as teacher data. Let the computer execute the process,
The object detection algorithm,
A vehicle state determination characterized in that, by inputting the captured image to a single neural network system, region extraction of the vehicle components in the captured image is collectively performed with attribute classification by a regression problem approach. program. The vehicle state determination program according to claim 7, wherein the object detection algorithm is YOLO or SSD. With reference to a surface learning model constructed by supervised learning using data on the constituent surfaces of the vehicle as teacher data, further comprising a surface determination step of determining a constituent surface of the vehicle in the captured image,
The said parts determination step filters the said vehicle parts based on the said component surface determined by the said surface determination step, and outputs the filtered vehicle parts as a determination result, The claim 7 characterized by the above-mentioned. Vehicle state determination program. The said surface determination step WHEREIN: When the vehicle part peculiar to a specific component surface exists in the said picked-up image, the component surface corresponding to the said specific vehicle component is included in a determination result. Vehicle state determination program. The repair cost associated with the damage state of the vehicle part is estimated by referring to the estimation table held for each vehicle part, and the vehicle repair cost is estimated from the damage state of each vehicle part determined in the state determination step. The vehicle state determination program according to any one of claims 7 to 10, further comprising an estimate calculation step. At least one of a transition of processing from the surface determination step to the component determination step, a transition of processing from the component determination step to the state determination step, and a transition of processing from the state determination step to the estimation calculation step 12. The vehicle state determination program according to claim 7, wherein the determination is performed on condition that the user's approval is obtained while allowing the user to correct the determination result. In a vehicle state determination method for determining a damage state of a vehicle from a captured image,
A component determination step of determining a vehicle component in a captured image by referring to a component learning model constructed by supervised learning using data on the vehicle component as teacher data using an object detection algorithm by deep learning,
A state determination step of determining a damage state of each vehicle component determined by the component determination unit with reference to a state learning model constructed by supervised learning using data on a damaged state of the vehicle component as teacher data. And
The object detection algorithm,
A vehicle state determination characterized in that, by inputting the captured image to a single neural network system, region extraction of the vehicle components in the captured image is collectively performed with attribute classification by a regression problem approach. Method. 14. The method according to claim 13, wherein the object detection algorithm based on the deep learning is YOLO or SSD. With reference to a surface learning model constructed by supervised learning using data on the constituent surfaces of the vehicle as teacher data, further comprising a surface determination step of determining a constituent surface of the vehicle in the captured image,
14. The component determination step according to claim 13, wherein the vehicle component is filtered based on the constituent surface determined in the surface determination step, and the filtered vehicle component is output as a determination result. Vehicle state determination method. 16. The method according to claim 15, wherein, when the captured image includes a vehicle component unique to a specific component surface in the captured image, a component surface corresponding to the specific vehicle component is included in the determination result. Vehicle state determination method. The repair cost associated with the damage state of the vehicle part is estimated by referring to the estimation table held for each vehicle part, and the vehicle repair cost is estimated from the damage state of each vehicle part determined in the state determination step. 17. The method according to claim 13, further comprising an estimate calculating step. At least one of a transition of processing from the surface determination step to the component determination step, a transition of processing from the component determination step to the state determination step, and a transition of processing from the state determination step to the estimation calculation step 18. The vehicle state determination method according to claim 13, wherein the determination is performed on condition that the user's approval is obtained while allowing the user to correct the determination result.

Images