JP7060841B2 - Operation evaluation device, operation evaluation method, and operation evaluation program - Google Patents

Description

The present invention relates to an operation evaluation device, an operation evaluation method, and an operation evaluation program.

Patent Document 1 discloses an inattentive detection system that detects an inattentiveness of a driver driving a vehicle and performs an operation for causing the driver to avoid the danger of inattentiveness. The inattentive detection system described in Patent Document 1 includes an image pickup device and an inattentive detection device.

The image pickup device is installed at a position where the driver's face can be photographed substantially from the front, and the photographed image (face image) is supplied to the inattentive detection device. The inattentive detection device includes a line-of-sight detection unit, an inattentive detection unit, and a vehicle control unit.

The line-of-sight detection unit detects the direction of the driver's line of sight based on the face image. The inattentive detection unit detects inattentiveness in the vertical and horizontal directions of the driver based on the direction of the driver's line of sight. By controlling the operation of the own vehicle and various in-vehicle devices provided in the own vehicle, the vehicle control unit calls the driver to pay attention to inattentiveness with a strength according to the degree of danger.

[Problems to be solved by the invention]
In the inattentive detection system described in Patent Document 1, the image pickup device is installed at a position where the driver's face can be photographed substantially from the front. Therefore, the image pickup device can detect the driver's left and right line-of-sight directions in the same range on the left and right.

By the way, in such a system, the image pickup device may not be installed near the front of the driver. For example, the image pickup device may be installed near the center of the dashboard. When the driver of a right-hand drive vehicle is imaged from this position, the image pickup device is located diagonally to the left of the driver. Become. In the case of inattentive detection, even if the detection range of the face direction in the right direction is narrower to some extent than in the left direction, it is sufficient if it can be determined whether or not the direction of the line of sight exceeds a predetermined range. The effect on is relatively small.

However, when the driver's safety confirmation operation at an intersection is detected instead of inattentive detection, the effect of narrowing the detection range of the face direction in the right direction as compared with the left direction becomes large.

For example, when the driver confirms the safety in the right direction (from the front right to the rear right) when turning right at an intersection, the driver turns his face to the right. In particular, the longer the vehicle length, such as a truck or bus, the larger the driver's face swing angle to the right. Furthermore, depending on traffic conditions, the driver may stick his head out of the driver's seat window to check the rear right. Therefore, the image pickup device may not be able to properly detect the driver's face during safety confirmation.

In such a case, the inattentive detection device described in Patent Document 1 cannot appropriately determine whether or not the phenomenon that the driver's face is not detected by the image pickup device is due to safety confirmation, and the driver performs safety confirmation. However, there is a problem that there is a possibility that a process such as erroneously determining that the driver is looking aside and calling attention to the driver may be performed.

Japanese Patent No. 5082834

Means for solving problems and their effects

The present invention has been made in view of the above problems, and provides a driving evaluation device, a driving evaluation method, and a driving evaluation program capable of more appropriately determining whether or not the driver has confirmed safety at an intersection. The purpose is.

In order to achieve the above object, the driving evaluation device (1) according to the present disclosure is a driving evaluation device that evaluates the driving of the driver of the vehicle.
A data acquisition unit that acquires data on a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected while the vehicle passes through the intersection. When,
Whether the face non-detection state is due to the driver's safety confirmation operation based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired by the data acquisition unit. A judgment unit that determines whether or not it is
It is characterized by including an evaluation unit for evaluating the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination unit.

According to the operation evaluation device (1), the face non-detection state is the face non-detection state based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired by the data acquisition unit. It is determined whether or not it is due to the driver's safety confirmation operation. Therefore, when the face non-detection state is detected while the vehicle passes through the intersection, it is possible to appropriately determine whether or not the face non-detection state is due to the driver's safety confirmation operation. It is possible to accurately evaluate the safety confirmation operation of the driver at the intersection.

Further, in the operation evaluation device (2) according to the present disclosure, in the operation evaluation device (1), the determination unit determines the orientation of the driver's face in at least one of the face detection states before and after the face non-detection state. It is characterized in that it is determined whether or not the face non-detection state is due to the safety confirmation operation of the driver in consideration of the temporal change of the above.

According to the driving evaluation device (2), in the determination unit, the face is taken into consideration in consideration of the temporal change of the face orientation of the driver in at least one of the face detection states before and after the face non-detection state. It is determined whether or not the non-detection state is due to the driver's safety confirmation operation. Therefore, by considering the temporal change of the face orientation of the driver in at least one of the face detection states before and after the face non-detection state, the face non-detection state is due to the driver's safety confirmation operation. It is possible to more accurately determine whether or not the safety confirmation is performed, and it is possible to more accurately evaluate the safety confirmation operation of the driver at the intersection.

Further, in the operation evaluation device (3) according to the present disclosure, in the operation evaluation device (1) or (2), the data acquired by the determination unit and the data acquisition unit can be obtained from the face detection state to the face. When it indicates that the face has not been detected and the face has been detected again within a predetermined time, it is determined that the face non-detection state is due to the driver's safety confirmation operation. There is.

According to the operation evaluation device (3), the data acquired by the data acquisition unit changes from the face detection state to the face non-detection state and returns to the face detection state again within a predetermined time. When it is shown, it is determined that the face non-detection state is due to the safety confirmation operation of the driver. Therefore, it is possible to more accurately determine whether or not the face non-detection state is due to the driver's safety confirmation operation while the vehicle passes through the intersection, and it is possible to more accurately determine whether or not the driver's safety confirmation operation at the intersection. The operation can be evaluated more accurately.

Further, in the operation evaluation device (4) according to the present disclosure, in the operation evaluation device (1) or (2), the data acquired by the determination unit and the data acquisition unit can be obtained from the face detection state to the face. When it indicates that the face has not been detected and the face has not been detected again within a predetermined time, it is determined that the face non-detection state is not due to the driver's safety confirmation operation. It is supposed to be.

According to the operation evaluation device (4), the data acquired by the data acquisition unit changed from the face detection state to the face non-detection state and did not return to the face detection state again within a predetermined time. Is indicated, it is determined that the face non-detection state is not due to the driver's safety confirmation operation. Therefore, even if the face non-detection state is not due to the driver's safety confirmation operation, it can be appropriately determined, and the driver's safety confirmation operation at the intersection can be accurately evaluated.

Further, in the operation evaluation device (5) according to the present disclosure, in any of the operation evaluation devices (1) to (3), the evaluation unit is used by the determination unit, and the face non-detection state is the safety of the driver. When it is determined that the operation is due to the confirmation operation, the driver is evaluated as having performed the safety confirmation operation in the face non-detection state.

According to the driving evaluation device (5), when the determination unit determines that the face non-detection state is due to the driver's safety confirmation operation, the driver's safety confirmation in the face non-detection state. It can be properly evaluated as if the operation was performed.

Further, the driving evaluation method according to the present disclosure is a driving evaluation method for evaluating the driving of the driver of the vehicle.
A data acquisition step for acquiring data regarding a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected while the vehicle passes through the intersection. When,
Whether the face non-detection state is due to the driver's safety confirmation operation based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired by the data acquisition step. Judgment step to determine whether or not,
It is characterized by including an evaluation step for evaluating the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination step.

According to the driving evaluation method, the face non-detection state is the driver's based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired by the data acquisition step. Judge whether it is due to the safety confirmation operation. Therefore, when the face non-detection state is detected while the vehicle passes through the intersection, it is possible to appropriately determine whether or not the face non-detection state is due to the driver's safety confirmation operation. It is possible to accurately evaluate the safety confirmation operation of the driver at the intersection.

Further, the driving evaluation program according to the present disclosure is a driving evaluation program for causing at least one computer to execute a process of evaluating the driving of the driver of the vehicle.
To the at least one computer
A data acquisition step for acquiring data regarding a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected while the vehicle passes through the intersection. When,
Whether the face non-detection state is due to the driver's safety confirmation operation based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired by the data acquisition step. Judgment step to determine whether or not,
It is characterized in that an evaluation step for evaluating the safety confirmation operation of the driver at the intersection is executed in consideration of the determination result by the determination step.

According to the driving evaluation program, the face non-detection state is the driver's based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired by the data acquisition step. Judge whether it is due to the safety confirmation operation. Therefore, when the face non-detection state is detected while the vehicle passes through the intersection, it is possible to appropriately determine whether or not the face non-detection state is due to the driver's safety confirmation operation. It is possible to realize a device capable of accurately evaluating the safety confirmation operation of the driver at the intersection.

It is a schematic block diagram which shows an example of the operation evaluation system to which the operation evaluation apparatus which concerns on embodiment is applied. It is a block diagram which shows the main part of the in-vehicle device used in the operation evaluation system which concerns on embodiment. It is a figure which shows an example of the structure of the data transmitted from an in-vehicle device to a server device. It is a functional block diagram which shows the characteristic part of the server apparatus which concerns on embodiment. It is a flowchart which shows the processing operation performed by the driver monitoring unit of the vehicle-mounted device which concerns on embodiment. It is a flowchart which shows the processing operation performed by the control unit of the vehicle-mounted device which concerns on embodiment. It is a flowchart which shows the processing operation performed by the central processing unit of the server apparatus which concerns on embodiment. It is a flowchart which shows the processing operation of the safety confirmation evaluation performed by the central processing unit of the server apparatus which concerns on embodiment. It is a flowchart which shows the processing operation of the safety confirmation evaluation performed by the central processing unit of the server apparatus which concerns on embodiment.

Hereinafter, embodiments of an operation evaluation device, an operation evaluation method, and an operation evaluation program according to the present invention will be described with reference to the drawings.

[Application example]
FIG. 1 is a schematic configuration diagram showing an example of an operation evaluation system to which the operation evaluation device according to the embodiment is applied.
The driving evaluation system 1 is a system for evaluating the driving of the driver 3 of the vehicle 2, and is a system in which the on-board unit 10 mounted on at least one or more vehicles 2 and the data acquired by each on-board unit 10 are used. It is configured to include at least one or more server devices 40 to be processed. The server device 40 is an example of the "operation evaluation device" according to the present invention.

The vehicle 2 on which the vehicle-mounted device 10 is mounted is not particularly limited. In this application example, a vehicle managed by a business operator engaged in various businesses may be targeted. For example, trucks managed by transportation companies, buses managed by bus companies, taxis managed by taxi companies, car sharing vehicles managed by car sharing companies, car rentals managed by car rental companies, owned by companies. It may be a company-owned vehicle that is owned by a company or a company-owned vehicle that is leased and used by a car leasing company.

The vehicle-mounted device 10 and the server device 40 are configured to be able to communicate with each other via the communication network 4. The communication network 4 may include a wireless communication network such as a mobile phone network (3G / 4G) including a base station or a wireless LAN (Local Area Network), a wired communication network such as a public telephone network, the Internet, or a dedicated network. It may include a telecommunication line such as.

Further, the terminal device 80 of the business operator that manages the vehicle 2 (hereinafter referred to as the business operator terminal) is configured to be able to communicate with the server device 40 via the communication network 4. The business terminal 80 may be a personal computer having a communication function, or may be a mobile information terminal such as a mobile phone, a smartphone, or a tablet device. Further, the operator terminal 80 may be configured to be able to communicate with the vehicle-mounted device 10 via the communication network 4.

The on-board unit 10 monitors the state of the driver 3 of the vehicle 2 and acquires data used for the driving evaluation of the driver 3. For example, the acquired data when a predetermined event is detected is transferred to the server device 40. It is a device to transmit.
The on-board unit 10 includes a platform unit 11 that performs processing for detecting the position of the vehicle 2 and the behavior of the vehicle 2, and a driver monitoring unit 20 that captures an image of the driver 3 and detects the state of the driver 3. It is configured to include. The data indicating the state of the driver 3 includes face image information such as the direction of the face of the driver 3 detected from the image of the driver 3. Further, a drive recorder unit 30 for photographing the outside and the inside of the vehicle 2 is connected to the in-vehicle device 10.

The on-board unit 10 has a compact configuration in which the platform unit 11 and the driver monitoring unit 20 are housed in one housing. The location of the vehicle-mounted device 10 in the vehicle is not particularly limited as long as the image pickup unit of the driver monitoring unit 20 can image at least the field of view including the driver's face. For example, the vehicle-mounted device 10 may be installed near the center of the dashboard of the vehicle 2, but may be installed at the steering wheel column portion, the vicinity of the meter panel, the position near the rearview mirror, the A pillar portion, or the like.

When the vehicle-mounted device 10 detects that the vehicle 2 has passed the intersection (for example, a right turn or a left turn), the driver monitoring unit 20 and the platform unit 11 detect the vehicle 2 while passing the intersection. A process of transmitting data including the state of the driver 3, the position of the vehicle 2, and the behavior of the vehicle 2 to the server device 40 is executed.

The server device 40 includes a communication unit 41, a server computer 50, and a storage unit 60, and these are connected via a communication bus 42.

The communication unit 41 is composed of a communication device for realizing transmission / reception of various data and signals to and from the vehicle-mounted device 10 and the business terminal 80 via the communication network 4.

The server computer 50 is configured to include a central processing unit 51 including one or more CPUs (Central Processing Units) and a main memory 53 in which the control program 52 is stored. The central processing unit 51 executes various processes according to the control program 52 in the main memory 53. The control program 52 includes an example of the "operation evaluation program" according to the present invention.

The storage unit 60 is composed of one or more large-capacity storage devices such as a hard disk drive and a solid state drive, and stores information of the data storage unit 61, the evaluation data storage unit 62, and the driver 3. It is configured to include a storage unit 63.

The server device 40 acquires data including the state of the driver 3, the position of the vehicle 2, and the behavior of the vehicle 2, which are transmitted from each on-board unit 10 when a predetermined event is detected, and is a storage unit. It is stored in the data storage unit 61 of 60.

The server device 40 uses the data of each on-board unit 10 stored in the data storage unit 61 to evaluate the operation of each driver 3. The driving evaluation item includes an evaluation regarding the driver's safety confirmation operation at an intersection, but may further include an evaluation regarding the driver's driving operation behavior, an evaluation regarding a concentration state during driving, and the like. good.

The server device 40 may execute the evaluation process of each item after the one-day operation of each vehicle 2 is completed by using the data acquired from each on-board unit 10, or the server device 40 may execute the evaluation process of each item at regular intervals. The evaluation process of each item may be executed within a certain period, and the execution timing of the operation evaluation process is not particularly limited. Then, the server device 40 executes a process of storing the operation evaluation data of each driver 3 obtained in the operation evaluation process in the evaluation data storage unit 62 of the storage unit 60.

One of the features of the server device 40 according to the present embodiment is the determination process of the safety confirmation operation of the driver 3 at the intersection.
The determination process of the safety confirmation operation of the driver 3 at the intersection is executed based on the data stored in the data storage unit 61.
The data stored in the data storage unit 61 includes the operation detected by the on-board unit 10 (driver monitoring unit 20, platform unit 11) when the vehicle 2 is passing through an intersection (for example, turning right or left). Data including the state of the person 3, the position of the vehicle 2, and the behavior of the vehicle 2 are included.

The data indicating the state of the driver 3 includes data related to the face detection state such as the face orientation of the driver 3 detected by the driver monitoring unit 20 (hereinafter, this data is also referred to as face image information). ing.
When the vehicle-mounted device 10 is installed near the center of the dashboard of the vehicle 2, for example, when the driver monitoring unit 20 of the vehicle-mounted device 10 captures an image of the driver 3 of the right-hand drive vehicle from this position, the vehicle-mounted device 10 is captured. Is located diagonally to the left and forward of the driver 3, so that the detection range of the face orientation in the right direction of the driver 3 is narrower than that in the left direction.

Therefore, for example, when the vehicle 2 makes a right turn at an intersection, if the driver 3 swings his face to the right and confirms the right direction, there is a period during which the driver monitoring unit 20 cannot detect the direction of the driver 3's face. It may occur.
Further, when the vehicle 2 is a large vehicle such as a truck or a bus having a long vehicle length, when the driver 3 puts his head out of the window of the driver's seat and checks the right direction when turning right at the intersection, the driver monitoring unit At 20, there may be a period during which the face of the driver 3 cannot be detected. It should be noted that such a phenomenon occurs when the on-board unit 10 is installed in a place other than the vicinity of the center of the dashboard of the vehicle 2, when the vehicle 2 is a left-hand drive vehicle, or when the vehicle 2 turns left at an intersection. But it can happen.

In such a case, the data indicating the state of the driver 3 includes the data in the state where the face (including the direction of the face) of the driver 3 is not detected.
Therefore, in the server device 40 according to the embodiment, the face (including the direction of the face) of the driver 3 is not detected in the data indicating the state of the driver 3 when the vehicle 2 is passing through the intersection. When the data of the state (hereinafter, also referred to as the face non-detection state) is included, it is determined whether or not the face non-detection state is due to the safety confirmation operation of the driver 3, and the determination result is taken into consideration. The process of evaluating the driver's safety confirmation operation at the intersection is executed.

In the determination of whether or not the face non-detection state is due to the safety confirmation operation of the driver 3, for example, the face of the driver 3 when the vehicle 2 is passing through an intersection (turning right or left) is detected. It is determined based on the temporal relationship between the present state (hereinafter, also referred to as the face detection state) and the face non-detection state of the driver 3.

For example, when the driver 3 confirms the turning direction when turning at an intersection (when turning right or left), the driver 3's face is swung from the front facing direction to the turning direction, and then swung in the front direction. And return to the original front facing state.
Therefore, the server device 40 changes from the face detection state to the face non-detection state when the data acquired from the vehicle-mounted device 10 changes in the direction in which the driver 3's face turns when turning at the intersection. If it indicates that the face detection state has been restored within the time specified, it may be determined that the face non-detection state is due to the safety confirmation operation of the driver 3. The first time is set to a time necessary and appropriate for safety confirmation (for example, around 1 second).

Further, the determination of whether or not the face non-detection state is due to the safety confirmation operation of the driver 3 is determined by the temporal change of the face orientation of the driver 3 in at least one of the face detection states before and after the face non-detection state. May be determined in consideration of.

For example, in the server device 40, when the data acquired from the in-vehicle device 10 is used, when the vehicle 2 turns at an intersection, the face direction (swing angle) of the driver 3 in the turning direction becomes larger with time, and then the face. When the non-detection state is reached and then the swing angle changes with time, it may be determined that the face non-detection state is due to the safety confirmation operation of the driver 3.

Alternatively, the server device 40 obtains data from the vehicle-mounted device 10 after the face orientation (swing angle) of the driver 3 in the bending direction of the driver 3 increases with time when the vehicle 2 turns at an intersection. If it indicates that the face has not been detected and the face has been detected again within a predetermined time, it may be determined that the face non-detection state is due to the safety confirmation operation of the driver 3.

Alternatively, the server device 40 changes the direction of the face of the driver 3 in the direction in which the face of the driver 3 turns when the vehicle 2 turns at the intersection, and the data acquired from the vehicle-mounted device 10 changes from the face detection state to the face non-detection state. After that, when the direction (swing angle) of the face in the bending direction of the driver 3 shows a change that becomes smaller with time, it is determined that the face non-detection state is due to the safety confirmation operation of the driver 3. You may.

On the other hand, the server device 40 does not return to the face detection state again within a predetermined second time from the face detection state to the face non-detection state when the vehicle 2 turns at the intersection. If it indicates that, it may be determined that the face non-detection state is not due to the safety confirmation operation of the driver 3. The second time is longer than the first time, and is set to a time for determining that the driver 3 cannot normally drive.

According to such a server device 40, even if the face non-detection state is detected while the vehicle 2 passes through the intersection, the face non-detection state is due to the safety confirmation operation of the driver 3. It becomes possible to appropriately determine whether or not the safety confirmation is performed, and it is possible to accurately evaluate the safety confirmation operation of the driver 3 at the intersection.

[Configuration example]
FIG. 2 is a block diagram schematically showing a main part of the vehicle-mounted device 10 used in the operation evaluation system 1 according to the embodiment.
The on-board unit 10 includes a platform unit 11 and a driver monitoring unit 20. Further, the drive recorder unit 30 is connected to the vehicle-mounted device 10.

The platform unit 11 is equipped with an acceleration sensor 12 that measures the acceleration of the vehicle 2, an angular velocity sensor 13 that detects the rotational angular velocity of the vehicle 2, and a GPS (Global Positioning System) receiving unit 14 that detects the position of the vehicle 2. There is. Further, the platform unit 11 includes a communication unit 15 that performs communication processing with an external device via the communication network 4, a notification unit 16 that outputs a predetermined sound or voice for issuing a warning, a storage unit 17, and an external device. The interface (external I / F) 18 is equipped. Further, the platform unit 11 is equipped with a control unit 19 that controls the processing operation of each unit.

The acceleration sensor 12 is composed of, for example, a three-axis acceleration sensor that measures acceleration in three directions of the XYZ axes. As the 3-axis accelerometer, a semiconductor type accelerometer such as a piezo resistance type can be used in addition to the capacitance type. A 2-axis and 1-axis accelerometer may be used as the accelerometer 12. The acceleration data measured by the acceleration sensor 12 is stored in the RAM 19b of the control unit 19 in association with the detection time.

The angular velocity sensor 13 is a sensor capable of detecting at least the angular velocity corresponding to the rotation around the vertical axis (yaw direction), that is, the angular velocity data corresponding to the rotation (turning) of the vehicle 2 in the left-right direction, for example, a gyro sensor (yaw rate). It is also called a sensor).
The angular velocity sensor 13 includes a uniaxial gyro sensor around the vertical axis, a biaxial gyro sensor that detects an angular velocity around the horizontal axis in the left-right direction (pitch direction), and a horizontal axis around the front-back direction (roll direction). A 3-axis gyro sensor that also detects the angular velocity of the gyro may be used. As these gyro sensors, in addition to the vibration type gyro sensor, an optical type or a mechanical type gyro sensor can be used.

Further, regarding the detection direction of the angular velocity around the vertical axis of the angular velocity sensor 13, for example, clockwise may be set in the positive direction and counterclockwise may be set in the negative direction. In this case, if the vehicle 2 turns to the right, positive angular velocity data is detected, and if the vehicle 2 turns to the left, negative angular velocity data is detected. In the angular velocity sensor 13, the angular velocity is detected in a predetermined cycle (for example, a cycle of several tens of ms), and the detected angular velocity data is stored in the RAM 19b of the control unit 19 in association with the detection time, for example. For the acceleration sensor 12 and the angular velocity sensor 13, an inertial sensor in which these are mounted in one package may be used.

The GPS receiving unit 14 receives GPS signals from artificial satellites via the antenna 14a at a predetermined cycle, and detects position data (latitude and longitude) of the current location. The position data detected by the GPS receiving unit 14 is stored in the RAM 19b of the control unit 19 in association with the detection time. The device for detecting the position of the vehicle 2 is not limited to the GPS receiving unit 14. For example, positioning devices compatible with other satellite positioning systems such as the Quasi-Zenith Satellite in Japan, GLONASS in Russia, Galileo in Europe, or Compass in China may be used.

The communication unit 15 includes a communication module that performs data output processing and the like on the server device 40 via the communication network 4.

The storage unit 17 is composed of, for example, a detachable storage device such as a memory card, or one or more storage devices such as a hard disk drive (HDD) and a solid state drive (SSD). The storage unit 17 stores, for example, data acquired from the acceleration sensor 12, the angular velocity sensor 13, the GPS receiving unit 14, the driver monitoring unit 20, or the drive recorder unit 30.

The external I / F 18 includes an interface circuit, a connector, and the like for exchanging data and signals with an in-vehicle device such as a drive recorder unit 30.

The control unit 19 is composed of a microcomputer including a CPU (Central Processing Unit) 19a, a RAM (Random Access Memory) 19b, and a ROM (Read Only Memory) 19c. The control unit 19 performs a process of storing various acquired data in the RAM 19b or the storage unit 17. Further, the control unit 19 reads out various data stored in the RAM 19b or the storage unit 17 as needed in addition to the program stored in the ROM 19c, and executes the program.

The driver monitoring unit 20 includes a driver camera 21, an image analysis unit 22, and an interface (I / F) 23.
The driver camera 21 includes, for example, a lens unit (not shown), an image sensor unit, a light irradiation unit, and a camera control unit that controls each of these units, and is configured as an image pickup unit.

Information including the specifications of the driver camera 21 (for example, the angle of view, the number of pixels (vertical x horizontal), etc.) and the position / orientation (for example, the mounting angle and the distance from a predetermined origin (the center position of the handle, etc.)) is the driver monitoring unit. It may be stored in 20 or the platform unit 11. Further, the driver monitoring unit 20 may be configured separately from the platform unit 11 in addition to the form integrated with the platform unit 11.

The image pickup device unit includes, for example, an image pickup device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), a filter, a microlens, and the like. The image pickup element unit includes an infrared sensor such as a CCD, CMOS, or a photodiode capable of receiving infrared rays such as near infrared rays or ultraviolet rays to form an image pickup image, in addition to those capable of forming an image pickup image by receiving light in the visible region. It may be included. The light irradiation unit includes a light emitting element such as an LED (Light Emitting Diode), and an infrared LED or the like may be used so that the state of the driver can be imaged day or night. The driver camera 21 may be a monocular camera or a stereo camera.

The camera control unit includes, for example, a processor and the like. The camera control unit controls the image pickup element unit and the light irradiation unit to irradiate light (for example, near infrared rays) from the light irradiation unit, and the image pickup element unit captures the reflected light. I do. The driver camera 21 captures an image at a predetermined frame rate (for example, 30 to 60 frames per second), and the data of the image captured by the driver camera 21 is output to the image analysis unit 22.

The image analysis unit 22 is configured to include, for example, an image processing processor, and the presence / absence of the driver's face, the direction of the driver's face, the direction of the line of sight, and the opening of the eyes are determined from the image captured by the driver camera 21. It performs processing such as detecting facial image information including at least the presence or absence of the driver's face. The driver's face image information, image data, and imaging date / time data detected by the image analysis unit 22 are transmitted to the platform unit 11 via the interface (I / F) 23, and the RAM 19b or the storage unit 17 of the platform unit 11 Is remembered in.

The orientation of the driver's face detected by the image analysis unit 22 is, for example, a pitch angle which is an angle (vertical orientation) around the X axis (left and right axis) of the driver's face, and a Y axis of the face ( It may be indicated by the Yaw angle, which is the angle (horizontal axis) around (vertical axis), and the roll angle, which is the angle (left-right tilt) around the Z axis (front-back axis) of the face, at least left and right. Includes yaw angle to indicate orientation. Further, these angles can be indicated by an angle with respect to a predetermined reference direction, and for example, the reference direction may be set to the front direction of the driver.

Further, the direction of the driver's line of sight detected by the image analysis unit 22 is, for example, the direction of the driver's face detected from the image and information on the eye region (position of the inner corner of the eye, the outer corner of the eye, or the pupil). It is estimated from the relationship with and can be indicated by a line-of-sight vector V (three-dimensional vector) on three-dimensional coordinates. The line-of-sight vector V is, for example, a pitch angle which is an angle (up and down direction) around the X axis (left and right axis) of the driver's face, and a yaw which is an angle (left and right direction) around the Y axis (up and down axis) of the face. It may be estimated from at least one of the angle and the roll angle (left-right tilt) around the Z axis (front-back axis) of the face and the information in the eye area. Further, the line-of-sight vector V shows a part of the value of the three-dimensional vector in common with the value of the vector of the face orientation (for example, the origin of the three-dimensional coordinates is common), or the vector of the face orientation is used as a reference. It may be indicated by the relative angle (relative value of the vector of the direction of the face).

Further, the driver's eye opening detected by the image analysis unit 22 is, for example, information on the driver's eye region (the inner corner of the eye, the outer corner of the eye, the position of the upper and lower eyelids, the position of the pupil, etc.) detected from the image. ) Is estimated.

The drive recorder unit 30 is connected to the external I / F 18 of the platform unit 11. The drive recorder unit 30 includes an outside camera 31 and an inside camera 32.

The out-of-vehicle camera 31 is a camera that captures an image in front of the vehicle 2, and the in-vehicle camera 32 is a camera that captures an image of the interior of the vehicle 2. The outside camera 31 and the inside camera 32 may be composed of, for example, a visible light camera, but may be composed of a near infrared camera or the like.

The out-of-vehicle camera 31 and the in-vehicle camera 32 each capture an image at a predetermined frame rate (for example, 30 to 60 frames per second), and the images captured by the out-of-vehicle camera 31 and the in-vehicle camera 32 and data such as the imaging date and time are stored in the platform unit. It is sent to 11 and stored in the RAM 19b or the storage unit 17 of the platform unit 11. The drive recorder unit 30 may be configured to include only the external camera 31.

FIG. 3 is a diagram showing an example of the structure of data transmitted from the vehicle-mounted device 10 to the server device 40.
The data transmitted from the in-vehicle device 10 to the server device 40 includes the identification information (serial number, etc.) of the in-vehicle device 10, the transmission date and time, the data of the face detection flag F (for example, F = 1 or F = 0), and the operation. Face orientation (pitch, yaw, and roll), line-of-sight direction (pitch, yaw), eye opening (right and left eyes), vehicle acceleration (front and back, left, right, and up and down), angular velocity (yaw) , Driver image, outside image, vehicle position data (latitude and longitude), and running speed. The face detection flag F is a face detection state (F = 1) in which the driver's face is detected from the image captured by the driver camera 21 and a face non-detection state (F = 0) in which the driver's face is not detected. Is shown. When the face detection flag F is set to 0, the data transmitted to the server device 40 does not include the face direction, the line-of-sight direction, the eye opening, and the driver image. Further, the structure of the data transmitted to the server device 40 is not limited to the structure shown in FIG.

The data of the structure illustrated in FIG. 3 is transmitted, for example, when the vehicle-mounted device 10 detects that the vehicle 2 has passed the intersection. The on-board unit 10 states that, for example, the vehicle 2 has passed the intersection when a predetermined time has elapsed since the integrated value of the angular velocity detected by the angular velocity sensor 13 exceeds a predetermined threshold value (threshold value for estimating the approach to the intersection). It is possible to judge.

FIG. 4 is a functional configuration block diagram showing a characteristic portion of the server device 40 according to the embodiment.
The central processing unit 51 of the server computer 50 includes a data acquisition unit 54, a determination unit 55, and an evaluation unit 56. Further, the central processing unit 51 includes a detection unit 57.

The data acquisition unit 54 acquires data for a predetermined operation period (a period such as a day, a week, or a month) acquired from the vehicle-mounted device 10 to be evaluated from the data storage unit 61. The data acquired from the data storage unit 61 includes data indicating the state of the driver 3 while the vehicle 2 on which the vehicle-mounted device 10 is mounted passes through each intersection. This data includes data indicating a face detection state in which the face of the driver 3 is detected and a face non-detection state in which the face of the driver 3 is not detected, which are detected while the vehicle 2 passes through the intersection. It is included. Further, the face image information may be included in the data indicating the state of the driver 3.

The determination unit 55 is the data acquired by the data acquisition unit 54, that is, the face detection state and the face non-detection state of the driver 3 detected while the vehicle 2 passes through each intersection (for example, right turn or left turn). The detection data (for example, the data of the face detection flag F) is read, and the face non-detection state is based on the temporal relationship between the face detection state and the face non-detection state while passing through each intersection indicated by these detection data. Is executed by the process of determining whether or not is due to the safety confirmation operation of the driver 3.

For example, when the data acquired by the data acquisition unit 54 is the detection data indicating the face non-detection state while the vehicle 2 passes through the intersection, the determination unit 55 determines at least one of the faces before and after the face non-detection state. It is determined whether or not the face non-detection state is due to the safety confirmation operation of the driver 3 in consideration of the temporal change of the face orientation of the driver 3 in the detection state (for example, the change with time of the angle data). You may.

For the temporal change of the face orientation of the driver 3 in at least one of the face detection states before and after the face non-detection state, for example, when the vehicle 2 turns at an intersection, the face orientation of the driver 3 in the bending direction is used. The indicated angle data includes either a change in which the face is not detected after becoming larger with time, or a change in which the angle data detected after becoming a face non-detection state becomes smaller with time.

Alternatively, the determination unit 55 changes the data acquired by the data acquisition unit 54 from the face detection state to the face non-detection state while the vehicle 2 passes through the intersection, and detects the face again within a predetermined first time. When it indicates that the state has returned, it may be determined that the face non-detection state is due to the safety confirmation operation of the driver 3. The first time is set to a time necessary and appropriate for safety confirmation (for example, around 1 second).

On the other hand, the determination unit 55 changes the face detection state to the face non-detection state when the vehicle 2 passes through the intersection, and the data acquired by the data acquisition unit 54 changes to the face detection state again within a predetermined second time. If it indicates that the state has not been returned, it may be determined that the face non-detection state is not due to the safety confirmation operation of the driver 3. The second time is longer than the first time, and is set to a time for determining that the driver 3 cannot normally drive.

The evaluation unit 56 acquires the result determined by the determination unit 55, and executes a process of evaluating the safety confirmation operation of the driver 3 at the intersection in consideration of the determination result.
For example, when the determination unit 55 determines that the face non-detection state detected when passing through the intersection is due to the safety confirmation operation of the driver 3, the evaluation unit 56 determines that the face non-detection state is the safety confirmation. The driving of the driver 3 is evaluated as if it is due to the operation, that is, the safety confirmation is performed at the intersection.

Further, when the determination unit 55 determines that the face non-detection state detected when passing through the intersection is not due to the safety confirmation operation of the driver 3, the evaluation unit 56 determines that the face non-detection state is the safety confirmation operation. The driver 3's driving is evaluated as if the safety has not been confirmed at the intersection.

Next, an example of the processing executed by the data acquisition unit 54, the determination unit 55, the evaluation unit 56, and the detection unit 57 of the central processing unit 51 will be described.
The detection unit 57 performs a process of detecting the approach time and the turning direction of the vehicle 2 to the intersection. For example, the detection unit 57 calculates an integral value of the angular velocity data (angular velocity data transmitted from the in-vehicle device 10 after passing through the intersection) acquired by the data acquisition unit 54, and the calculated integral value is a predetermined integral. The time when the ratio is reached is estimated as the time of entry to the intersection. Further, the direction in which the vehicle 2 turns at the intersection is estimated based on the positive / negative of the angular velocity data. For example, a positive value is estimated to be a right turn, and a negative value is estimated to be a left turn.

Further, the detection unit 57 detects the swing angle and swing time of the driver's face at a predetermined time before and after the approach time based on the face image information acquired by the data acquisition unit 54. Further, the detection unit 57 determines the direction of the face of the driver 3 based on the angle data (for example, the angle data in the yaw direction) indicating the direction of the driver's face included in the face image information of the driver 3. Temporal changes (eg, changes over time in angle data) may be detected.

Next, the evaluation unit 56 performs a process of evaluating the safety confirmation operation before entering the intersection based on the swing angle and the swing time at a predetermined time (before approach) before the approach time.

For example, regarding safety confirmation in the right direction, (1) whether or not the confirmation timing in the right direction is appropriate, (2) whether or not the swing angle in the right direction is equal to or greater than a predetermined angle, and (3) the predetermined value. It is determined whether or not the state of the angle or more continues for a predetermined time or more. (1) Whether or not the confirmation timing in the right direction is appropriate can be determined by, for example, whether or not the swing angle in the right direction is equal to or greater than the predetermined angle by a predetermined time before the approach time. ..

In the case of determining the states (2) and (3) above, if the swing angle to the right cannot be detected, that is, there is a period during which data indicating the face non-detection state is acquired, the determination unit 55 determines. Based on the temporal relationship between the face detection state and the face non-detection state, a process of determining whether or not the face non-detection state is due to the safety confirmation operation in the right direction of the driver 3 is performed.

For example, when the data acquired by the data acquisition unit 54 indicates that the face detection state has changed to the face non-detection state and then returned to the face detection state within the first time, the face non-detection state. Is determined to be due to the driver 3's rightward safety confirmation operation.

Alternatively, when the detection unit 57 detects a temporal change in the direction of the face of the driver 3 before the face non-detection state, for example, a change with time in which the swing angle to the right increases, or the face non-detection state. When a temporal change in the direction of the face of the driver 3, for example, a change with time in which the swing angle to the right becomes smaller is detected, the determination unit 55 determines that the face non-detection state is in the right direction of the driver 3. It may be determined that it is due to the safety confirmation operation.

On the other hand, when the data acquired by the data acquisition unit 54 changes from the face detection state to the face non-detection state and does not return to the face detection state again within the second time, the face. It is determined that the non-detection state is not due to the safety confirmation operation in the right direction of the driver 3.

Similarly, regarding safety confirmation in the left direction, (4) whether or not the confirmation timing in the left direction is appropriate, (5) whether or not the swing angle in the left direction is equal to or greater than a predetermined angle, and (6) the above. It is determined whether or not the state of the predetermined angle or more continues for the predetermined time or more. (4) Whether or not the confirmation timing in the left direction is appropriate can be determined by, for example, whether or not the swing angle in the left direction is equal to or greater than the predetermined angle by a predetermined time before the approach time. ..

When determining the states (5) and (6) above, if there is a period during which the swing angle to the left cannot be detected, that is, if there is a period during which data indicating the face non-detection state is acquired. The determination unit 55 performs a process of determining whether or not the face non-detection state is due to the safety confirmation operation in the left direction of the driver 3 based on the temporal relationship between the face detection state and the face non-detection state. ..

Further, the evaluation unit 56 evaluates the safety confirmation operation of the course change direction at the intersection, for example, the right turn destination or the left turn destination, based on the swing angle and the swing time at a predetermined time after the approach time (that is, during approach). Perform the processing.
For example, regarding the safety confirmation of the course change direction, (7) whether or not the confirmation timing of the course change direction is appropriate, and (8) whether or not the swing angle in the course change direction of turning right or left is equal to or more than a predetermined angle. Or (9) it is determined whether or not the state of the predetermined angle or more continues for a predetermined time or more. Whether or not the confirmation timing of the above (7) course change direction is appropriate is determined by, for example, whether or not the swing angle in the course change direction is equal to or greater than the predetermined angle by a predetermined time after the approach time. be able to.

In the case of determining the states (8) and (9) above, there is a period in which the swing angle in the course change direction cannot be detected, that is, there is a period in which data indicating the face non-detection state is acquired. Based on the temporal relationship between the face detection state and the face non-detection state, the determination unit 55 determines whether the face non-detection state is due to the safety confirmation operation in the direction of changing the course of the driver 3 to turn right or left. Performs the process of determining whether or not.

Further, the evaluation unit 56 may perform a process of evaluating the driver's deceleration consciousness based on the traveling speed of the vehicle 2 at a predetermined time before and after the approach time. For example, (10) deceleration by determining whether the maximum value of the traveling speed of the vehicle 2 at a predetermined time before and after the approach time is equal to or less than a predetermined speed (for example, an upper limit speed at which a vehicle can safely turn and pass through an intersection). You may evaluate the presence or absence of consciousness.

The evaluation unit 56 evaluates each confirmation action set under a predetermined evaluation condition based on the determination methods of the above items (1) to (10), and performs a process of calculating an evaluation point for each intersection. For example, for each confirmation action set under predetermined evaluation conditions, the evaluation results of confirmation timing, confirmation angle, and confirmation time are totaled, necessary statistical processing (averaging, normalization processing, etc.) is performed, and the intersection Calculate the score for each score or ranking.
Then, the evaluation unit 56 performs a process of storing the calculated evaluation points for each intersection in the evaluation data storage unit 62 of the storage unit 60.

Further, the server device 40 may be configured to be able to provide a cloud service. For example, the server device 40 may be composed of a server system including a Web server, an application server, and a database server. The Web server processes access requests and the like from the browser of the business terminal 80. The application server accesses the database server, for example, in response to a request from the Web server, and performs processing such as searching for data necessary for processing and extracting data. The database server manages various data acquired from the in-vehicle device 10, for example, and performs processing such as search, extraction, and storage of the data in response to a request from the application server.

For example, the server device 40 processes various requests requested from the browser of the business operator terminal 80, for example, a login to a website, a request for transmitting a driving evaluation report, and the like. Further, the server device 40 transmits the processing result, for example, the data of the created operation evaluation report to the business operator terminal 80 through a browser or the like, and executes a process of displaying the operation evaluation report on the display screen of the business operator terminal 80. do.

The business operator confirms the driving evaluation report of each driver 3 displayed on the display screen of the business operator terminal 80, and gives guidance to each driver 3 to improve safety awareness. By providing such guidance, it is possible to raise the safety awareness of the driver 3 and reduce the risk of accidents.

[Operation example]
FIG. 5 is a flowchart showing a processing operation performed by the driver monitoring unit 20 in the vehicle-mounted device 10 according to the embodiment. This processing operation is executed, for example, at the timing when an image is captured by the driver camera 21 (for example, every frame or every frame at a predetermined interval).

First, the image analysis unit 22 acquires an image captured by the driver camera 21 (step S1).

Next, the image analysis unit 22 performs a process of detecting the driver's face or a face area from the acquired image (step S2). The method for detecting a face from an image is not particularly limited, but it is preferable to adopt a method for detecting a face at high speed and with high accuracy.

Next, the image analysis unit 22 determines whether or not the driver's face is detected from the image acquired in step S1 (step S3).

If the image analysis unit 22 determines in step S3 that the driver's face has been detected, the image analysis unit 22 next sets the face detection flag F to 1 (step S4). The state in which the face detection flag F is set to 1 indicates that the face detection flag F is in the face detection state.

Next, the image analysis unit 22 performs a process of detecting the position and shape of facial organs such as eyes, nose, mouth, and eyebrows from the facial region detected in step S2 (step S5). The method for detecting the facial organs from the facial region in the image is not particularly limited, but it is preferable to adopt a method that can detect the facial organs at high speed and with high accuracy. For example, a method in which the image analysis unit 22 creates a three-dimensional face shape model, fits the three-dimensional face shape model to the face region on the two-dimensional image, and detects the position and shape of each organ of the face can be adopted. According to this method, it is possible to accurately detect the position and shape of each organ of the face regardless of the installation position of the driver camera 21 or the orientation of the face in the image. As a technique for fitting a three-dimensional face shape model to a human face in an image, for example, the technique described in JP-A-2007-249280 can be applied, but the technique is not limited thereto.

Next, the image analysis unit 22 detects the orientation of the driver's face based on the position and shape data of each organ of the face obtained in step S5 (step S6). For example, the pitch angle of vertical rotation (around the X axis), the yaw angle of the horizontal rotation (around the Y axis), and the roll angle of the total rotation (around the Z axis) included in the parameters of the above three-dimensional face shape model are set. It may be detected as information regarding the orientation of the driver's face.

Next, the image analysis unit 22 determines the orientation of the driver's face determined in step S6, and the position and shape of the driver's facial organs determined in step S5, particularly the characteristic points of the eyes (outer corners of the eyes, inner corners of the eyes, and pupils). The direction of the line of sight is detected based on the position and shape (step S7).
The direction of the line of sight is, for example, the feature amount of the image of the eye in various face directions and the direction of the line of sight (relative position of the outer corner of the eye, the inner corner of the eye, the pupil, or the relative position of the sclera (so-called white eye) part and the iris (so-called black eye) part. , Shading, texture, etc.) may be detected by learning in advance using a learning device and evaluating the degree of similarity with the learned feature amount data. Alternatively, using the fitting result of the three-dimensional face shape model, the size and center position of the eyeball are estimated from the size and orientation of the face and the position of the eyes, and the position of the pupil is detected to determine the center of the eyeball. The vector connecting the center of the pupil may be detected as the line-of-sight direction.

Next, the image analysis unit 22 determines the eye opening degree based on the position and shape of the driver's facial organs obtained in step S5, particularly the position and shape of the characteristic points of the eyes (outer corners of the eyes, inner corners of the eyes, pupils, and eyelids). Detect (step S8).

Next, the image analysis unit 22 determines the data of the face detection flag F = 1 set in step S4, the direction of the driver's face detected in step S6, the direction of the driver's line of sight detected in step S7, and the direction of the driver's line of sight. The driver's eye opening detected in step S8 is associated with the imaging time and transmitted as face image information to the platform unit 11 (step S9), after which the image analysis unit 22 returns to step S1 and processes. repeat. The face image information transmitted in step S9 may include the driver's image acquired in step S1. Further, the processing of steps S5 to S8 may be omitted, and the data of the face detection flag F = 1 set in step S4 may be transmitted to the platform unit 11 in step S9.

On the other hand, if the image analysis unit 22 determines in step S3 that the driver's face has not been detected, then the face detection flag F is set to 0 (step S10). The state in which the face detection flag F is set to 0 indicates that the face is not detected.

Next, the image analysis unit 22 transmits data indicating the face non-detection state, that is, data of the face detection flag F = 0 to the platform unit 11 (step S11), and then the image analysis unit 22 returns to step S1. , Repeat the process.

FIG. 6 is a flowchart showing a processing operation performed by the control unit 19 in the vehicle-mounted device 10 according to the embodiment. This processing operation is executed, for example, in a predetermined cycle of several tens of ms to several seconds.

The control unit 19 acquires the acceleration data measured by the acceleration sensor 12 and stores it in the RAM 19b (step S21).

Further, the control unit 19 acquires the angular velocity data detected by the angular velocity sensor 13 and stores it in the RAM 19b (step S22).

Further, the control unit 19 acquires the face detection information transmitted from the driver monitoring unit 20 and stores it in the RAM 19b (step S23). The face detection information includes data of the face detection flag F set for each image, that is, setting data of the face detection flag F = 1 or F = 0. In step S23, when the face detection flag F is 1, the face image information may also be acquired from the driver monitoring unit 20.

Further, the control unit 19 acquires the data (data of the image outside the vehicle and the image inside the vehicle) transmitted from the drive recorder unit 30 and stores it in the RAM 19b (step S24).

Further, the control unit 19 acquires the position data detected by the GPS receiving unit 14 (step S25). The order of processing in steps S21 to S25 does not matter.

Next, the control unit 19 calculates the traveling speed based on the unit-time change of the position data acquired in step S25, and stores the position data and the traveling speed in the RAM 19b (step S26).

Next, the control unit 19 determines whether or not the vehicle 2 has entered the intersection (step S27). For example, the control unit 19 determines whether or not the integrated value of the angular velocity is equal to or higher than the threshold value indicating the approach of the vehicle to the intersection and the traveling speed is equal to or lower than the predetermined upper limit speed indicating the approach to the intersection.

In step S27, if the control unit 19 determines that the vehicle 2 has entered the intersection, then the control unit 19 determines whether or not the vehicle 2 has passed the intersection (step S28). For example, it is determined whether or not the integrated value of the angular velocity is equal to or less than the threshold value indicating that the vehicle has passed the intersection and the traveling speed is equal to or more than the predetermined lower limit speed indicating the passage of the intersection.

If the control unit 19 determines that the vehicle 2 has passed the intersection, the control unit 19 reads the data acquired at a predetermined time (about several tens of seconds) before and after the vehicle 2 enters the intersection from the RAM 19b, and reads the data from the RAM 19b. Is controlled to execute a process of transmitting data to the server device 40 as data when passing through an intersection (step S29). After that, the control unit 19 returns to step S21 and repeats the process. FIG. 3 shows an example of the structure of data transmitted from the vehicle-mounted device 10 to the server device 40 when passing through an intersection.

The control unit 19 may store the data read from the RAM 19b in the storage unit 17 (for example, a removable storage medium) instead of the processing in step S29. Then, after the end of the day's driving, the driver 3 removes the storage unit 17 from the vehicle-mounted device 10, reads the data stored in the storage unit 17 into the business terminal 80, and the business terminal 80 causes the server device 40 to read the data. You may send it.

FIG. 7 is a flowchart showing a processing operation performed by the central processing unit 51 in the server device 40 according to the embodiment.
The central processing unit 51 determines whether or not the data transmitted from the vehicle-mounted device 10 has been received (step S31).

If the central processing unit 51 determines that the data has been received from the vehicle-mounted device 10, then the data received from the vehicle-mounted device 10 is stored in the data storage unit 61 in association with the identification information of the vehicle-mounted device 10 (step). S32).

After that, the central processing unit 51 determines whether or not it is the timing to execute the evaluation of the driver's safe driving behavior (step S33). Whether or not it is the timing to execute the evaluation may be determined, for example, by whether or not a signal indicating the end of operation of the vehicle 2 is received from the vehicle-mounted device 10.

If the central processing unit 51 determines that it is not the timing to execute the evaluation, the process returns to step S31.

On the other hand, if the central processing unit 51 determines that it is time to execute the evaluation, the central processing unit 51 performs an evaluation process of the safety confirmation operation at the intersection of the driver 3 of the vehicle 2 on which the vehicle-mounted device 10 is mounted (step S34). The evaluation process executed in step S34 will be described with reference to the flowcharts of FIGS. 8A and 8B.

After that, the central processing unit 51 performs a process of storing the result of the evaluation process in the evaluation data storage unit 62 in association with the information of the vehicle-mounted device 10 or the driver 3 (step S35), and ends the process.

8A and 8B are examples of flowcharts showing the processing operation of the safety confirmation evaluation performed by the central processing unit 51 in the server device 40 according to the embodiment. It should be noted that this flowchart shows a processing operation for evaluating the safety confirmation operation of the driver 3 at each intersection that has passed by the end of the day's operation of the vehicle 2 on which the on-board unit 10 is mounted.

The central processing unit 51 acquires data from the data storage unit 61 when the vehicle-mounted device 10 passes through the intersection (step S41).

Next, the central processing unit 51 estimates the time when the vehicle 2 enters the intersection (step S42). For example, the central processing device 51 calculates the integrated value of the angular velocity data included in the data at the time of passing through the intersection, and estimates the time when the calculated integrated value reaches a predetermined integration ratio as the approach time to the intersection.

Next, the central processing unit 51 estimates the direction in which the vehicle 2 turns at the intersection (step S43). For example, the central processing unit 51 estimates the direction in which the vehicle 2 turns at an intersection based on the positive / negative of the angular velocity data. For example, a positive value is estimated to be a right turn, and a negative value is estimated to be a left turn.

Next, the central processing unit 51 reads the face detection information of the driver 3 at a predetermined time before and after the approach time to the intersection from the data acquired in step S41 (step S44). The central processing unit 51 reads the face detection information included in the data at the time of passing through the intersection, that is, the data of the face detection flag F.

Next, the central processing unit 51 determines whether or not the face non-detection state (the period of the face detection flag F = 0) is detected within a predetermined time before the time of entering the intersection (step S45).

If the central processing unit 51 determines in step S45 that the face non-detection state is detected within a predetermined time before the time of approach to the intersection, the central processing unit 51 detects it before the time of approach to the intersection. It is determined whether or not the face non-detection state is due to the safety confirmation operation of the driver 3 (step S46).

For example, the central processing device 51 changes the face detection information within a predetermined time before the entry time to the intersection from the face detection state to the face non-detection state, and returns to the face detection state again within the first time. If it indicates that, it is determined that the face non-detection state is due to the safety confirmation operation of the driver 3. The first time is set to a time necessary and appropriate for safety confirmation.

Further, the central processing device 51 detects a temporal change in the direction of the face of the driver 3 before the face non-detection state, for example, a change with time in which the swing angle of the face in the bending direction becomes large, or the face is not detected. When a temporal change in the direction of the face of the driver 3 after the detection state, for example, a change over time in which the swing angle of the face in the bending direction becomes smaller is detected, the face non-detection state is due to the safety confirmation operation of the driver 3. It may be determined that.

In step S46, if the central processing unit 51 determines that the face non-detection state is due to the safety confirmation operation of the driver 3, the central processing unit 51 performs the safety confirmation operation during the period of the face non-detection state. Evaluate as part of the period (step S47).

Then, in the next step S49, the central processing device 51 is detected in the face non-detection state evaluated as a part of the period in which the safety confirmation operation is performed and the face detection state before and after the face non-detection state. Based on the collected data, the safety confirmation operation (right or left confirmation timing, face swing angle, swing time, etc.) before entering the intersection is evaluated by comparing with predetermined evaluation conditions, and the process of step S50 is performed. Proceed to.

On the other hand, if the central processing unit 51 determines in step S46 that the face non-detection state is not due to the safety confirmation operation of the driver 3, the central processing unit 51 performs the safety confirmation operation during the period of the face non-detection state. It is evaluated as a period not performed (step S48). The period during which the safety confirmation operation is not performed may be evaluated as the period during which the driver 3 has not been able to operate normally.

Then, in the next step S49, the central processing apparatus 51 uses the data detected in the face non-detection state evaluated as the period during which the safety confirmation operation is not performed and the face detection state before and after the face non-detection state. Based on this, the safety confirmation operation (confirmation timing in the right or left direction, face swing angle, swing time, etc.) before entering the intersection is evaluated by comparing with predetermined evaluation conditions, and the process proceeds to step S50.

Further, in step S45, if the central processing unit 51 determines that the face non-detection state has not been detected within a predetermined time before the approach time to the intersection, the central processing unit 51 performs the processing in step S49. Based on the data detected in the face detection state, the safety confirmation operation (right or left confirmation timing, swing angle, swing time, etc.) before entering the intersection is compared with the predetermined evaluation conditions for evaluation. Then, the process proceeds to step S50.

In step S50, the central processing unit 51 determines whether or not the face non-detection state (the period of the face detection flag F = 0) is detected within a predetermined time after the approach time to the intersection. If the central processing unit 51 determines in step S50 that the face non-detection state is detected within a predetermined time after the entry time to the intersection, the central processing unit 51 is detected after the entry time to the intersection. It is determined whether or not the face non-detection state is due to the safety confirmation operation of the driver 3 (step S51).

The central processing device 51, for example, changed the face detection information within a predetermined time after the entry time to the intersection from the face detection state to the face non-detection state, and returned to the face detection state again within the first time. If it indicates that, it is determined that the face non-detection state is due to the safety confirmation operation of the driver 3. The first time is set to a time necessary and appropriate for safety confirmation.

Further, the central processing device 51 detects a temporal change in the direction of the face of the driver 3 before the face non-detection state, for example, a change with time in which the swing angle of the face in the bending direction becomes large, or the face is not detected. When a temporal change in the direction of the face of the driver 3 after the detection state, for example, a change over time in which the swing angle of the face in the bending direction becomes smaller is detected, the face non-detection state is due to the safety confirmation operation of the driver 3. It may be determined that.

In step S51, if the central processing unit 51 determines that the face non-detection state is due to the safety confirmation operation of the driver 3, the central processing unit 51 performs the safety confirmation operation during the period of the face non-detection state. Evaluate as part of the period (step S52).

Then, in the next step S54, the central processing device 51 is detected in the face non-detection state evaluated as a part of the period in which the safety confirmation operation is performed and the face detection state before and after the face non-detection state. Based on the collected data, the safety confirmation operation (right or left confirmation timing, face swing angle, swing time, etc.) after entering the intersection is evaluated by comparing with predetermined evaluation conditions, and the process of step S55 is performed. Proceed to.

On the other hand, if the central processing unit 51 determines in step S51 that the face non-detection state is not due to the safety confirmation operation of the driver 3, the central processing unit 51 performs the safety confirmation operation during the period of the face non-detection state. It is evaluated as a period not performed (step S53). The period during which the safety confirmation operation is not performed may be evaluated as the period during which the driver 3 has not been able to operate normally.

Then, in the next step S54, the central processing apparatus 51 uses the data detected in the face non-detection state evaluated as the period during which the safety confirmation operation is not performed and the face detection state before and after the face non-detection state. Based on this, the safety confirmation operation (confirmation timing in the right or left direction, face swing angle, swing time, etc.) after entering the intersection is evaluated by comparing with predetermined evaluation conditions, and the process proceeds to step S55.

Further, if the central processing unit 51 determines in step S50 that the face non-detection state has not been detected within a predetermined time after the approach time to the intersection, the central processing unit 51 performs the processing in step S54. Based on the data detected in the face detection state, the safety confirmation operation (right or left confirmation timing, swing angle, swing time, etc.) after entering the intersection is compared with the predetermined evaluation conditions for evaluation. Then, the process proceeds to step S55.

In step S55, the central processing unit 51 evaluates whether the traveling speed of the vehicle 2 at a predetermined time before and after the approach time to the intersection is appropriate. For example, it is evaluated whether or not the traveling speed of the vehicle 2 is equal to or less than a predetermined speed at which safe traveling is possible in the intersection.

Next, the central processing unit 51 aggregates the points evaluated in steps S49, S54, and S55 or the ranked scores, calculates the operation evaluation points at the intersection (step S56), and converts the calculated operation evaluation points into the vehicle. It is stored in the evaluation data storage unit 62 of the storage unit 60 in association with the identification information of the machine 10 or the information of the driver 3 (step S57).

Next, the central processing unit 51 determines whether or not the evaluation at all the passing intersections has been completed (step S58), and if it is determined that the evaluation has been completed, then the processing is completed, but it is determined that the evaluation has not been completed. Then, the process returns to step S41 and the process is repeated.

[Action / Effect]
According to the server device 40 according to the above embodiment, even if the face non-detection state of the driver 3 is detected while the vehicle 2 passes through the intersection, the face detection state and the face non-detection state are obtained. By considering the time relationship of, it is possible to appropriately and efficiently determine whether or not the face non-detection state is due to the safety confirmation operation of the driver 3, and the safety confirmation of the driver 3 at the intersection. The evaluation accuracy of the operation can be improved. In particular, it is possible to improve the accuracy of driving evaluation of a driver of a vehicle having a long vehicle length such as a truck or a bus, which tends to have a large operation for safety confirmation.

As a temporal relationship between the face detection state of the driver 3 and the face non-detection state, it is determined that the face detection state of the driver 3 changed to the face non-detection state and returned to the face detection state again within the first time. By detecting, it is possible to more accurately determine whether or not the face non-detection state while the vehicle 2 passes through the intersection is due to the safety confirmation operation of the driver 3.

Further, by considering the temporal change of the face orientation of the driver 3 at least before and after the face non-detection state, whether or not the face non-detection state is due to the safety confirmation operation of the driver 3 can be determined. It can be determined more accurately.

Further, as a temporal relationship between the face detection state of the driver 3 and the face non-detection state, the face detection state of the driver 3 changes to the face non-detection state and does not return to the face detection state again within the second time. By detecting this, it is possible to appropriately determine even if the face non-detection state while the vehicle 2 passes through the intersection is not due to the safety confirmation operation of the driver 3.

[Modification example]
Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in all respects. Needless to say, various improvements and changes can be made without departing from the scope of the present invention.
For example, in the above embodiment, the server device 40 has been described as an example of the "operation evaluation device" according to the present invention, but in another embodiment, the configuration of the "operation evaluation device" according to the present invention is configured as an in-vehicle device. It may be equipped in 10.

More specifically, the data acquisition unit 54, the determination unit 55, the evaluation unit 56, and the detection unit 57 included in the server computer 50 shown in FIG. 4 may be equipped in the control unit 19 of the vehicle-mounted device 10. .. In this case, the data acquisition unit 54 may acquire face image information including the face detection information of the driver 3 from the RAM 19b or the storage unit 17 of the vehicle-mounted device 10. According to this configuration, the vehicle-mounted device 10 can execute the above-mentioned determination and evaluation processing. Further, the control unit 19 of the vehicle-mounted device 10 may be equipped with the data acquisition unit 54, the determination unit 55, and the detection unit 57, and the evaluation unit 56 may be equipped with the server computer 50.

[Additional Notes]
Embodiments of the present invention may also be described as, but are not limited to, the following appendices.
(Appendix 1)
A driving evaluation device (40) that evaluates the driving of the driver of the vehicle (2).
Acquire data on a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected while the vehicle (2) passes through an intersection. Data acquisition unit (54) and
The face non-detection state is based on the driver's safety confirmation operation based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired by the data acquisition unit (54). The determination unit (55) for determining whether or not the data is
An operation evaluation device including an evaluation unit (56) that evaluates the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination unit (55).

(Appendix 2)
It is a driving evaluation method for evaluating the driving of the driver of the vehicle (2).
Acquire data on a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected while the vehicle (2) passes through an intersection. Data acquisition step (S41) and
The face non-detection state is based on the driver's safety confirmation operation based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired in the data acquisition step (S41). Judgment step (S46, S51) for determining whether or not
An operation evaluation method comprising an evaluation step (S49, S54) for evaluating the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination step (S46, S51).

(Appendix 3)
It is a driving evaluation program that causes at least one computer (50) to execute a process of evaluating the driving of the driver of the vehicle (2).
To the at least one computer (50)
Acquire data on a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected while the vehicle (2) passes through an intersection. Data acquisition step (S41) and
The face non-detection state is based on the driver's safety confirmation operation based on the temporal relationship between the face detection state and the face non-detection state indicated by the data acquired in the data acquisition step (S41). Judgment step (S46, S51) for determining whether or not
An operation evaluation program characterized by executing an evaluation step (S49, S54) for evaluating the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination step (S46, S51).

1 Driving evaluation system 2 Vehicle 3 Driver 4 Communication network 10 In-vehicle device 11 Platform unit 12 Accelerometer 13 Angular velocity sensor 14 GPS receiver 14a Antenna 15 Communication unit 16 Notification unit 17 Storage unit 18 External interface (external I / F)
19 Control unit 19a CPU
19b RAM
19c ROM
20 Driver monitoring unit 21 Driver camera 22 Image analysis unit 23 Interface (I / F)
30 Drive recorder section 31 Outside camera 32 Inside camera 40 Server device (driving evaluation device)
41 Communication unit 50 Server computer 51 Central processing device 52 Main memory 53 Control program 54 Data acquisition unit 55 Judgment unit 56 Evaluation unit 57 Detection unit 60 Storage unit 61 Data storage unit 62 Evaluation data storage unit 63 Driver information storage unit 80 Operator Terminal

Claims (7)

It is a driving evaluation device that evaluates the driving of the driver of the vehicle.
Flag data indicating a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected at predetermined intervals while the vehicle passes through the intersection, are shown. A data acquisition unit that acquires data related to the driver's condition, including flag data,
Based on the temporal relationship between the face detection state and the face non-detection state, which is indicated by the flag data indicating the face detection state and the flag data indicating the face non-detection state acquired by the data acquisition unit. A determination unit that determines whether or not the face non-detection state is due to the driver's safety confirmation operation, and
An operation evaluation device including an evaluation unit that evaluates the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination unit. The determination unit
Considering the temporal change in the orientation of the driver's face during at least one of the periods when the flag data indicating the face detection state is detected before and after the period when the flag data indicating the face non-detection state is detected. The operation evaluation device according to claim 1, wherein it is determined whether or not the face non-detection state is due to the driver's safety confirmation operation. The determination unit
The flag data indicating the face detection state and the flag data indicating the face non-detection state acquired by the data acquisition unit change from the face detection state to the face non-detection state and detect the face again within a predetermined time. The operation evaluation device according to claim 1 or 2, wherein when it indicates that the state has returned, it is determined that the face non-detection state is due to the safety confirmation operation of the driver. The determination unit
The flag data indicating the face detection state and the flag data indicating the face non-detection state acquired by the data acquisition unit change from the face detection state to the face non-detection state and detect the face again within a predetermined time. The operation evaluation device according to claim 1 or 2, wherein when it indicates that the state has not been returned, it is determined that the face non-detection state is not due to the driver's safety confirmation operation. .. The evaluation unit
When the determination unit determines that the face non-detection state is due to the driver's safety confirmation operation, it is evaluated as if the driver's safety confirmation operation was performed in the face non-detection state. The operation evaluation device according to any one of claims 1 to 3. It is a driving evaluation method that evaluates the driving of the driver of the vehicle.
Flag data indicating a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected at predetermined intervals while the vehicle passes through the intersection, are shown. A data acquisition step to acquire data about the driver's condition, including flag data, and
Based on the temporal relationship between the face detection state and the face non-detection state, which is indicated by the flag data indicating the face detection state and the flag data indicating the face non-detection state acquired by the data acquisition step. A determination step for determining whether or not the face non-detection state is due to the driver's safety confirmation operation, and
A driving evaluation method comprising an evaluation step for evaluating the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination step. A driving evaluation program that causes at least one computer to execute a process for evaluating the driving of a vehicle driver.
To the at least one computer
Flag data indicating a face detection state in which the driver's face is detected and a face non-detection state in which the driver's face is not detected, which are detected at predetermined intervals while the vehicle passes through the intersection, are shown. A data acquisition step to acquire data about the driver's condition, including flag data, and
Based on the temporal relationship between the face detection state and the face non-detection state, which is indicated by the flag data indicating the face detection state and the flag data indicating the face non-detection state acquired by the data acquisition step. A determination step for determining whether or not the face non-detection state is due to the driver's safety confirmation operation, and
A driving evaluation program comprising executing an evaluation step for evaluating the safety confirmation operation of the driver at the intersection in consideration of the determination result by the determination step.

Images