WO2008038376A1 - Signal recognition device, signal recognition method, signal recognition program, and recording medium - Google Patents

Abstract

A prediction section (103) predicts the display status of a signal in the advance direction based on signal display status information acquired by a signal information acquisition section (101). A behavior information acquisition section (102) acquires the behavior of a driver. Then, based on information on the predicted signal display status and information on the behavior of the driver, a determination section (104) determines the output mode of information to be notified to an occupant of a mobile body, and a notification section (105) notifies the notification information by the determined output mode.

Description

 Specification

 Signal recognition apparatus, signal recognition method, signal recognition program, and recording medium

 The present invention relates to a signal recognition device that recognizes a display state of a traffic light, a signal recognition method, a signal recognition program, and a recording medium. However, the use of the present invention is not limited to the signal recognition device, the signal recognition method, the signal recognition program, and the recording medium described above. Background art

 Conventionally, there is an in-vehicle display device that captures the front of a vehicle with a camera and converts a portion other than red in the captured image to monochrome. According to this in-vehicle display device, when the lighting color of the traffic light in the image displayed on the display screen is red, the lighting color of the traffic light is easily visually enhanced by highlighting red. Can be easily determined to be red (for example, see Patent Document 1 below). O

[0003] Patent Document 1: Japanese Unexamined Patent Publication No. 2006-115376

 Disclosure of the invention

 Problems to be solved by the invention

[0004] However, in the above-described conventional technology, if there is a red light emitted from other than the traffic light near the traffic light, the red light that is near the traffic light even if the traffic light turns blue One example is the problem that it is difficult to notice that the lighting color of the traffic light has changed. Also, if you are not looking at the display screen, the lighting color will not change. For example, while you are looking a little away, even if the lighting color of the traffic light in front changes to blue, you will not notice it. One example is the problem.

[0005] In addition, the lighting color of the front traffic light can be displayed in real time. The lighting color of the front traffic light is not captured. For example, the lighting color of the front traffic light changes after a predetermined time has elapsed. One example is the problem of not being able to notify passengers of information that is expected to be prepared and prompting the vehicle to start preparations.

 Means for solving the problem

[0006] In order to solve the above-described problems and achieve the object, the signal recognition according to the invention of claim 1 The apparatus includes: a signal information acquisition unit that acquires information on a display state of a traffic signal; and an output form of notification information that is notified to a passenger of the moving body based on the information on the display state of the traffic signal acquired by the signal information acquisition unit. Judgment means for judging, and notifying means for notifying the notice information by the output form judged by the judging means are provided.

[0007] Further, the signal recognition method according to the invention of claim 7 includes a signal information acquisition step of acquiring information on a display state of a traffic signal, and information on the display state of the traffic signal acquired by the signal information acquisition step. Based on a determination step of determining an output form of notification information to be notified to a passenger of a moving body, and a notification step of notifying notification information based on the output form determined by the determination step. .

 [0008] A signal recognition program according to the invention of claim 8 causes a computer to execute the signal recognition method according to claim 7.

 [0009] Further, a recording medium according to the invention of claim 9 is characterized in that the signal recognition program according to claim 8 is recorded in a computer-readable state.

 Brief Description of Drawings

 FIG. 1 is a block diagram showing a functional configuration of a signal recognition apparatus according to the present embodiment.

 FIG. 2 is a flowchart showing a signal recognition processing procedure of the signal recognition apparatus according to the present embodiment.

 [FIG. 3] FIG. 3 is an explanatory view showing an example of the vicinity of a dashboard of a vehicle in which a navigation device that is effective in the present embodiment is installed.

 [FIG. 4] FIG. 4 is a block diagram showing an example of a hardware configuration of a navigation apparatus that is effective in the present embodiment.

 [FIG. 5] FIG. 5 is a flowchart showing the contents of the processing of the navigation device which is effective in the present embodiment.

 [FIG. 6] FIG. 6 is an explanatory diagram showing the confirmation order of traffic lights at an intersection. Explanation of symbols

[0011] 100 signal recognition device 101 Signal information acquisition unit

 102 Behavior information acquisition unit

 103 prediction part

 104 Judgment part

 105 Notification section

 BEST MODE FOR CARRYING OUT THE INVENTION

[0012] Exemplary embodiments of a signal recognition apparatus, a signal recognition method, a signal recognition program, and a recording medium according to the present invention will be described below in detail with reference to the accompanying drawings.

[0013] (Embodiment)

 (Functional configuration of signal recognition device 100)

 First, the functional configuration of the signal recognition apparatus 100 that is relevant to the embodiment of the present invention will be described. FIG. 1 is a block diagram showing a functional configuration of the signal recognition apparatus 100 according to the present embodiment.

 In FIG. 1, the signal recognition apparatus 100 includes a signal information acquisition unit 101, a behavior information acquisition unit 102, a prediction unit 103, a determination unit 104, and a notification unit 105.

 [0015] The signal information acquisition unit 101 acquires information on the display state of the traffic light. The display state of the traffic light is, for example, information on the color of the light that is lit on the traffic light and information on whether or not it is blinking. Specifically, in the case of a vehicle traffic signal, the lighting color is one of blue, yellow, and red information, and in the case of a pedestrian traffic signal, any one of blue lighting, blue flashing, and red lighting is on. One piece of information. Moreover, the information which cannot acquire the display state of a traffic signal may be sufficient. Specifically, the signal information acquisition unit 101 may determine the display state by, for example, acquiring the image information of the traffic signal! /, Not shown !, the communication unit included in the management server or the traffic signal Information on the display state of the traffic signal to be transmitted may be acquired.

[0016] The behavior information acquisition unit 102 acquires the behavior information of the driver of the moving object. The behavior information is, for example, information on the behavior performed by the driver to move the moving object. Specifically, for example, information on depression of an accelerator pedal or a brake pedal, information on operation of a side brake or a steering wheel, current position information of a vehicle, information on an output value of G sensor force, and the like. [0017] When the signal information acquisition unit 101 acquires the display state information of the traffic light located on the side of the moving body, the prediction unit 103 calculates the prediction information of the display state of the traffic light positioned in the traveling direction of the mobile body. To do. The traffic light located on the side of the moving body is, for example, a traffic light other than the traffic light according to the moving body on which the passenger is on board. The prediction of the display state is a relatively short time prediction that can be expressed by, for example, “coming soon” until the lighting color of the traffic light in the traveling direction changes to blue.

 [0018] Further, when the signal information acquisition unit 101 cannot acquire the display state of the traffic signal, the prediction unit 103, for example, selects a forward moving body, an opposing moving body, or a route that intersects the traveling route. You may calculate the prediction information of the display state of the traffic signal located in the advancing direction of the moving body from the behavior information of the moving moving body.

 [0019] Based on the information on the display state of the traffic light acquired by the signal information acquisition unit 101, the determination unit 104 determines the output form of the notification information to be notified to the passenger of the moving body. Specifically, for example, when the lighting color of the traffic light located in the traveling direction of the moving body changes to blue, a signal for performing an operation for notifying the passenger of the moving body is output to the notification unit 105 described later. In addition, the determination unit 104 determines the output form of the notification information to be notified to the passenger of the moving body according to the prediction information of the display state of the traffic signal located in the traveling direction of the moving body calculated by the prediction unit 103. . Specifically, for example, a signal for performing a foreseeing operation to the passenger of the moving body is output to the notification unit 105.

 [0020] Further, the determination unit 104 is based on the traffic signal display state information acquired by the signal information acquisition unit 101 and the driver behavior information acquired by the behavior information acquisition unit 102. To determine the output form of the notification information to be notified. Specifically, for example, if the driver does not show a behavior to move the moving object even after a predetermined time has elapsed since the lighting color of the traffic light located in the traveling direction of the moving object turns blue, the moving object A signal is output to perform a prompting action for the passenger. In addition, as an output form of the urge action, the volume, light intensity, and action amount may be increased every time a predetermined time elapses, or only the action, action and sound, action and sound and light, etc. It may be changed.

[0021] Further, the determination unit 104 acquires the information on the display state of the traffic signal by the signal information acquisition unit 101! /, In this case, acquires the information on the display state of the traffic signal! /, Mobile body The output form of the notification information notified to the passenger is determined. More specifically, for example, a signal for performing an action indicating an intention of notifying a search object is output. This unrecognizable operation is, for example, an operation that appears as if the notification unit 105 is powered to the left and right in small increments or is squeezed.

 [0022] In addition, the notification information may be stored in a storage unit (not shown). In this case, the determination unit 104 outputs the optimum notification information output form from the stored notification information. To judge. The output form of the broadcast information to be determined may be a combination of multiple output forms. Specifically, for example, light output, sound output, and operation output may be classified, and a combination of these may be notified to the passenger of the moving body.

 The notification unit 105 notifies the notification information determined by the determination unit 104. In addition, the notification unit 105 notifies the notification information determined by the determination unit 104 by at least one output of light output, operation output, and sound output. The notification unit 105 may be a vehicle-mounted robot that imitates the shape of a person or animal. Furthermore, the vehicle-mounted robot may express a pseudo-intention or emotion that the vehicle-mounted robot expresses the intention or emotion by combining light output, motion output, and sound output. The pseudo-intention and the pseudo-emotion are, for example, an intention to notify something or an emotion of joy.

 (Signal recognition processing procedure of signal recognition apparatus 100)

 Next, a signal recognition processing procedure of the signal recognition apparatus 100 according to the present embodiment will be described. FIG. 2 is a flowchart showing a signal recognition processing procedure of the signal recognition apparatus 100 according to the present embodiment. In the flowchart of FIG. 2, first, the information on the display state of the traffic light is acquired by the signal information acquisition unit 101 (step S201). In step S 201, as the information on the display state of the traffic light, specifically, image information around the moving body is acquired by a camera mounted on the moving body.

Next, based on the information on the display state of the traffic light acquired in step S 201, the display state of the traffic signal in the traveling direction is predicted by the prediction unit 103 (step S 202). In step S202, it may be omitted if the information on the display state acquired in step S201 is only the traffic light located in the traveling direction. Further, the behavior information acquisition unit 102 acquires the behavior information of the driver (step S203). [0026] Next, either or both of the display state information acquired in step S201 and the traffic signal display state information predicted in step S202, and the operation acquired in step S203. Based on the person's behavior information, the output form of the notification information notified to the passenger of the moving body is determined (step S204). In addition, when the display state information acquired in step S201 is only the traffic light located in the traveling direction, the display state information of the traffic light acquired in step S201 and the information acquired in step S204. Based on the behavior information of the driver, the output form of the notification information to notify the passenger of the moving body is determined. Then, the notification unit 105 notifies the notification information according to the output form determined in step S204 (step S205), and the series of processing ends.

 [0027] Note that, in the case where the display state information acquired in step S201 is only the traffic light located in the traveling direction, the display state information of the traffic light located beside the moving body cannot be acquired. If the information on the display status of the traffic light located in the direction of travel is acquired before the information on the display status of the traffic signal located on the side of the moving object is acquired, There may be a case where the output form can be determined. In these cases, since the prediction state of the traffic signal located in the traveling direction cannot be predicted by the prediction unit 103, the display state information acquired in step S201 and the display state information acquired in step S203. Based on the behavior information, the output form of the notification information is determined.

 [0028] On the other hand, if the display state information acquired in step S201 is only the display state information of the traffic light located on the side of the moving body, the traffic signal display predicted in step S202 is performed. Based on the state information and the behavior information acquired in step S203, the output form of the notification information is determined. Note that when the acquired display state information is only the display state information of the traffic light located on the side of the moving body, the display state information of the traffic light located in the traveling direction may not be obtained. Conceivable.

[0029] In contrast, the display state information acquired in step S201 is the display state information of the traffic light located in the traveling direction and the display state information of the traffic light located on the side of the moving body. If the information on the display status of the traffic light located on the side of the moving body is obtained first, and the information on the display status of the traffic light located in the traveling direction is obtained later, the display status of the traffic light located in the traveling direction is obtained. Information (display obtained in step S201) Status information), the display state information of the traffic light predicted in step S202, and the behavior information acquired in step S203, the output form of the notification information is determined.

 [0030] In addition, in step S201, if the information on the display state of the misplaced traffic signal cannot be acquired, for example, the operation of the invisible operation described later by detecting that the moving body has traveled is detected. Even if you do it.

 In the flowchart of FIG. 2, the force for acquiring the driver's behavior information in step S203 is not limited to this. For example, driver behavior information need not be acquired. In this case, in step S204, the information on the display state of the traffic signal obtained in step S201 and / or the information on the display state of the traffic signal predicted in step S202, or both of them. Based on this, the output form of the notification information to notify the passenger of the moving body is determined.

 [0032] In the flowchart of FIG. 2, the force for reporting the notification information in step S205 is not limited to this. Specifically, for example, after a predetermined time elapses, the driver's behavior information may be further acquired, and the notification information may be continuously notified until the driver shows the behavior of starting. Further, every time a predetermined time elapses, the notification information may be changed and notified.

 [0033] It should be noted that the signal recognition apparatus, signal recognition method, signal recognition program, and recording medium readable by the computer according to the present invention have the functions realized by the signal recognition apparatus 100 shown in FIG. However, it is not limited to the signal recognition device 100, and a plurality of devices may be used as long as the configuration includes the functional unit shown in FIG. When connecting each functional unit as a different device, the connection between the devices may be established by performing communication by Bluetooth (registered trademark) or the like, regardless of wired or wireless.

[0034] As described above, according to the signal recognition device 100 of the present embodiment, the passenger of the moving body is determined by the determination unit 104 according to the information on the display state of the traffic signal acquired by the signal information acquisition unit 101. It is possible to determine the output form of the notification information to be notified, and to notify the notification information based on the determined output form. Therefore, it is possible to recognize the change in the lighting color of the traffic light and make the passenger aware of it. This allows the passenger to keep an eye on the traffic light, for example. When the traffic light is on, you can notice even if the lighting color of the traffic light changes to blue, and you can drive smoothly and comfortably.

 [0035] Further, for example, it may be difficult for the passenger to visually recognize the lighting color of the traffic light located in the traveling direction due to the reflection of sunlight, and even if the passenger confirms the lighting color of the traffic light, The signal recognition device 100 together with the passenger confirms the lighting color of the traffic light or predicts and notifies the change, so that the passenger can feel secure. Also, if the waiting time until the display state of the traffic signal changes (so-called signal waiting) is long, the signal recognition device 100 shares the display state of the traffic signal with the passenger, for example, feelings that are generally felt from there. Is expressed by this signal recognition device 100 and included in the notification information, the effect of reducing the frustration of the passenger can be obtained.

 [0036] Also, according to the signal recognition device 100 of the present embodiment, the information on the display state of the traffic signal acquired by the signal information acquisition unit 101 and the behavior information of the driver acquired by the behavior information acquisition unit 102 Accordingly, it is possible to determine the output form of the notification information notified to the passenger of the moving body by the determination unit 104. Therefore, when the driver starts the moving body promptly (for example, within a predetermined time), it is not possible to notify the passenger of the moving body that the lighting color of the traffic light has changed. As a result, the passenger can drive comfortably without unnecessary notification.

 [0037] Also, according to the signal recognition apparatus 100 of the present embodiment, the prediction unit 103 displays the display of the traffic signal located in the traveling direction of the mobile object from the information on the display state of the traffic signal located on the side of the mobile object. The prediction information of the state is calculated, and the determination unit 104 can determine the output form of the notification information notified to the passenger of the moving body according to the prediction information of the display state of the traffic signal located in the traveling direction of the moving body. . Therefore, before the lighting color of the traffic signal in the traveling direction changes to blue, it is possible to notify the passenger of the prediction that the lighting color of the traffic signal will change to blue. Thus, the passenger can prepare to start the moving body as soon as the lighting color of the traffic light changes to blue, and can drive smoothly.

[0038] Furthermore, according to the signal recognition device 100 of the present embodiment, when the signal information acquisition unit 101 has not acquired the information on the display state of the traffic signal, the information on the display state of the traffic signal is acquired by the determination unit 104. An output form of notification information for notifying a passenger of a mobile object to the effect The state can be judged. Therefore, it is possible to notify whether or not the traffic light is being recognized. As a result, the user watches the surroundings with caution and can immediately notice when the color of the signal light changes to blue.

 [0039] Furthermore, according to the signal recognition device 100 of the present embodiment, the signal information acquisition unit 101 can determine the display state of the traffic signal by acquiring the image information of the traffic signal. Therefore, image information can be acquired by a camera provided in a drive recorder or a navigation device. As a result, the user can obtain information on the display status of the traffic light without installing a new camera, so there is no need to add extra parts, and parts costs can be reduced. Absent.

 [0040] Also, according to the signal recognition device 100 of the present embodiment, the output form of the notification information determined by the determination unit 104 is notified by at least one output form of light output, operation output, and sound output. can do. Therefore, by combining light output, motion output, and sound output, it is possible to convey pseudo-intention and emotion to the passenger. This makes it possible to convey the nuances of various notification information such as foresight, awareness, and prompting in an easy-to-understand manner. This also allows passengers to enjoy driving comfortably while reducing the frustration caused by waiting for traffic lights, as if there are passengers and pets who can feel the same as waiting for traffic lights. Can do.

 Example

 [0041] Examples of the present invention will be described below. In the present embodiment, an example in which the signal recognition device of the present invention is implemented by a navigation device mounted on a moving body such as a vehicle (including a four-wheeled vehicle and a two-wheeled vehicle) will be described.

 [0042] (Peripheral device configuration of navigation device)

 First, the configuration of peripheral devices of the navigation device that is useful in this embodiment will be described with reference to FIG. FIG. 3 is an explanatory view showing an example of the vicinity of a dashboard of a vehicle in which a navigation device that is useful in this embodiment is installed.

In FIG. 3, the navigation apparatus 300 is installed on the dashboard of the vehicle. The navigation device 300 includes a main body M and a display unit (display) D. The display unit D displays the current location of the vehicle, map information, and the current time. [0044] Also, the navigation apparatus 300 is connected to an in-vehicle robot 310 installed on the dashboard. The in-vehicle robot 310 includes a camera, a lamp, a microphone, a speaker, and the like (not shown), and performs various outputs according to control of a control signal output from the navigation device 300.

 [0045] Further, the vehicle-mounted robot 310 has a shape imitating a human or an animal, for example, and includes a drive unit such as an arm or a drive unit such as a leg on the left and right sides. Therefore, the operation output of the drive unit is performed according to the control of the control signal output from the navigation device 300. Furthermore, the vehicle-mounted robot 310 may be configured to include a drive unit that swings the top and bottom, left and right, and a drive unit that moves the body part to the left and right, and controls the control signals output from the navigation device 300. Therefore, the operation output of the drive unit may be performed.

 [0046] Specifically, for example, the top of the vehicle-mounted robot 310 may function as a camera. The camera may rotate in the horizontal direction and the vertical direction to capture images inside and outside the vehicle. it can. Further, the vehicle-mounted robot 310 may be configured to have a lamp, microphone, and speaker with a lamp, microphone, and speaker on the body, and may collect and output light and sound with a lamp, microphone, and speaker. In addition, a display unit for displaying images, characters, or the like may be provided, or a function for speaking words may be provided.

 [0047] In addition, the vehicle-mounted robot 310 may be turned on and blinking in various colors in accordance with control of a control signal output from the navigation device 300 that may emit light entirely or partially. It is also possible to change the blinking pattern.

 [0048] With the configuration shown in FIG. 3, in this embodiment, the vehicle-mounted robot 310 performs various outputs according to the control of the control signal output from the navigation device 300, thereby driving the suspicious intention or the pseudo emotion. It becomes the structure shown to the person!

[0049] Although details will be described with reference to FIGS. 5 and 6, the vehicle-mounted robot 310 for indicating pseudo-intentions and pseudo-emotions according to the information on the display state of the traffic lights and the behavior information of the driver. Determine the operating program to be used. Then, the vehicle-mounted robot 310 that operates according to the determined operation program can notify the passenger of the change in the lighting color of the traffic light. [0050] In other words, the vehicle-mounted robot 310 is used as a driver's anxious presence, such as when a child or a pet is on board, and the driver is concerned about safe driving. The driver waits for the lighting color to change with the driver and expresses emotions such as the willingness to notify the power and the feeling of pleasure according to the driver's behavior. It is possible to notice changes in lighting color and to reduce frustration caused by waiting for a signal.

 [0051] (Hardware configuration of navigation device 300)

 Next, the hardware configuration of the navigation apparatus 300 that is useful in the present embodiment will be described with reference to FIG. FIG. 4 is a block diagram showing an example of a hardware configuration of a navigation apparatus that works on the present embodiment.

 In FIG. 4, a navigation device 300 is mounted on a moving body such as a vehicle, and includes a CPU 401, a ROM 402, a RAM 403, a magnetic disk drive 404, a magnetic disk 405, and an optical disk drive 406. , Optical disc 407, audio IZF (interface) 408, microphone 409, speaker 410, input device 411, video IZF412, display 413, communication IZF414, GPS unit 415, various sensors 416, camera 417 and a drive unit 418. Each component 401 to 418 is connected by a bus 420.

 First, the CPU 401 governs overall control of the navigation device 300. The ROM 402 stores programs such as a boot program, a route search program, a route guidance program, a voice generation program, a map information display program, an operation program, and a signal lighting color prediction program. The RAM 403 is used as a work area for the CPU 401.

 Here, the route search program searches for an optimum route from the departure point to the destination point using map information or the like recorded on the optical disc 407 described later. Here, the optimal route is the shortest (or fastest) route to the destination or the route that best meets the conditions specified by the user. The guidance route searched by executing the route search program is output to the audio IZF 408 and the video IZF 412 via the CPU 401.

[0055] In addition, the route guidance program includes the guidance route information searched by executing the route search program, the current state of the navigation device 300 acquired by the communication IZF414. Based on the location information and the map information read from the optical disc 407, real-time route guidance information is generated. The route guidance information generated by executing the route guidance program is output to the audio IZF 408 and the video IZF 412 via the CPU 401.

 [0056] The sound generation program generates tone and sound information corresponding to the pattern.

. That is, based on the route guidance information generated by executing the route guidance program, the virtual sound source corresponding to the guidance point is set and the voice guidance information is generated, and output to the voice IZF 408 via the CPU 401.

 In addition, the map information display program determines the display format of the map information displayed on the display 413 by the video I / F 412 and displays the map information on the display 413 according to the determined display format.

 [0058] Further, the operation program drives a drive unit 418, which will be described later, in accordance with the information on the display state of the traffic light and the behavior information of the driver. Details will be described with reference to FIGS. 5 and 6. The force operation program, for example, causes the drive unit 418 to indicate the pseudo-intention or the emotion selected by the information on the display state of the traffic lights or the behavior information of the driver. Select the operation program to be executed.

 [0059] Further, the signal lighting color prediction program calculates the prediction information of the display state of the traffic light in the traveling direction of the vehicle based on the display state information of the traffic light on the side of the vehicle. Details will be described with reference to FIG. 5 and FIG. 6, but the signal lighting color prediction program predicts the traffic signal in the traveling direction of the vehicle after a predetermined time, for example, by analyzing an image captured by a camera 417 described later. Display state information.

 [0060] The magnetic disk drive 404 controls reading and writing of data to the magnetic disk 405 according to the control of the CPU 401. The magnetic disk 405 records data written under the control of the magnetic disk drive 404. As the magnetic disk 405, for example, HD (node disk) or FD (flexible disk) can be used.

In addition, the optical disk drive 406 controls reading and writing of data with respect to the optical disk 407 according to the control of the CPU 401. The optical disc 407 is a detachable recording medium from which data is read according to the control of the optical disc drive 406. Light di The disk 407 can also use a writable recording medium. The removable recording medium may be a power MO of the optical disc 407, a memory card, or the like.

 As an example of information recorded on the magnetic disk 405 and the optical disk 407, there is map information used for route search and route guidance. The map information includes background data that represents features (features) such as buildings, rivers, and the ground surface, and road shape data that represents the shape of the road, and can be displayed in two or three dimensions on the display screen of the display 413. To be drawn. When the navigation device 300 is guiding a route, the map information and the current location of the host vehicle acquired by the GPS unit 415 described later are displayed in an overlapping manner.

 In this embodiment, the map information is recorded on the magnetic disk 405 and the optical disk 407. However, the present invention is not limited to this. The map information may be provided outside the navigation device 300, not the information recorded only in the one integrated with the navigation device 300 hardware. In that case, the navigation device 300 acquires the map information via the network through the communication IZF 414, for example. The acquired map information is stored in the RAM 403 or the like.

 The audio IZF 408 is connected to an audio input microphone 409 and an audio output speaker 410. Audio received by the microphone 409 is AZD converted in the audio IZF408. In addition, sound is output from the speaker 410. Note that the voice input from the microphone 409 can be recorded on the magnetic disk 405 or the optical disk 407 as voice data.

 [0065] The microphone 409 and the speaker 410, for example, may be installed in a drive unit 418 (for example, the vehicle-mounted robot 310 in FIG. 3) described later, and may be rotationally driven according to the control of the operation program described above. It is also possible to input / output audio.

 [0066] Further, examples of the input device 411 include a remote controller, a keyboard, a mouse, a touch panel, and the like provided with a plurality of keys for inputting characters, numerical values, various instructions, and the like.

The video I / F 412 is connected to the display 413 and the camera 417. Specifically, the video I / F 412 includes, for example, a graphic controller that controls the entire display 413, a buffer memory such as VRAM (Video RAM) that temporarily records image information that can be displayed immediately, and a graphic Based on image data output from the controller And a control IC for controlling the display of the display 413.

[0068] The display 413 displays icons, cursors, menus, windows, or various data such as characters and images. As this display 413, for example, a CRT, a TFT liquid crystal display, a plasma display, or the like can be adopted. The display 41 3 is installed, for example, in a manner like the display unit D in FIG.

[0069] Further, the display 413 and the camera 417 are, for example, imaged according to the control of the operation program described above, which may be installed in a drive unit 418 (for example, the vehicle-mounted robot 310 in FIG. 3) described later. Or light output.

[0070] Specifically, for example, the camera 417 captures an image inside or outside the vehicle.

 The image may be either a still image or a moving image. For example, the behavior of the passenger inside the vehicle may be imaged by the camera 417, or the display state of the traffic signal outside the vehicle may be imaged. For example, the captured video is output to a recording medium such as a magnetic disk 405 or an optical disk 407 via the video I / F 412, and used to acquire information on the display state of a traffic light and driver's behavior information described later. As well.

 In addition, the communication IZF 414 is connected to a network via radio and functions as an interface between the navigation device 300 and the CPU 401. The communication IZF 414 is further connected to a communication network such as the Internet via radio, and also functions as an interface between the communication network and the CPU 401.

 [0072] Communication networks include LANs, WANs, public line networks and mobile phone networks. Specifically, the communication IZF414 is composed of, for example, FM tuner, VICS (Vehicle Information and Communication System) Z beacon Resino, wireless navigation device, and other navigation devices. Get road traffic information such as traffic regulations. VICS is a registered trademark.

[0073] Further, the GPS unit 415 uses a received wave from the GPS satellite and output values from various sensors 416 (for example, an angular velocity sensor, an acceleration sensor, a tire rotation number, etc.) described later, Information indicating the current location of the location piggy-on device 300 is calculated. The information indicating the current location is information that identifies one point on the map information, such as latitude'longitude and altitude. The GPS unit 415 uses the output values from the various sensors 416 to Outputs domain, speed change, and direction change. This makes it possible to analyze dynamics such as sudden braking and sudden handling.

 [0074] The various sensors 416 are a vehicle speed sensor, an acceleration sensor, an angular velocity sensor, and the like, and their output values are used for calculation of the current position by the GPS unit 415 and measurement of changes in speed and direction. The various sensors 416 also include sensors that detect each operation of the vehicle by the driver. The detection of each operation of the vehicle may be configured to detect, for example, steering wheel operation, blinker input, side brake operation, accelerator pedal depression, or brake pedal depression.

 [0075] The signal information acquisition unit 101, the behavior information acquisition unit 102, the prediction unit 103, the determination unit 104, and the notification unit 105 included in the signal recognition device 100 illustrated in FIG. 1 are the same as those in the navigation device 300 illustrated in FIG. The CPU 401 executes a predetermined program using programs and data recorded in the ROM 402, RAM 403, magnetic disk 405, optical disk 407, etc., and controls each part in the navigation device 300 to realize its function.

 That is, the navigation device 300 of the embodiment includes the signal recognition device 100 shown in FIG. 1 by executing the signal recognition program recorded in the ROM 402 as a recording medium in the navigation device 300. The function can be executed by the signal recognition processing procedure shown in Fig. 2.

 [0077] (Contents of processing of navigation device)

 Next, the contents of the processing of the navigation device 300 that is useful in the present embodiment will be described. FIG. 5 is a flowchart showing the contents of the processing of the navigation apparatus that is useful in this embodiment. In the flowchart of FIG. 5, first, the vehicle waits until it stops (step S501: No loop), and when the vehicle stops (step S501: Yes), confirms that there is a traffic signal around the vehicle (step S501: No). S502). In step S502, whether or not there is a traffic light is specifically checked by, for example, a camera in which the traffic light is reflected in an image around the vehicle captured by the camera mounted on the vehicle-mounted robot 310. Check whether it is a success or failure.

 [0078] Next, it is determined whether or not the lighting color of the traffic signal in the traveling direction is blue (step S503).

The traffic signal in the direction of travel is a traffic signal that is located on the road on which the vehicle is traveling and that the vehicle follows. If it is determined in step S503 that the traffic light is blue (step Step S503: Yes), the process proceeds to Step S510 described later. Also, in step S503, if the lighting color of the traffic light in the direction of travel is not blue (step S503: No), or if the lighting color of the traffic light in the direction of travel cannot be confirmed, the side traffic light is monitored. (Step S504). The side traffic light will be described in detail in FIG. 6. For example, when the vehicle stops at the intersection, it is a traffic light other than the traffic light located in the traveling direction of the vehicle.

 [0079] Then, it waits until the side pedestrian traffic light blinks (step S505: No loop). As will be described in detail in Fig. 6, in general, when a pedestrian traffic signal and a vehicle traffic signal are installed in the same direction, the lighting color of the pedestrian traffic signal is first blue, and then flashes blue. , Change in red order. The lighting color of the vehicle traffic light changes from blue to yellow to red. If the side pedestrian traffic light blinks in step S505 (step S505: Yes), the system waits until the side vehicle traffic light turns yellow (step S506: No loop).

 [0080] In step S506, when the side vehicle traffic light turns yellow (step S506: Yes), the vehicle robot 310 performs the foreseeing operation (step S507). O In step S507, The foreseeing action is an action that indicates to the vehicle passenger that the lighting color of the signal in the direction of travel will soon turn blue. For example, it is an operation that includes the intention of foreseeing notification and the feeling of expectation for resuming traveling, such that the vehicle-mounted robot 310 is preparing to run. Specifically, for example, imitating the posture of a person ready to run, bend the drive part such as the arm of the in-vehicle robot 310 and slightly bend the drive part such as the foot, To do. Further, force voice information such as a speaker provided in the vehicle-mounted robot 310 may be output. Specifically, the voice information is, for example, information such as “It will soon be blue”.

[0081] Next, the traffic signal in the traveling direction is monitored (step S508). If the lighting color of the traffic light in the traveling direction cannot be confirmed at the stage of step S503, the process proceeds to step S510 after step S507. Wait until the traffic light in the direction of travel turns blue (step S509: No loop). In step S509, when the lighting color of the traffic light in the traveling direction turns blue (step S509: Yes), the vehicle-mounted robot 310 performs a notice operation (step S510). Awareness movements let the vehicle occupants This is an action that indicates that the color of the traffic light has changed to blue. For example, this is an operation including the intention of notifying and the feeling of expectation for the further resumption of driving, for example, the operation in which the vehicle-mounted robot 310 starts to move. Specifically, the drive unit such as the arm of the in-vehicle robot 310 is powered up or down to point straight in the direction of the traffic light, and the drive unit such as the foot is bent slightly to power up and down slowly. To do. Also, the sound information such as a speaker provided in the vehicle-mounted mouth bot 310 may be output. Specifically, the audio information is information such as “It turned blue”.

[0082] Further, the behavior information of the driver is acquired (step S511), and it is determined whether or not the vehicle has started (step S512) based on the behavior information acquired in step S511. Specifically, in step S512, the determination of whether or not there is a starting action is made by, for example, determining whether or not the driver has released the side brake that was being pulled while the vehicle was stopped, and whether or not the driver released the brake pedal. No, whether the driver stepped on the accelerator pedal, whether the driver powered the steering wheel, whether the vehicle started driving, whether the vehicle changed its current position, etc. Is determined from information detected by the GPS unit 415 and various sensors 416.

 If there is no start operation in step S512 (step S512: No), the process waits until a predetermined time has passed (step S513: No loop). In step S513, the predetermined time is generally the time required to make a force start operation after visually confirming that the lighting color of the traffic light in the traveling direction is blue. Here, the predetermined time may be a predetermined time obtained by calculating an average value or a maximum value excluding a singular value from a history of past behavior information for each driver.

[0084] Further, in step 513, it may be determined from the image around the vehicle imaged by the camera whether or not the power is in a state where the vehicle can start running after the lighting color of the signal turns blue. For example, from the image around the vehicle, even if the lighting color of the traffic light changes to blue, the traffic light in the direction of travel is recognized to be congested ahead (for example, the previous vehicle is not traveling) and starts traveling. It can be judged that the situation is not possible. In this case, it is assumed that the travel cannot be started even after the predetermined time has elapsed, so the restriction of the predetermined time is removed, and when the travel is started, the pleasure operation of step S515 described later is performed. May [0085] When a predetermined time has elapsed in step S513 (step S513: Yes), the in-car mouth bot 310 performs a prompting operation (step S514). The prompting action is an action that prompts the vehicle occupant to start the vehicle. For example, this is a behavior where a child is obsessed with “Let's go fast.” Specifically, a drive unit such as an arm provided in the vehicle-mounted robot 310 is raised upward and directed toward the driver, and the drive unit such as a foot is bent slightly to power back and forth. Also, for example, you can output a pseudo-intention such as “Let's go fast” or “Don't panic”!

 On the other hand, if there is a start motion in step S512 (step S512: Yes), a joyful motion is performed by the vehicle-mounted robot 310 (step S515), and the series of processes is terminated. The joy operation is an operation that expresses the joy that the vehicle has started. Specifically, the drive unit such as the torso of the in-vehicle robot 310 is moved to the right so that it is directed toward the driver, the drive unit such as the top of the head is slowly moved up and down, and Move the drive unit to the left again in the original direction. Also, for example, audio information such as “let's go” may be output. Also, if the traffic light in the direction of travel is blue in step S503 (step S503: Yes), proceed directly to step S510 to perform the subsequent processing!

 In the flowchart of FIG. 5, although it is assumed that there is a traffic light in step S502, the present invention is not limited to this. For example, when the traffic light cannot be confirmed, the vehicle-mounted robot 310 may perform a search operation. The action of looking for is an action that indicates the intention of notifying that it is trying to find a traffic light. Specifically, for example, a driving unit such as the top of the vehicle-mounted robot 310 is vigorously moved to the left and right to perform operations such as searching for the kiyoguchi and the surrounding area. For example, audio information such as “I can't see the traffic light” may be output.

[0088] Further, in the flowchart of FIG. 5, the force for monitoring the side traffic light in step S504 is not limited to this. Specifically, for example, when a side traffic signal cannot be monitored from the host vehicle, such as when a side traffic signal cannot be monitored by a vehicle ahead or an oncoming vehicle, the process proceeds to step S508, and a traffic signal in the direction of travel is set. You can monitor it.

 Further, in the flowchart of FIG. 5, in step S502 and step S508, the traffic signal in the traveling direction is confirmed and monitored, but the present invention is not limited to this. Specifically, for example, when a traffic light in the direction of travel cannot be confirmed and monitored by a large vehicle such as a truck in front of the vehicle, the process proceeds from step S501 to step S504, and the traffic light on the side is monitored. Proceed from step S507 to step S510 to perform the action you noticed.

 In the flowchart of FIG. 5, the foreseeing operation is performed in step S507, but the present invention is not limited to this. For example, the first foreseeing action is performed when the sidewalk traffic lights blink, and then the second foreseeing is performed when the sidewalk traffic lights turn red. The third foreseeing operation is performed when the side vehicle traffic light turns yellow, and the side vehicle traffic light turns red. The foreseeing operation may be performed. In this case, even if the operation is continued until the first foreseeing operation changes to the fourth foreseeing operation, the movement may gradually increase or become faster. Good. The power of the first foreseeing The operation of the foreseeing until the fourth foreseeing may be set by the occupant, or determined from the past history of the occupant's behavior information It may be configured.

 Further, in the flowchart of FIG. 5, the force for performing the prompting operation in step S514 is not limited to this. Specifically, for example, the operation, volume, and amount of light may be increased every time a predetermined time elapses. Also, the light color and blinking pattern may be changed. Further, for example, initially, only the operation is performed without outputting the voice or light, and the output of the voice or light may be added as a predetermined time elapses.

In the flowchart of FIG. 5, the prompting operation is performed in step S514, but the present invention is not limited to this. Specifically, for example, when it is determined that the driver's line of sight is detected and the traffic light is viewed, the prompting operation may not be performed. Further, in the flowchart of FIG. 5, the lighting color and lighting state of each traffic light may be output as audio information or the like by the operation of the passenger. Passenger operations can be performed using touch panels and sound. Even voice input.

 [0093] (Confirmation order of traffic lights at intersections)

 Next, the confirmation order of traffic lights at the intersection will be described with reference to FIG. Fig. 6 is an explanatory diagram showing the confirmation order of traffic lights at the intersection. FIG. 6 shows the host vehicle 601, the vehicle 602 ahead, the vehicle traffic signal 610 in the traveling direction, the side pedestrian traffic signal 620, and the side vehicle traffic signal 630. In the vehicle traffic signal 610 in the traveling direction, the respective lighting colors are lit in the order of blue 611, yellow 612, and red 613.

 [0094] Next, the lighting color of the side vehicle traffic light 630 and the side pedestrian traffic light 620 turns blue. After the predetermined time has elapsed, first, the blue 621 light on the side pedestrian traffic light 620 flashes and changes to red 622. After that, the lighting color of the side vehicle traffic light 630 changes from blue 631 to yellow 632 and then red 633. Next, the lighting color of the vehicle traffic signal 610 in the traveling direction changes to blue 611. In this way, at the intersection, the traffic lights are colored in the order of the pedestrian traffic light 620 on the side and the traffic light 630 on the side when the lighting color of the vehicle traffic light 610 in the traveling direction is yellow 612 and red 613. Next, it is possible to measure the approximate time until the lighting color of the vehicle traffic signal 610 in the traveling direction becomes blue 611.

 Therefore, as shown in steps S504 to S507 in the flowchart of FIG. 5, the display state information of the side pedestrian traffic light 620 and the side vehicle traffic light 630 is acquired, and the process proceeds. It is possible to predict that the lighting color of the vehicle traffic signal 610 in the direction will turn blue.

[0096] As described above, according to the navigation device 300 of the present embodiment, the determination unit 104 determines whether the vehicle passenger is in accordance with the information on the display state of the traffic signal acquired by the signal information acquisition unit 101. It is possible to determine the output form of the notification information to be notified and notify the notification information in the determined output form. Therefore, it is possible to recognize the change in the lighting color of the traffic light and make the passenger aware of it. This makes it possible for passengers to notice when the traffic light changes to blue when they keep their eyes away from the traffic light, for example. it can. [0097] In addition, for example, it may be difficult for the passenger to see the lighting color of the traffic light located in the traveling direction due to the reflection of sunlight, and even if the passenger confirms the lighting color of the traffic light, Since the lighting color of the traffic light is confirmed together with the passenger or the change is predicted and notified, the passenger can feel secure. In addition, if the waiting time until the display state of the traffic signal changes (so-called signal waiting) is long, the display state of the traffic signal is shared with the passenger, for example, expressing emotions that are generally felt from there, By including it, the effect of reducing the frustration of the passenger can be obtained.

 Further, according to the navigation device 300 of the present embodiment, the information on the display state of the traffic light acquired by the signal information acquisition unit 101, the behavior information of the driver acquired by the behavior information acquisition unit 102, and Accordingly, the determination unit 104 can determine the output form of the notification information notified to the passenger of the moving object. Therefore, when the driver starts the moving body promptly (for example, within a predetermined time), it is not possible to notify the passenger of the moving body that the lighting color of the traffic light has changed. As a result, the passenger can drive comfortably without unnecessary notification.

 [0099] Further, according to the navigation device 300 of the present embodiment, the prediction state 103 causes the display state of the traffic signal located in the traveling direction of the mobile object from the information on the display state of the traffic signal located on the side of the mobile object. The prediction information can be calculated, and the determination unit 104 can determine the output form of the notification information notified to the passenger of the moving body according to the prediction information of the display state of the traffic signal located in the traveling direction of the moving body. Therefore, before the lighting color of the traffic signal in the traveling direction changes to blue, it is possible to notify the passenger of the prediction that the lighting color of the traffic signal will change to blue. Thus, the passenger can prepare to start the moving body as soon as the lighting color of the traffic light changes to blue, and can drive smoothly. In addition, for example, even if a traffic light is not visible due to the presence of a large vehicle such as a truck, it is possible to prepare to start smoothly following the movement of the large vehicle.

[0100] Also, according to the navigation device 300 of the present embodiment, when the signal information acquisition unit 101 has not acquired the information on the display state of the traffic signal, the determination unit 104 acquires the information on the display state of the traffic signal. An output form of notification information for notifying the passenger of the moving body The state can be judged. Therefore, it is possible to notify whether or not the traffic light is being recognized. As a result, the user watches the surroundings with caution and can immediately notice when the color of the signal light changes to blue.

 [0101] Further, according to the navigation device 300 of the present embodiment, the signal information acquisition unit 101 can determine the display state of the traffic signal by acquiring the image information of the traffic signal. Therefore, image information can be acquired by a camera provided in a drive recorder or a navigation device. As a result, the user can obtain information on the display status of the traffic light without installing a new camera, so there is no need to add extra parts, and parts costs can be reduced. Absent.

[0102] Also, according to the navigation device 300 of the present embodiment, the output form of the notification information determined by the determination unit 104 is notified by at least one of the light output, the operation output, and the sound output. be able to. Therefore, by combining light output, motion output, and sound output, it is possible to convey pseudo-intention and emotion to the passenger. As a result, it is possible to notice the foreseeing and convey the nuances of various notification information such as prompts in an easy-to-understand manner. This also allows the passengers to feel as if there are other passengers and pets, reducing the frustration of waiting for traffic lights and having fun! ¹⁾ Can drive properly.

As described above, according to the signal recognition device, signal recognition method, signal recognition program, and recording medium of the present invention, the navigation device 300 is a vehicle imaged by the camera 417 mounted on the vehicle. Depending on the information on the display status of surrounding traffic signals, at least one of light output, motion output, and sound output from the drive unit (vehicle-mounted robot) can be used to change the traffic signal located in the direction of travel for the vehicle occupant. Can be notified by. As a result, the passenger can share emotions with the drive unit, and if the lighting color of the traffic light changes, the passenger can notice the change in the lighting color by being notified easily. At the same time, it reduces the frustration caused by waiting for traffic lights and makes it fun! ¹⁾ You can drive properly.

[0104] The signal recognition method described in the present embodiment is such that a program prepared in advance is executed on a computer such as a personal computer or a workstation. It can be realized more. This program is recorded on a computer-readable recording medium such as a hard disk, a flexible disk, a CD-ROM, an MO, and a DVD, and is executed by reading the recording medium force by the computer. The program may be a transmission medium that can be distributed through a network such as the Internet.

Claims

The scope of the claims  [1] Signal information acquisition means for acquiring information on the display state of the traffic light,  Determination means for determining an output form of notification information to notify a passenger of a moving body based on information on a display state of the traffic light acquired by the signal information acquisition means;  A signal recognizing device comprising: notifying means for notifying notification information in accordance with the output form determined by the determining means.  [2] comprising behavior information acquisition means for acquiring the behavior information of the driver of the moving body,  The determination means is notification information for informing a passenger of the moving body based on the information on the display state of the traffic signal acquired by the signal information acquisition means and the behavior information of the driver acquired by the behavior information acquisition means. The signal recognition apparatus according to claim 1, wherein the output form is determined. [3] When the information on the display state of the traffic signal located on the side of the moving body is acquired by the signal information acquisition means, the prediction for calculating the prediction information on the display state of the traffic signal positioned in the traveling direction of the mobile body With means,  The determination unit determines an output form of notification information to be notified to a passenger of the mobile body according to the prediction information of the display state of the traffic signal located in the traveling direction of the mobile body calculated by the prediction unit. The signal recognition device according to claim 1, wherein: [4] The determination means obtains the information on the display state of the traffic light by the signal information acquisition means! /, In this case, obtains the information on the display state of the traffic light! /, 2. The signal recognition apparatus according to claim 1, wherein an output form of notification information notified to a passenger of a moving body is determined.  [5] The signal recognition device according to claim 1, wherein the signal information acquisition unit acquires image information of the traffic light.  6. The informing means informs the output form of the informing information determined by the determining means by at least one output form of light output, operation output, and sound output. Signal recognition apparatus as described in any one of -5. [7] A signal information acquisition step of acquiring information on the display state of the traffic light, Based on the information on the display state of the traffic signal acquired by the signal information acquisition step, a determination step of determining an output form of notification information to notify a passenger of a moving body;  A notification step of notifying notification information according to the output form determined by the determination step.  [8] A signal recognition program for causing a computer to execute the signal recognition method according to claim 7.  [9] A computer-readable recording medium on which the signal recognition program according to claim 8 is recorded.