WO2015099463A1 - Vehicle driving assist device and vehicle having same - Google Patents

Abstract

The present invention relates to a vehicle driving assist device and a vehicle having the same. A vehicle driving assist device according to an embodiment of the present invention includes a processor generating a tunnel mode control signal for calculating whether a vehicle enters a tunnel and operating a vehicle in a tunnel mode when the calculated vehicle enters the tunnel, on the basis of a stereo camera and a stereo image received from the stereo camera, wherein the tunnel mode control signal includes a control signal for controlling at least one of a sunroof drive unit, a lamp drive unit, a window drive unit, and an air conditioning drive unit within a vehicle. Therefore, on the basis of a photographed image, a vehicle interior configuration can simply be performed when entering the tunnel.

Description

Vehicle driving assistance device and vehicle having same

The present invention relates to a vehicle driving assistance apparatus and a vehicle having the same, and more particularly, to a vehicle driving assistance apparatus and a vehicle having the same, based on the photographed image, which can easily perform the internal setting of the vehicle when entering the tunnel. will be.

The vehicle is a device for moving in the direction desired by the user on board. An example is a car.

On the other hand, for the convenience of the user using the vehicle, various types of sensors, electronic devices, etc. are provided. In particular, various devices for driving convenience of the user have been developed.

SUMMARY OF THE INVENTION An object of the present invention is to provide a vehicle driving assistance device and a vehicle having the same, which can easily set up a vehicle interior when entering a tunnel based on a photographed image.

Vehicle driving assistance apparatus according to an embodiment of the present invention for achieving the above object, based on a stereo camera and a stereo image received from the stereo camera, calculates whether the vehicle enters the tunnel, when the calculated vehicle tunnel entry, And a processor configured to generate a tunnel mode control signal for operating the vehicle in the tunnel mode, wherein the tunnel mode control signal includes a control signal for controlling at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioning driver in the vehicle. Include.

On the other hand, the vehicle driving assistance apparatus according to an embodiment of the present invention for achieving the above object, based on the mono camera, the radar, the mono image received from the mono camera and the distance information from the radar, whether the vehicle enters the tunnel And a processor for generating a tunnel mode control signal for operating the vehicle in the tunnel mode when the vehicle enters the tunnel, wherein the tunnel mode control signal includes a sunroof driver, a lamp driver, a window driver, and air conditioning in the vehicle. And a control signal for controlling at least one of the drivers.

On the other hand, the vehicle driving assistance apparatus according to an embodiment of the present invention for achieving the above object, based on the rider for scanning the external object and the scan image received from the rider, calculates whether the vehicle enters the tunnel And a processor configured to generate a tunnel mode control signal for operating the vehicle in the tunnel mode when the vehicle enters the tunnel, wherein the tunnel mode control signal includes at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioning driver of the vehicle. It includes a control signal for controlling one.

On the other hand, the vehicle according to an embodiment of the present invention for achieving the above object, the sensor unit for sensing the vehicle state, the sunroof driver for driving the sunroof, the lamp driver for driving the lamp, the window for driving the window On the basis of the driving unit, the air conditioning driving unit for driving the air conditioning unit, the sunroof driving unit, the lamp driving unit, the control unit for controlling the window driving unit, the air conditioning driving unit, and the stereo image received from the stereo camera, the stereo camera, whether the vehicle enters the tunnel And a vehicle driving assistance device having a processor configured to generate a tunnel mode control signal for operating the vehicle in a tunnel mode when the vehicle enters the calculated tunnel. The tunnel mode control signal includes: a sunroof driver in a vehicle, a ramp And a control signal for controlling at least one of the driver, the window driver, and the air conditioning driver.

The vehicle driving assistance apparatus according to an embodiment of the present invention calculates whether the vehicle enters a tunnel based on a stereo image, and generates a tunnel mode control signal for operating the vehicle in the tunnel mode when the calculated vehicle enters the tunnel. As a result, at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioning driver in the vehicle can be controlled. Accordingly, based on the captured image, it is possible to easily perform the internal setting of the vehicle when entering the tunnel.

On the other hand, based on the vehicle surrounding information, and calculates whether or not to exit the tunnel, when exiting the tunnel, by generating a pre-tunnel control signal for operating the vehicle in the pre-tunnel mode, the sunroof driver, lamp driver, window driver in the vehicle At least one of the air conditioning driving unit may be controlled. Accordingly, based on the captured image, it is possible to set the internal setting of the vehicle at the time of exiting the tunnel in the same manner as before entering the tunnel.

Meanwhile, the processor of the vehicle driving assistance apparatus may generate the tunnel mode control signal based on the stereo image and the map information, and thus, the accuracy of the tunnel mode control signal may be improved.

On the other hand, based on a stereo image, when detecting an object, the disparity can be calculated using the stereo image, and the object can be detected based on the disparity information, thereby reducing the data processing speed and the like.

1 is a view showing the appearance of a vehicle having a stereo camera according to an embodiment of the present invention.

FIG. 2 is a view illustrating an appearance of a stereo camera attached to the vehicle of FIG. 1.

3A to 3B illustrate various examples of an internal block diagram of a vehicle driving assistance apparatus according to an embodiment of the present invention.

4A-4B illustrate various examples of internal block diagrams of the processor of FIGS. 3A-3B.

5A through 5B are views referred to for describing an operation of the processor of FIGS. 4A through 4B.

6A to 6B are views referred to for describing the operation of the vehicle driving assistance apparatus of FIGS. 3A to 3B.

FIG. 7 is an example of an internal block diagram of the electronic control apparatus in the vehicle of FIG. 1.

8 is a flowchart illustrating a method of operating a vehicle driving assistance apparatus according to an embodiment of the present invention.

9 to 13C are views referred to for describing the operating method of FIG. 8.

14 is a diagram illustrating an exterior of a vehicle including a mono camera according to another embodiment of the present invention.

15A illustrates an example of an internal block diagram of a vehicle driving assistance apparatus according to another embodiment of the present invention.

15B illustrates an example of an internal block diagram of a vehicle driving assistance apparatus according to another embodiment of the present invention.

16A-16B illustrate various examples of internal block diagrams of the processor of FIG. 15A.

17A through 17B are views for explaining an operation of the processor of FIG. 16A.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The suffixes "module" and "unit" for components used in the following description are merely given in consideration of ease of preparation of the present specification, and do not impart any particular meaning or role by themselves. Therefore, the "module" and "unit" may be used interchangeably.

The vehicle described herein may be a concept including an automobile and a motorcycle. In the following, a vehicle is mainly described for a vehicle.

Meanwhile, the vehicle described herein may be a concept including a vehicle having an engine, a hybrid vehicle having an engine and an electric motor, an electric vehicle having an electric motor, and the like. Hereinafter, the description will be given mainly on a vehicle provided with an engine.

Meanwhile, the vehicle driving assistance apparatus described in the present specification may be referred to as an Advanced Driver Assistance Systems (ADAS) or an Advanced Driver Assistance Apparatus (ADAA). Hereinafter, a vehicle driving assistance apparatus for a vehicle and a vehicle having the same according to various embodiments of the present disclosure will be described.

1 is a view showing the appearance of a vehicle having a stereo camera according to an embodiment of the present invention.

Referring to the drawings, the vehicle 200 includes wheels 103FR, 103FL, 103RL,... Rotated by a power source, a steering wheel 150 for adjusting a traveling direction of the vehicle 200, and an interior of the vehicle 200. It may be provided with a stereo camera 195 provided in.

The stereo camera 195 may include a plurality of cameras, and the stereo image obtained by the plurality of cameras may be signal processed in the vehicle driving assistance apparatus (100 of FIG. 3).

Meanwhile, the drawing illustrates that the stereo camera 195 includes two cameras.

FIG. 2 is a view illustrating an appearance of a stereo camera attached to the vehicle of FIG. 1.

Referring to the drawings, the stereo camera module 195 may include a first camera 195a having a first lens 193a and a second camera 195b having a second lens 193b.

Meanwhile, the stereo camera module 195 includes a first light shield 192a and a second light for shielding light incident on the first lens 193a and the second lens 193b, respectively. The shield 192b may be provided.

The stereo camera module 195 of the drawing may be a structure that can be attached to or detached from the ceiling or the windshield of the vehicle 200.

A vehicle driving assistance device (100 in FIG. 3) having such a stereo camera module 195 obtains a stereo image of the front of the vehicle from the stereo camera module 195 and based on the stereo image, a disparity. ) Detect, perform object detection on the at least one stereo image based on the disparity information, and continuously track the movement of the object after object detection.

3A to 3B illustrate various examples of an internal block diagram of a vehicle driving assistance apparatus according to an embodiment of the present invention.

The vehicle driving assistance apparatus 100 of FIGS. 3A to 3B may process the stereo image received from the stereo camera 195 based on computer vision to generate vehicle related information. The vehicle related information may include vehicle control information for direct control of the vehicle, or vehicle driving assistance information for driving guide to the vehicle driver.

First, referring to FIG. 3A, the vehicle driving assistance apparatus 100 of FIG. 3A includes a communication unit 120, an interface unit 130, a memory 140, a processor 170, a power supply unit 190, and a stereo camera. 195. In addition, an audio input unit (not shown) and an audio output unit (not shown) may be provided.

The communication unit 120 may exchange data with the mobile terminal 600 or the server 500 in a wireless manner. In particular, the communication unit 120 may exchange data wirelessly with a mobile terminal of a vehicle driver. As a wireless data communication method, various data communication methods such as Bluetooth, WiFi Direct, WiFi, and APiX are possible.

The communication unit 120 may receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information from the mobile terminal 600 or the server 500. In the meantime, the vehicle driving assistance apparatus 100 may transmit the real-time traffic information grasped based on the stereo image to the mobile terminal 600 or the server 500.

On the other hand, when the user rides in the vehicle, the user's mobile terminal 600 and the vehicle driving assistance device 100 may perform pairing with each other automatically or by executing the user's application.

The interface unit 130 may receive vehicle-related data or transmit a signal processed or generated by the processor 170 to the outside. To this end, the interface unit 130 may perform data communication with the ECU 770, the AVN (Audio Video Navigation) device 400, the sensor unit 760, etc. in the vehicle by wired communication or wireless communication. have.

The interface unit 130 may receive map information related to vehicle driving through data communication with the AVN device 400.

The interface unit 130 may receive sensor information from the ECU 770 or the sensor unit 760.

Here, the sensor information includes vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, tire information, vehicle It may include at least one of lamp information, vehicle interior temperature information, vehicle interior humidity information.

Such sensor information may include heading sensors, yaw sensors, gyro sensors, position modules, vehicle forward / reverse sensors, wheel sensors, vehicle speed sensors, It may be obtained from a vehicle body tilt sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle interior temperature sensor, a vehicle interior humidity sensor, and the like. Meanwhile, the position module may include a GPS module for receiving GPS information.

Meanwhile, among the sensor information, the vehicle direction information, the vehicle position information, the vehicle angle information, the vehicle speed information, the vehicle tilt information, and the like related to the vehicle driving may be referred to as vehicle driving information.

The memory 140 may store various data for operations of the overall vehicle driving assistance apparatus 100, such as a program for processing or controlling the processor 170.

The audio output unit (not shown) converts an electrical signal from the processor 170 into an audio signal and outputs the audio signal. To this end, a speaker or the like may be provided. The audio output unit (not shown) may output sound corresponding to the operation of the input unit 110, that is, the button.

The audio input unit (not shown) may receive a user voice. To this end, a microphone may be provided. The received voice may be converted into an electrical signal and transmitted to the processor 170.

The processor 170 controls the overall operation of each unit in the vehicle driving assistance apparatus 100.

In particular, the processor 170 performs computer vision-based signal processing. Accordingly, the processor 170 obtains a stereo image of the front of the vehicle from the stereo camera 195, performs a disparity operation on the front of the vehicle based on the stereo image, and based on the calculated disparity information. , Object detection may be performed on at least one of the stereo images, and after the object detection, the movement of the object may be continuously tracked.

In particular, when detecting an object, the processor 170 may perform lane detection, vehicle detection, pedestrian detection, traffic sign detection, road surface detection, and the like. Can be.

The processor 170 may perform a distance calculation on the detected surrounding vehicle, a speed calculation of the detected surrounding vehicle, a speed difference calculation with the detected surrounding vehicle, and the like.

Meanwhile, the processor 170 may receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information through the communication unit 120.

On the other hand, the processor 170, the vehicle driving assistance apparatus 100 may grasp, in real time, the traffic situation information around the vehicle, which is determined based on the stereo image.

The processor 170 may receive map information or the like from the AVN device 400 through the interface unit 130.

Meanwhile, the processor 170 may receive sensor information from the ECU 770 or the sensor unit 760 through the interface unit 130. Here, the sensor information includes vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, tire information, vehicle It may include at least one of lamp information, vehicle interior temperature information, vehicle interior humidity information.

The power supply unit 190 may supply power required for the operation of each component under the control of the processor 170. In particular, the power supply unit 190 may receive power from a battery inside the vehicle.

The stereo camera 195 may include a plurality of cameras. Hereinafter, as described in FIG. 2 and the like, two cameras are provided.

The stereo camera 195 may be detachable from the ceiling or the windshield of the vehicle 200, and may include a first camera 195a having a first lens 193a and a second camera having a second lens 193b. 195b.

On the other hand, the stereo camera 195 has a first light shield 192a and a second light shield for shielding light incident on the first lens 193a and the second lens 193b, respectively. The part 192b may be provided.

Next, referring to FIG. 3B, the vehicle driving assistance apparatus 100 of FIG. 3B may further include an input unit 110 and a display 180 as compared to the vehicle driving assistance apparatus 100 of FIG. 3A. Hereinafter, only the description of the input unit 110 and the display 180 will be described.

The input unit 110 may include a plurality of buttons or a touch screen attached to the vehicle driving assistance apparatus 100, particularly, the stereo camera 195. Through a plurality of buttons or a touch screen, the vehicle driving assistance apparatus 100 may be turned on and operated. In addition, various input operations may be performed.

The display 180 may display an image related to the operation of the vehicle driving assistance apparatus. For displaying such an image, the display 180 may include a cluster or a head up display (HUD) on the front surface of the vehicle. On the other hand, when the display 180 is a HUD, it may include a projection module for projecting an image on the windshield of the vehicle 200.

4A-4B illustrate various examples of internal block diagrams of the processor of FIGS. 3A-3B, and FIGS. 5A-5B are diagrams referred to in describing the operation of the processor of FIGS. 4A-4B.

First, referring to FIG. 4A, FIG. 4A is an example of an internal block diagram of the processor 170. The processor 170 in the vehicle driving assistance apparatus 100 includes an image preprocessor 410 and a disparity calculator 420. ), An object detector 434, an object tracking unit 440, and an application unit 450 may be provided.

The image preprocessor 410 receives a stereo image from the stereo camera 195 and performs preprocessing.

In detail, the image preprocessing unit 410 may include noise reduction, rectification, calibration, color enhancement, and color space conversion for a stereo image. CSC), interpolation, camera gain control, and the like. Accordingly, a stereo image that is clearer than a stereo image captured by the stereo camera 195 may be obtained.

The disparity calculator 420 receives a stereo image signal-processed by the image preprocessor 410, performs stereo matching on the received stereo images, and performs stereo matching. Obtain a disparity map. That is, disparity information about the stereo image of the front of the vehicle may be obtained.

In this case, the stereo matching may be performed in units of pixels of stereo images or in units of predetermined blocks. The disparity map may refer to a map in which stereo parallax information of stereo images, that is, left and right images, is numerically represented.

The segmentation unit 432 may perform segmentation and clustering on at least one of the stereo images based on the disparity information from the disparity calculator 420.

In detail, the segmentation unit 432 may separate the background and the foreground from at least one of the stereo images based on the disparity information.

For example, an area in which the disparity information is less than or equal to a predetermined value in the disparity map may be calculated in the background, and the corresponding part may be excluded. Thereby, the foreground can be relatively separated.

As another example, an area in which the disparity information is greater than or equal to a predetermined value in the disparity map may be calculated in the foreground and a corresponding portion may be extracted. Thereby, the foreground can be separated.

As described above, by separating the foreground and the background based on the disparity information information extracted based on the stereo image, the signal processing speed, the signal processing amount, and the like can be shortened in the subsequent object detection.

Next, the object detector 434 may detect the object based on the image segment from the segmentation unit 432.

That is, the object detector 434 may detect an object with respect to at least one of the stereo images based on the disparity information information.

In detail, the object detector 434 may detect an object with respect to at least one of the stereo images. For example, an object can be detected from the foreground separated by image segments.

Next, an object verification unit 436 classifies and verifies the separated object.

To this end, the object verification unit 436 may include an identification method using a neural network, a support vector machine (SVM) method, a method of identifying by AdaBoost using a haar-like feature, or a histograms of oriented gradients (HOG). Techniques can be used.

The object checking unit 436 may check the objects by comparing the objects stored in the memory 140 with the detected objects.

For example, the object checking unit 436 may check surrounding vehicles, lanes, road surfaces, signs, dangerous areas, tunnels, and the like, which are positioned around the vehicle.

The object tracking unit 440 performs tracking on the identified object. For example, in order to sequentially identify the object in the obtained stereo images, calculate the motion or motion vector of the identified object, track the movement of the object, etc. based on the calculated motion or motion vector. Can be. Accordingly, it is possible to track surrounding vehicles, lanes, road surfaces, signs, dangerous areas, tunnels, and the like, which are located around the vehicle.

Next, the application unit 450 may calculate a risk of the vehicle 200 based on various objects located around the vehicle, for example, another vehicle, a lane, a road surface, a sign, and the like. In addition, it is possible to calculate the possibility of colliding with the vehicle ahead, whether the vehicle slips.

The application unit 450 may output, as vehicle driving assistance information, a message for informing the user of such information, based on the calculated risk, the possibility of colliding or the slip. Alternatively, a control signal for attitude control or driving control of the vehicle 200 may be generated as vehicle control information.

4B is another example of an internal block diagram of a processor.

Referring to the drawings, the processor 170 of FIG. 4B has the same internal configuration unit as the processor 170 of FIG. 4A, but the signal processing order is different. Only the differences are described below.

The object detector 434 may receive a stereo image and detect an object with respect to at least one of the stereo images. Unlike FIG. 4A, based on the disparity information, for the segmented image, the object may be detected directly from the stereo image, instead of detecting the object.

Next, the object verification unit 436 classifies the detected and separated objects based on the image segments from the segmentation unit 432 and the objects detected by the object detection unit 434. , Verify.

To this end, the object verification unit 436 may include an identification method using a neural network, a support vector machine (SVM) method, a method of identifying by AdaBoost using a haar-like feature, or a histograms of oriented gradients (HOG). Techniques can be used.

5A and 5B are views referred to for describing a method of operating the processor 170 of FIG. 4A based on stereo images obtained in the first and second frame sections, respectively.

First, referring to FIG. 5A, during the first frame period, the stereo camera 195 acquires a stereo image.

The disparity calculator 420 in the processor 170 receives the stereo images FR1a and FR1b signal-processed by the image preprocessor 410 and performs stereo matching on the received stereo images FR1a and FR1b. To obtain a disparity map 520.

The disparity map 520 is a leveling disparity between the stereo images FR1a and FR1b. The greater the disparity level is, the closer the vehicle is to the distance, and the smaller the disparity level is, the lower the disparity map is. We can calculate that distance of is far.

On the other hand, when displaying such a disparity map, the disparity map may be displayed such that the larger the disparity level, the higher the luminance, and the smaller the disparity level, the lower the luminance.

In the drawing, in the disparity map 520, the first to fourth lanes 528a, 528b, 528c, 528d and the like have corresponding disparity levels, respectively, the construction area 522, the first front vehicle 524. For example, each of the second front vehicles 526 has a corresponding disparity level.

The segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 may perform segment, object detection, and detection on at least one of the stereo images FR1a and FR1b based on the disparity map 520. Perform object verification.

In the figure, using the disparity map 520, it illustrates that object detection, and verification for the second stereo image FR1b is performed.

That is, in the image 530, the first to fourth lanes 538a, 538b, 538c, 538d, the construction area 532, the first front vehicle 534, and the second front vehicle 536 detect an object. And confirmation can be performed.

Next, referring to FIG. 5B, during the second frame period, the stereo camera 195 acquires a stereo image.

The disparity calculator 420 in the processor 170 receives the stereo images FR2a and FR2b signal-processed by the image preprocessor 410 and performs stereo matching on the received stereo images FR2a and FR2b. To obtain a disparity map 540.

In the drawing, in the disparity map 540, the first to fourth lanes 548a, 548b, 548c, 548d and the like have corresponding disparity levels, respectively, the construction area 542, the first front vehicle 544. For example, each of the second front vehicles 546 has a corresponding disparity level.

The segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 may perform segment, object detection, and detection on at least one of the stereo images FR2a and FR2b based on the disparity map 520. Perform object verification.

In the figure, using the disparity map 540, it illustrates that object detection, and verification is performed on the second stereo image FR2b.

That is, in the image 550, the first to fourth lanes 558a, 558b, 558c and 558d, the construction area 552, the first front vehicle 554, and the second front vehicle 556 detect the object. And confirmation can be performed.

Meanwhile, the object tracking unit 440 may perform tracking on the identified object by comparing FIGS. 5A and 5B.

In detail, the object tracking unit 440 may track the movement of the corresponding object based on the motion or the motion vector of each object identified in FIGS. 5A and 5B. Accordingly, tracking of a lane, a construction area, a first front vehicle, a second front vehicle, and the like located around the vehicle can be performed.

6A to 6B are views referred to for describing the operation of the vehicle driving assistance apparatus of FIG. 3.

First, FIG. 6A is a diagram illustrating a situation in front of a vehicle captured by the stereo camera 195 provided in a vehicle. In particular, the vehicle front situation is displayed in a bird eye view.

Referring to the drawings, from the left to the right, the first lane 642a, the second lane 644a, the third lane 646a, the fourth lane 648a is located, the first lane 642a and the second A construction area 610a is located between the lanes 644a, a first front vehicle 620a is located between the second lane 644a and the third lane 646a, and the third lane 646a and the fourth lane. It can be seen that the second front vehicle 630a is disposed between the lanes 648a.

Next, FIG. 6B illustrates displaying the vehicle front situation detected by the vehicle driving assistance apparatus together with various types of information. In particular, the image as shown in FIG. 6B may be displayed on the display 180 or the AVN device 400 provided in the vehicle driving assistance apparatus.

6B illustrates that information display is performed based on an image captured by the stereo camera 195, unlike FIG. 6A.

Referring to the drawings, from the left to the right, the first lane 642b, the second lane 644b, the third lane 646b, the fourth lane 648b, the first lane 642b and the second lane A construction area 610b is located between the lanes 644b, a first front vehicle 620b is located between the second lane 644b and the third lane 646b, and the third lane 646b and the fourth lane. It can be seen that the second front vehicle 630b is disposed between the lanes 648b.

The vehicle driving assistance apparatus 100 performs signal processing based on the stereo image captured by the stereo camera 195 to provide the construction area 610b, the first front vehicle 620b, and the second front vehicle 630b. You can check the object. In addition, the first lane 642b, the second lane 644b, the third lane 646b, and the fourth lane 648b may be identified.

On the other hand, in the drawing, to illustrate the object confirmation for the construction area 610b, the first front vehicle 620b, the second front vehicle 630b, it is illustrated that each is highlighted with a border.

On the other hand, the vehicle driving assistance apparatus 100 is based on the stereo image captured by the stereo camera 195, the distance to the construction area 610b, the first front vehicle 620b, the second front vehicle 630b. Information can be calculated.

In the drawing, the calculated first distance information 611b, second distance information 621b, and third distance corresponding to each of the construction area 610b, the first front vehicle 620b, and the second front vehicle 630b. It illustrates that information 631b is displayed.

Meanwhile, the vehicle driving assistance apparatus 100 may receive sensor information about the vehicle from the ECU 770 or the sensor unit 760. In particular, the vehicle speed information, the gear information, the yaw rate information (yaw rate) indicating the speed at which the rotation angle (concave angle) of the vehicle changes, and the angle information of the vehicle can be received, and such information can be displayed.

In the drawing, the vehicle speed information 672, the gear information 671, and the yaw rate information 673 are displayed on the vehicle front image upper portion 670, and the angle of the vehicle is displayed on the vehicle front image lower portion 680. While the information 682 is illustrated, various examples are possible. In addition, the width information 683 of the vehicle and the curvature information 681 of the road may be displayed together with the angle information 682 of the vehicle.

On the other hand, the vehicle driving assistance apparatus 100 may receive speed limit information, etc. for the road on which the vehicle is traveling, through the communication unit 120 or the interface unit 130. In the figure, it illustrates that the speed limit information 640b is displayed.

The vehicle driving assistance apparatus 100 may display various pieces of information illustrated in FIG. 6B through the display 180. Alternatively, the vehicle driving assistance device 100 may store various pieces of information without additional display. In addition, the information may be used for various applications.

FIG. 7 is an example of an internal block diagram of the electronic control apparatus in the vehicle of FIG. 1.

Referring to the drawings, the vehicle 200 may include an electronic control apparatus 700 for controlling the vehicle. The electronic control apparatus 700 can exchange data with the vehicle driving assistance apparatus 100 and the AVN apparatus 400 described above.

The electronic control apparatus 700 includes an input unit 710, a communication unit 720, a memory 740, a lamp driver 751, a steering driver 752, a brake driver 753, a power source driver 754, and a sunroof driver. 755, suspension driver 756, air conditioning driver 757, window driver 758, airbag driver 759, sensor unit 760, ECU 770, display unit 780, audio output unit 785. The power supply unit 790 may be provided.

The input unit 710 may include a plurality of buttons or a touch screen disposed in the vehicle 200. Through a plurality of buttons or touch screens, it is possible to perform various input operations.

The communication unit 720 may exchange data with the mobile terminal 600 or the server 500 in a wireless manner. In particular, the communication unit 720 may exchange data wirelessly with the mobile terminal of the vehicle driver. As a wireless data communication method, various data communication methods such as Bluetooth, WiFi Direct, WiFi, and APiX are possible.

The communication unit 720 may receive weather information, road traffic information, for example, TPEG (Transport Protocol Expert Group) information from the mobile terminal 600 or the server 500.

On the other hand, when the user is in the vehicle, the mobile terminal 600 and the electronic control apparatus 700 of the user can perform pairing with each other automatically or by executing the user's application.

The memory 740 may store various data for operating the entire electronic control apparatus 700, such as a program for processing or controlling the ECU 770.

The lamp driver 751 may control turn on / off of lamps disposed inside and outside the vehicle. In addition, it is possible to control the intensity, direction, etc. of the light of the lamp. For example, control of a direction indicator lamp, a brake lamp, and the like can be performed.

The steering driver 752 may perform electronic control of a steering apparatus in the vehicle 200. As a result, the traveling direction of the vehicle can be changed.

The brake driver 753 may perform electronic control of a brake apparatus in the vehicle 200. For example, the speed of the vehicle 200 may be reduced by controlling the operation of the brake disposed on the vehicle. As another example, by varying the operation of the brakes disposed on the left wheels and the right wheels, the traveling direction of the vehicle 200 may be adjusted to the left or the right.

The power source driver 754 may perform electronic control of the power source in the vehicle 200.

For example, when the fossil fuel-based engine is a power source, the power source driver 754 may perform electronic control of the engine. Thereby, the output torque of an engine, etc. can be controlled.

As another example, when the electric-based motor is a power source, the power source driver 754 may perform control on the motor. Thereby, the rotation speed, torque, etc. of a motor can be controlled.

The sunroof driver 755 may perform electronic control of a sunroof apparatus in the vehicle 200. For example, the opening or closing of the sunroof can be controlled.

The suspension driver 756 may perform electronic control of a suspension apparatus in the vehicle 200. For example, when the road surface is curved, the suspension device may be controlled to control the vibration of the vehicle 200 to be reduced.

The air conditioning driver 757 may perform electronic control of an air cinditioner in the vehicle 200. For example, when the temperature inside the vehicle is high, the air conditioner may operate to control the cool air to be supplied into the vehicle.

The window driver 758 may perform electronic control of a suspension apparatus in the vehicle 200. For example, the opening or closing of the left and right windows of the side of the vehicle can be controlled.

The airbag driver 759 may perform electronic control of an airbag apparatus in the vehicle 200. For example, in case of danger, the airbag can be controlled to burst.

The sensor unit 760 senses a signal related to traveling of the vehicle 100. To this end, the sensor unit 760 may include a heading sensor, a yaw sensor, a gyro sensor, a position module, a vehicle forward / reverse sensor, and a wheel sensor. , A vehicle speed sensor, a vehicle body tilt sensor, a battery sensor, a fuel sensor, a tire sensor, a steering sensor by steering wheel rotation, a vehicle internal temperature sensor, a vehicle internal humidity sensor, and the like.

Accordingly, the sensor unit 760 includes vehicle direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle acceleration information, vehicle tilt information, vehicle forward / reverse information, battery information, fuel information, A sensing signal may be acquired for tire information, vehicle lamp information, vehicle interior temperature information, vehicle interior humidity information, and the like.

On the other hand, the sensor unit 760 may include an accelerator pedal sensor, a pressure sensor, an engine speed sensor, an air flow sensor (AFS), an intake temperature sensor (ATS), a water temperature sensor (WTS), and a throttle. Position sensor (TPS), TDC sensor, crank angle sensor (CAS), etc. may be further provided.

The ECU 770 may control the overall operation of each unit in the electronic control apparatus 700.

By the input by the input unit 710, a specific operation may be performed, or a signal sensed by the sensor unit 760 may be received and transmitted to the vehicle driving assistance apparatus 100, and the map information may be transmitted from the AVN device 400. It can receive, and can control the operation of the driving unit (751, 752, 753, 754, 756).

In addition, the ECU 770 may receive weather information, road traffic condition information, for example, TPEG (Transport Protocol Expert Group) information from the communication unit 720.

The display unit 780 may display an image related to the operation of the vehicle driving assistance apparatus. For displaying such an image, the display unit 780 may include a cluster or a head up display (HUD) on the front surface of the vehicle. On the other hand, when the display unit 780 is a HUD, it may include a projection module for projecting an image on the windshield of the vehicle 200. Meanwhile, the display unit 780 may include a touch screen that can be input.

The audio output unit 785 converts the electrical signal from the ECU 770 into an audio signal and outputs the audio signal. To this end, a speaker or the like may be provided. The audio output unit 785 may output a sound corresponding to the operation of the input unit 710, that is, the button.

The power supply unit 790 may supply power required for the operation of each component under the control of the ECU 770. In particular, the power supply unit 790 may receive power from a battery (not shown) in the vehicle.

8 is a flowchart illustrating a method of operating a vehicle driving assistance apparatus according to an embodiment of the present invention.

Referring to the drawings, the processor 170 of the vehicle driving assistance apparatus 100 receives a stereo image from a stereo camera (S810). Then, the processor 170 of the vehicle driving assistance apparatus 100 calculates vehicle surrounding information based on the received stereo image (S820).

As described with reference to FIG. 2, the processor 170 of the vehicle driving assistance apparatus 100 receives the first image through the first camera 195a through the stereo camera 195 disposed inside the vehicle. Through the two cameras 195b, a second image is received.

Since there is a distance between the first camera 195a and the second camera 195b, disparity occurs in the first image and the second image.

The processor 170 of the vehicle driving assistance apparatus 100 calculates a disparity between the first image and the second image, and uses at least one of the first image and the second image by using the calculated disparity information. , Segment, object detection, and object verification.

In addition, the processor 170 of the vehicle driving assistance apparatus 100 performs tracking on the identified object, for example, a front vehicle, a lane, a structure, a road surface, a sign, a tunnel, and the like. In addition, the processor 170 of the vehicle driving assistance apparatus 100 may obtain distance information about the tunnel or the front vehicle located in front of the vehicle. In addition, the processor 170 of the vehicle driving assistance apparatus 100 may acquire speed information, relative speed information, and the like, for the front vehicle located in front of the vehicle.

Here, the vehicle surrounding information based on the stereo image may include lanes, structures, surrounding vehicles, tunnels, signs, and may include at least one of distance information with surrounding vehicles, speed information of surrounding vehicles, and the like.

Next, the processor 170 of the vehicle driving assistance apparatus 100 may calculate whether the vehicle enters a tunnel based on the calculated vehicle surrounding information (S830).

In particular, when the tunnel is detected in operation 820, the processor 170 of the vehicle driving assistance apparatus 100 may determine whether the vehicle enters the tunnel based on the distance between the vehicle 200 and the tunnel. Can be. For example, when the distance between the vehicle 200 and the tunnel is less than or equal to the predetermined distance, it may be calculated by entering the tunnel.

In addition to the stereo image, the processor 170 of the vehicle driving assistance apparatus 100 may further include vehicle driving information (eg, sensor information received from the ECU 770 or the sensor 760 of the vehicle through the interface unit 130). Vehicle speed information, vehicle location information, etc.) may be calculated.

The predetermined distance mentioned above can be changed according to the speed of the vehicle. For example, as the speed of the vehicle 200 is faster, the predetermined distance is preferably increased.

Next, when the vehicle enters the tunnel, the processor 170 of the vehicle driving assistance apparatus 100 may generate a tunnel mode control signal for operating the vehicle in the tunnel mode (S840).

The tunnel mode control signal may be a control signal for controlling at least one of the sunroof driver 755, the lamp driver 751, the window driver 758, and the air conditioning driver 757 in the vehicle.

The generated tunnel mode control signal Stu is output to an external, in particular, ECU 770 of the vehicle, and the ECU 770 includes a sunroof driver 755, a lamp driver 751, and a window driver ( 758 and the air conditioning driving unit 757 may be controlled.

For example, the sunroof may be closed, the lamp may be turned on, the window may be closed, or the air conditioner may be operated. As a result, when the tunnel enters, it is possible to perform the tunnel mode operation of the vehicle.

Next, the processor 170 of the vehicle driving assistance apparatus 100 may calculate whether the vehicle escapes the tunnel based on the calculated vehicle surrounding information (S850).

The processor 170 of the vehicle driving assistance apparatus 100 may calculate whether the vehicle escapes the tunnel based on the detected tunnel exit based on the stereo image.

For example, when the distance between the vehicle 200 and the tunnel is less than or equal to the predetermined distance, it may be calculated by entering the tunnel.

In addition to the stereo image, the processor 170 of the vehicle driving assistance apparatus 100 may further include vehicle driving information (eg, sensor information received from the ECU 770 or the sensor 760 of the vehicle through the interface unit 130). Vehicle exit information may be calculated based on vehicle speed information, vehicle position information, and the like.

In detail, the processor 170 of the vehicle driving assistance apparatus 100 may calculate whether the vehicle escapes the tunnel based on the tunnel distance information, the current vehicle speed information, and the like, which can be obtained from the map information.

The predetermined distance mentioned above can be changed according to the speed of the vehicle. For example, as the speed of the vehicle 200 is faster, the predetermined distance is preferably increased.

Next, when the vehicle escapes the tunnel, the processor 170 of the vehicle driving assistance apparatus 100 may generate a pre-tunnel mode control signal for operating the vehicle in the pre-tunnel mode (S860).

The pre-tunnel mode control signal may be a control signal for controlling at least one of the sunroof driver 755, the lamp driver 751, the window driver 758, and the air conditioning driver 757 in the vehicle.

The generated tunnel mode control signal Stu is output to an external, in particular, ECU 770 of the vehicle, and the ECU 770 includes a sunroof driver 755, a lamp driver 751, and a window driver ( 758 and the air conditioning driving unit 757 may be controlled.

For example, the sunroof can be opened again, the lamp can be turned off, the window opened again, or the air conditioner turned off.

9 illustrates an example of an internal block diagram of a processor for generating tunnel mode control signals.

Referring to the drawings, the processor 170 of the vehicle driving assistance apparatus 100 includes a tunnel detector 905, a lane detector 910, a distance detector 920, a speed detector 930, and a tunnel mode control signal generator ( 940 may be provided.

The processor 170 of the vehicle driving assistance apparatus 100 receives a stereo image Simg from the stereo camera 195, and receives map information Smp from the AVN device 400 through the interface unit 130. The vehicle driving information Scar may be received from the ECU 770 or the sensor unit 760.

Here, the vehicle driving information may include vehicle driving direction information, vehicle position information (GPS information), vehicle angle information, vehicle speed information, vehicle tilt information, side entry information, backward information, and the like. The vehicle driving information may be part of the sensor information.

The tunnel detector 905 may perform a tunnel sword based on the stereo image Simg received from the front stereo camera 195. In detail, the tunnel may be detected based on the disparity of the stereo image Simg.

Meanwhile, the tunnel detector 905 may perform tunnel detection based on the map information Smp received through the interface unit 130. Specifically, the location of the current vehicle may be mapped on the map, and based on the mapped location, the tunnel on the map may be detected.

The lane detector 910 may perform lane detection based on the stereo image Simg received from the front stereo camera 195. In detail, the lane toward the front may be detected based on the disparity of the stereo image Simg.

Similarly, the distance detector 920 and the speed detector 930 may detect the distance, speed, etc. of the front vehicle based on the stereo image Simg received from the front stereo camera 195.

The tunnel mode control signal generator 940 may generate the tunnel mode control signal (based on at least one of the detected tunnel information, the detected lane information, the distance information of the surrounding vehicle, the speed information, or the vehicle driving information Scar). Stu) can be created.

The generated tunnel mode control signal Stu is output to an external, in particular, ECU 770 of the vehicle, and the ECU 770 is a sunroof driver 755, a lamp driver 751, a window driver 758. The air conditioning driving unit 757 may control at least one.

For example, the sunroof may be closed, the lamp may be turned on, the window may be closed, or the air conditioner may be operated. As a result, when the tunnel enters, it is possible to perform the tunnel mode operation of the vehicle.

10A illustrates another example of an internal block diagram of the processor of FIG. 9.

Referring to the drawing, the processor 170 of the vehicle driving assistance apparatus 100 may detect image based vehicle surrounding information based on a stereo image received from the front stereo camera 195. Here, the image-based vehicle surrounding information may include at least one of distance information, surrounding vehicle distance information, speed information, lane information, road surface information, display panel information, tunnel information, sign information, and the like.

As described above, the disparity information may be calculated based on the stereo image received from the front stereo camera 195, and the segment, the object detection, and the object confirmation may be performed based on the calculated disparity information.

Based on this, the processor 170 of the vehicle driving assistance apparatus 100 includes a vehicle detection 1010, a distance detection 1011, a lane detection 1012, a road surface detection 1014, and a visual driving for the front of the vehicle. Visual odometry 1016 and tunnel detection 1013 may be performed. In addition, structure detection can also be performed.

Next, the processor 170 of the vehicle driving assistance apparatus 100 may receive sensor information from the ECU 770 or the sensor unit 760.

The processor 170 of the vehicle driving assistance apparatus 100 performs vehicle dead reckoning 1030 based on vehicle driving information from the ECU 770 or the sensor unit 760.

In addition, the processor 170 of the vehicle driving assistance apparatus 100 performs the vehicle ego motion tracking 1040 based on the vehicle dead reckoning. At this time, in addition to vehicle dead reckoning, it is also possible to perform vehicle egomotion tracking 1040 based on visual odometry.

The processor 170 of the vehicle driving assistance apparatus 100 may perform a curvature calculation 1050 of the driving road based on the vehicle egomotion tracking 1040.

Meanwhile, the processor 170 of the vehicle driving assistance apparatus 100 performs the tracking direction 1060 of the vehicle based on the curvature calculation 1050, the lane detection 1010, and the road surface detection 1014. do.

In addition, the processor 170 of the vehicle driving assistance apparatus 100 may collide with the vehicle based on the progress tracking of the vehicle 1060, the vehicle detection 1010, the distance detection 1011, and the road surface detection 1014. Risk calculation, such as collision risk, may be performed 1065.

On the other hand, the processor 170 determines (1080) whether the tunnel entry of the calculated risk exceeds the allowable value, and if applicable, the tunnel mode control signal (Stu) for controlling the airbag driving unit provided in the vehicle May be generated 1090. On the other hand, if it does not exceed, it may be controlled to perform the normal driving (1085).

In addition, the processor 170 of the vehicle driving assistance apparatus 100 performs a vehicle path prediction 1070 based on the risk calculation 1065 and the vehicle egomotion tracking 1040. That is, the processor 170 of the vehicle driving assistance apparatus 100 estimates the vehicle path based on the estimated vehicle movement 1040.

The processor 170 of the vehicle driving assistance apparatus 100 performs the sleep operation 1080 on the basis of the vehicle path prediction 1070, the risk calculation 1065, and the vehicle motion tracking 1040. In case of no sleep, normal driving 1085 is performed. In case of sleep, the slip prevention control signal generation 1090 is performed.

The generated slip prevention control signal Sns is transmitted to the ECU 770, and the ECU 770 includes at least one of a steering driver 752, a brake driver 753, a power source driver 754, and a suspension driver 756. To control one.

For example, when the vehicle 200 slips to the left side relative to the driving road, the steering driver 752 controls the steering apparatus so that the right steering is performed or brakes based on the slip prevention control signal Sns. The drive unit 753 can operate the left brake.

As another example, when the vehicle 200 slips to the right side compared to the driving road, the steering driver 752 controls the steering apparatus so that the left steering is performed or the brake driver based on the slip prevention control signal Sns. 753 can operate the right brake.

Accordingly, the vehicle driving assistance apparatus 100 may perform slip prevention control based on a stereo image, sensor information, and the like.

The processor 170 of the vehicle driving assistance apparatus 100 may perform a tunnel mode control signal Stu generation 1091 based on the tunnel detection 1013. The tunnel mode control signal Stu may be a control signal for controlling at least one of the sunroof driver 755, the lamp driver 751, the window driver 758, and the air conditioning driver 757 in the vehicle.

The generated tunnel mode control signal Stu is output to an external, in particular, ECU 770 of the vehicle, and the ECU 770 includes a sunroof driver 755, a lamp driver 751, and a window driver ( 758 and the air conditioning driving unit 757 may be controlled.

For example, the sunroof may be closed, the lamp may be turned on, the window may be closed, or the air conditioner may be operated.

The processor 170 of the vehicle driving assistance apparatus 100 may generate a tunnel pre-mode control signal (not shown) based on the tunnel escape (not shown). The tunnel tunnel mode control signal may be a control signal for controlling at least one of the sunroof driver 755, the lamp driver 751, the window driver 758, and the air conditioning driver 757 in the vehicle.

The generated tunnel mode control signal Stu is output to an external, in particular, ECU 770 of the vehicle, and the ECU 770 includes a sunroof driver 755, a lamp driver 751, and a window driver ( 758 and the air conditioning driving unit 757 may be controlled.

For example, the sunroof can be opened again, the lamp can be turned off, the window opened again, or the air conditioner turned off.

FIG. 10B illustrates another example of an internal block diagram of the processor of FIG. 9.

Referring to FIG. 10B, it is almost similar to FIG. 10A, except that map information from the AVN device 400 is received, and map matching 1020 is further performed. Only the differences are described below.

The processor 170 of the vehicle driving assistance apparatus 100 performs map matching based on the vehicle driving information among the sensor information from the ECU 770 or the sensor unit 760 and the map information from the AVN device 400. Perform 1020. In addition, the position of the current vehicle on the map may be varied based on vehicle driving information such as vehicle speed information.

The processor 170 of the vehicle driving assistance apparatus 100 maps the current vehicle onto a map on the basis of GPS information, which is vehicle position information, and map information from the AVN device 400. You can.

Next, the processor 170 of the vehicle driving assistance apparatus 100 may further, in addition to the vehicle egomotion tracking 1040, calculate the curvature of the driving road based on the map matching information at the time of calculating the curvature of the driving road 1050. Calculation 1050 may be performed.

11A-11D illustrate various examples of tunnel detection.

First, FIG. 11A illustrates a tunnel detection method based on a sign.

On the road, the stereo camera 195 in the vehicle 200 may acquire a stereo image, and the processor 170 may perform object detection and verification based on the stereo image.

At this time, the detection and confirmation of the sign 1110 disposed around the road among the objects is possible. In particular, when the sign 1110 is a sign indicating that there is a tunnel in front, the processor 170 may check this and detect the tunnel 1100 located in front of the sign 1110.

In the figure, the vehicle 200 and the tunnel 1100 is in a state where the distance is Dx, and the sign 1110 is illustrated to detect the tunnel.

Next, FIG. 11B illustrates a tunnel detection method based on a difference in brightness or intensity of light in front.

When the vehicle 200 runs during the day, the entrance 1150 of the tunnel 1100 becomes darker than other parts of the road.

The processor 170 may detect the tunnel entrance 1150 area by using the luminance difference or the light intensity difference. The tunnel 1100 may be detected based on the tunnel inlet 1150.

Specifically, the brightness or light intensity of the first front region A1 of the stereo image acquired from the stereo camera 195 in the vehicle 200 is a first level, and the brightness of the second front region A2. Alternatively, the light intensity may be a second level.

When the difference between the first level and the second level is greater than or equal to the reference value, the processor 170 may detect, as a tunnel, the second front area A2 having a lower luminance or light intensity level in a daytime situation.

Alternatively, when the difference between the first level and the second level is greater than or equal to the reference value, the processor 170 may increase the size of the second front area A as the vehicle 200 travels. The front area A2 can be detected by the tunnel.

On the other hand, when the vehicle 200 runs at night, the entrance 1150 of the tunnel 1100 becomes brighter than other parts of the road due to the illumination inside the tunnel.

Using this, the processor 170 may detect the second front area A2 having a lower luminance or light intensity level as a tunnel in a night situation.

Next, FIG. 11C illustrates a tunnel detection method based on the brightness of a lane detected based on a stereo image, or the surrounding vehicle.

The stereo camera 195 in the vehicle 200 may acquire a stereo image, and the processor 170 may perform object detection and verification based on the stereo image. Accordingly, the front lane and the front vehicle can be detected and confirmed.

On the other hand, when there is an area in which the brightness or the light intensity of the front lane or the front vehicle varies, the stereo camera 195 calculates the area as the tunnel entrance 1150 and based on the tunnel entrance 1150, the tunnel ( 1100 may be detected.

As shown in the drawing, the luminance level of the first front vehicle 1120a located in the front first area A1 is a third level on the stereo image and is the second front vehicle (located in the front second area A2 ( The brightness level of 1120b may be a fourth level on the stereo image.

The processor 170 may calculate that the tunnel inlet 1150 is located in the front second area A2 based on the level difference, and detect the tunnel 1100 based on the tunnel inlet 1150. have.

Alternatively, when the second front vehicle 1120b is located in the front first area A1 and when the second front vehicle 1120b is located in the front second area A2, based on the stereo image obtained sequentially. Compare the luminance level difference between the two, and if the level difference is greater than or equal to the reference value, calculate that the tunnel inlet 1150 is located in the front second area A2, and based on the tunnel inlet 1150, the tunnel 1100 is determined. Can be detected.

Next, FIG. 11D illustrates a tunnel detection method based on the shape of the tunnel entrance.

The stereo camera 195 in the vehicle 200 may acquire a stereo image, and the processor 170 may perform object detection and verification based on the stereo image. Accordingly, the front lane and the front vehicle can be detected and confirmed.

In particular, the processor 170 may perform object detection on the tunnel entrance 1150. In order to identify the object, shape matching with respect to the tunnel entrance 1150 may be performed using a database stored in a server inside or outside the vehicle.

In the drawing, the tunnel inlet 1150 shape detected based on the stereo image is compared with the first shape 1150a, the second shape 1150b, and the third shape 1150c stored in the database. As a result of this comparison, the processor 170 may identify the tunnel inlet 1150 and may detect and confirm the tunnel 1100.

On the other hand, in tunnel detection, it is also possible to detect the tunnel by combining the methods of Figs. 11A to 11D.

12A to 12C illustrate that a tunnel mode is performed according to tunnel entry.

FIG. 12A illustrates that at a first distance da, the vehicle 200 detects the tunnel 1100 on a stereo image basis. In this case, since the first distance da is too early to operate in the tunnel mode, the processor 170 may determine that the tunnel mode is before entering the tunnel and may not generate the tunnel mode control signal.

Next, FIG. 12B illustrates that the vehicle continues to travel so that the distance between the vehicle 200 and the tunnel 1100 is the second distance Db.

Accordingly, the processor 170 may generate a tunnel mode control signal Stu. In the figure, for convenience of description, it is illustrated that a message 1251 indicating the start of the tunnel mode is displayed. The message 1251 may be output through the display 180 or the audio output unit (not shown) in the vehicle.

12B illustrates that the tunnel mode of the vehicle 200 is applied based on the tunnel mode control signal.

FIG. 12B (a) shows a tunnel-previous mode in which the front ramps 1260L and 1260R of the vehicle 200 are turned off, the left and right windows 1270La and 1270Ra are partially opened, and the sunroof 1280a is opened. Illustrates some opening.

Thereafter, upon entering the tunnel, the vehicle 200 enters the tunnel mode based on the tunnel mode control signal Stu.

12B (b) shows a tunnel mode in which the front ramps 1260L and 1260R of the vehicle 200 are turned on, the left and right windows 1270Lb and 1270Rb are all closed, and the sunroof 1280b is also closed. It illustrates that the internal air conditioner operation 1290b is performed. Accordingly, when entering the tunnel, the tunnel mode is automatically performed.

13A to 13C illustrate that the pre-tunnel mode according to the tunnel escape is performed.

FIG. 13A illustrates that the vehicle 200 travels in a state where the vehicle 200 enters the tunnel 1100, and the distance to the tunnel exit 1160 is Dm.

The processor 170 may maintain the tunnel mode and not generate the pre-tunnel control signal because it is early to terminate the tunnel mode and switch to the pre-tunnel mode despite the detected tunnel exit 1160. have.

Next, FIG. 13B illustrates that the vehicle continues to travel so that the distance between the vehicle 200 and the tunnel 1100 is Db.

Accordingly, the processor 170 may generate a pre-tunnel mode control signal. In the figure, for convenience of description, a message 1351 indicating the start of the pre-tunnel mode is displayed. The message 1351 may be output through the display 180 or the audio output unit (not shown) in the vehicle.

FIG. 13B illustrates that the pre-tunnel mode of the vehicle 200 is applied based on the pre-tunnel mode control signal.

FIG. 13B (a) shows a tunnel mode in which the front ramps 1260L and 1260R of the vehicle 200 are turned on, the left and right windows 1270Lb and 1270Rb are all closed, and the sunroof 1280b is also closed. It illustrates that the internal air conditioner operation 1290b has been performed.

Subsequently, upon exiting the tunnel, the vehicle 200 enters the pre-tunnel mode based on the pre-tunnel mode control signal.

FIG. 13B (b) shows a tunnel pre-mode, in which the front ramps 1260L and 1260R of the vehicle 200 are turned off, the left and right windows 1270La and 1270Ra are partially opened, and the sunroof 1280a is opened. Illustrates some opening. Accordingly, upon exiting the tunnel, the tunnel pre-transmission mode is automatically performed.

14 is a diagram illustrating an exterior of a vehicle including a mono camera according to another embodiment of the present invention.

Referring to the drawings, the vehicle 200 according to another embodiment of the present invention, the wheels (150FR, 135FL, 135RL, ...) rotating by the power source, the handle 150 for adjusting the traveling direction of the vehicle 200 And a mono camera 193 and a radar 194 provided in the vehicle 200.

The mono image through the mono camera 193 and the distance information through the radar 194 may be signal processed in the vehicle driving assistance apparatus 2100 of FIG. 15A.

15A illustrates an example of an internal block diagram of a vehicle driving assistance apparatus according to another embodiment of the present invention.

Referring to the drawings, the internal block of the vehicle driving assistance device 2100 of FIG. 15A is similar to the internal block of the vehicle driving assistance device 100 of FIG. 3, but based on a stereo image from the stereo camera 195. The difference is that the signal processing is performed based on the mono image through the mono camera 193 and the distance information through the radar 194, rather than the processing. That is, it is possible to generate a control signal whether the vehicle enters the tunnel. In the following, only the differences are described.

The mono camera 193 may be detachable from the ceiling or the windshield of the vehicle 200 and may include a single camera having a lens.

The radar 194 may be detachable from the ceiling or the windshield of the vehicle 200, transmit radio waves of a predetermined frequency in front of the vehicle, and receive radio waves reflected from an object in front of the vehicle.

The processor 2170 may calculate distance information based on the difference between the transmission wave and the reception wave in the radar 194. In addition, the object may be separated, detected, and recognized by matching the mono image and distance information through the mono camera 193.

Next, FIG. 15B illustrates an example of an internal block diagram of a vehicle driving assistance apparatus according to another embodiment of the present invention.

Referring to the drawings, the inner block of the vehicle driving assistance device 2150 of FIG. 15B is similar to the inner block of the vehicle driving assistance device 2100 of FIG. 15A, but not the mono camera 193 and the radar 194. The difference lies in having a rider (Lidar) 2101 that performs scanning on an external object.

In describing the difference, the rider 2101 may acquire a scan image of a forward situation by a laser scanning method, and the processor 2270 performs signal processing based on the scan image received from the rider. . That is, it is possible to generate a control signal whether the vehicle enters the tunnel.

16A-16B illustrate various examples of internal block diagrams of the processor of FIG. 15A, and FIGS. 17A-17B are diagrams for reference to an operation description of the processor of FIG. 16A.

The processor internal block of FIGS. 16A to 16B is similar to the processor of FIGS. 4A to 4B except that the processor internal block does not include a disparity calculator.

Instead, there is a difference in receiving the distance information Sd corresponding to the disparity information from the external radar 194. The distance information Sd is input to the segmentation unit 432 and may be used when segmenting an image.

17A to 17B are views referred to for describing a method of operating the processor 170 of FIG. 16A, based on a mono image acquired in each of the first and second frame periods.

Referring to FIGS. 17A through 17B, the method is similar to FIGS. 5A through 5B except that since a mono image and radar based distance information are used, a disparity map does not need to be generated.

First, referring to FIG. 17A, during a first frame period, the mono camera 193 obtains a mono image FR1, and the radar obtains distance information Sd1.

Accordingly, the segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 in the processor 2170 may be configured to include the first segment in the mono image FR1 based on the radar based distance information Sd1. The lanes to the fourth lanes 538a, 538b, 538c, 538d, the construction area 532, the first front vehicle 534, and the second front vehicle 536 can be detected and confirmed.

Next, referring to FIG. 17B, during the second frame period, the mono camera 193 obtains a mono image FR2 and the radar obtains distance information Sd2.

Accordingly, the segmentation unit 432, the object detection unit 434, and the object confirmation unit 436 in the processor 2170 may be configured to include the first segment in the mono image FR2 based on the radar based distance information Sd2. The lanes to the fourth lanes 558a, 558b, 558c, and 558d, the construction area 552, the first front vehicle 554, and the second front vehicle 556 can be detected and confirmed.

The object tracking unit 440 may perform tracking on the identified object by comparing FIGS. 17A and 17B.

In detail, the object tracking unit 440 may track the movement of the corresponding object based on the motion or the motion vector of each object identified in FIGS. 17A and 17B. Accordingly, tracking of a lane, a construction area, a first front vehicle, a second front vehicle, and the like located around the vehicle can be performed.

On the other hand, the vehicle driving assistance device 2100 including the mono camera 193 and the radar 194 described with reference to FIGS. 14 to 17B is similar to the mono image and the distance information as described with reference to FIGS. 8 to 13C. Based on the calculation, whether the vehicle enters the tunnel, and generates a tunnel mode control signal for controlling at least one of the sunroof driver, the lamp driver, the window driver, and the air conditioning driver in the vehicle based on the calculated whether the vehicle enters the tunnel. can do.

Specifically, the processor 170 of the vehicle driving assistance device 2100 calculates whether the vehicle enters a tunnel based on the mono image received from the mono camera 193 and the distance information detected from the radar 194. The tunnel mode control signal for controlling at least one of the sunroof driver, the lamp driver, the window driver, and the air conditioning driver in the vehicle may be generated based on the calculated whether the vehicle enters the tunnel.

Meanwhile, the vehicle driving assistance device 2150 having the rider 2101 described with reference to FIG. 15B calculates whether or not the vehicle enters a tunnel based on the scanned image, and calculates the operation, similar to those described with reference to FIGS. 8 through 13C. The tunnel mode control signal for controlling at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioning driver in the vehicle may be generated based on whether the vehicle enters the tunnel.

In detail, the processor 2270 of the vehicle driving assistance apparatus 2150 calculates whether the vehicle enters a tunnel based on the scanned image received from the rider 2101, and based on the calculated whether the vehicle enters the tunnel. A tunnel mode control signal for controlling at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioning driver in the vehicle may be generated.

The vehicle driving assistance apparatus and the vehicle having the same according to an embodiment of the present invention are not limited to the configuration and method of the embodiments described above, but the embodiments may be modified in various ways so that various modifications can be made. All or part of the examples may be optionally combined.

On the other hand, the vehicle driving assistance apparatus or the method of operating the vehicle of the present invention can be implemented as code that the processor can read on a recording medium that can be read by the processor provided in the vehicle driving assistance apparatus or the vehicle. The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet. . The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

The present invention can be applied to a vehicle driving assistance device and a vehicle having the same, and in particular, based on the photographed image, the vehicle driving assistance device and the vehicle having the same can be easily performed when entering a tunnel. Do.

Claims (16)

Stereo cameras; And a processor configured to calculate whether the vehicle enters a tunnel based on the stereo image received from the stereo camera, and generate a tunnel mode control signal for operating the vehicle in the tunnel mode when the calculated vehicle enters the tunnel. , The tunnel mode control signal, And a control signal for controlling at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioner driver in the vehicle. The method of claim 1, The processor, Compute vehicle surrounding information including at least one of a detected lane, a tunnel, a surrounding vehicle, and a sign based on the stereo image, and calculate whether to enter the tunnel based on the vehicle surrounding information, and calculate the tunnel of the vehicle. And generating the tunnel mode control signal for operating the vehicle in the tunnel mode when the vehicle enters the vehicle. The method of claim 1, An interface unit for exchanging data with at least one in-vehicle device; The interface unit receives map information, The processor, And the tunnel in front of the vehicle is detected based on the map information and the stereo image. The method of claim 1, The processor, A pre-tunnel mode control signal for calculating vehicle surrounding information based on the stereo image, calculating whether to exit a tunnel based on the vehicle surrounding information, and operating the vehicle in a pre-tunnel mode when the tunnel escapes It includes; Processor for generating; The tunnel pre-mode control signal, And a control signal for controlling at least one of the sunroof driver, the lamp driver, the window driver, and the air conditioning driver in the vehicle. The method of claim 1, The processor, A tunnel detector detecting a tunnel in front of the vehicle; A lane detecting unit detecting a lane in front of the vehicle; A distance detector for detecting a distance of the surrounding vehicle with respect to the front of the vehicle; A speed detector for detecting a speed of the surrounding vehicle; And And a tunnel mode control signal generator configured to generate the tunnel mode control signal based on the detected tunnel, the detected lane, the detected distance, and the detected speed. . The method of claim 1, The processor, Detecting the tunnel on the basis of detecting and confirming a sign in front of the vehicle; Detecting the tunnel based on a region having different luminance in the stereo image; Detecting the tunnel based on the lane detected based on the stereo image or the luminance of the surrounding vehicle; And the tunnel is detected based on a shape of an object detected based on the stereo image. The method of claim 1, The processor, A disparity calculator configured to perform a disparity operation of the stereo image; An object detector configured to detect an object of at least one of the stereo images based on the disparity information of the stereo image; And an object tracking unit that performs tracking on the detected object. The method of claim 7, wherein The processor, A segmentation unit segmenting an object in the stereo image based on disparity information of the stereo image; And And an object checking unit to classify the detected object. The object detector, And based on the segmented object, perform object detection on at least one of the stereo images. The method of claim 1, An interface unit for exchanging data with at least one in-vehicle device; The interface unit receives sensor information, The processor, Based on the stereo image, image-based vehicle surrounding information including at least one of a lane, a tunnel, a surrounding vehicle, a signage, distance information with the surrounding vehicle, and speed information of the surrounding vehicle is detected; Based on vehicle driving information among the sensor information, vehicle movement estimation is performed, And tracking the direction of travel of the vehicle based on the image-based vehicle surrounding information and the vehicle motion estimation. The method of claim 9, The interface unit further receives map information, The processor, Map matching is performed based on the map information and the vehicle driving information; And tracking the driving direction of the vehicle based on the map information, the image based vehicle surrounding information, and the vehicle motion estimation. Mono camera; Radar; Based on the mono image received from the mono camera and distance information from the radar, the vehicle calculates whether the vehicle enters the tunnel, and when the calculated vehicle enters the tunnel, a tunnel mode control signal for operating the vehicle in the tunnel mode It includes; Processor, The tunnel mode control signal, And a control signal for controlling at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioner driver in the vehicle. A rider performing scanning on an external object; And a processor configured to calculate whether the vehicle enters a tunnel based on the scan image received from the rider and to generate a tunnel mode control signal for operating the vehicle in the tunnel mode when the calculated vehicle enters the tunnel. The tunnel mode control signal, And a control signal for controlling at least one of a sunroof driver, a lamp driver, a window driver, and an air conditioner driver in the vehicle. A sensor unit for sensing a vehicle state; A sunroof driver for driving a sunroof; A lamp driver for driving a lamp; A window driver for driving a window; An air conditioning driving unit for driving the air conditioning apparatus; A controller for controlling the sunroof driver, the lamp driver, the window driver, and the air conditioning driver; And A processor for calculating whether a vehicle enters a tunnel based on a stereo camera and a stereo image received from the stereo camera, and generating a tunnel mode control signal for operating the vehicle in a tunnel mode when the calculated vehicle enters a tunnel. It includes a vehicle driving assistance device having a; The tunnel mode control signal, And a control signal for controlling at least one of the sunroof driver, the lamp driver, the window driver, and the air conditioning driver. The method of claim 13, The vehicle driving assistance device, An interface unit for exchanging data with at least one in-vehicle device; The interface unit receives map information, The processor, And detecting the tunnel in front of the vehicle based on the map information and the stereo image. The method of claim 13, The processor, A pre-tunnel mode control signal for calculating vehicle surrounding information based on the stereo image, calculating whether to exit a tunnel based on the vehicle surrounding information, and operating the vehicle in a pre-tunnel mode when the tunnel escapes It includes; Processor for generating; The tunnel pre-mode control signal, And a control signal for controlling at least one of the sunroof driver, the lamp driver, the window driver, and the air conditioning driver in the vehicle. The method of claim 13, The processor, Detecting the tunnel on the basis of detecting and confirming a sign in front of the vehicle; Detecting the tunnel based on a region having different luminance in the stereo image; Detecting the tunnel based on the lane detected based on the stereo image or the luminance of the surrounding vehicle; And detecting the tunnel based on a shape of an object detected based on the stereo image.

Images