KR20170135522A - Control device for a vehhicle and control metohd thereof - Google Patents

Abstract

The present invention relates to a control device for a vehicle and a control method thereof, wherein the control method includes: sensing a drivers eye position; selecting one distortion correction function corresponding to the sensed eye position from a plurality of distortion correction functions stored in a memory; and correcting an image being output toward a windshield of a vehicle using the selected distortion correction function.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a control apparatus for a vehicle,

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a vehicle control apparatus provided in a vehicle and a control method thereof.

A vehicle means a means of transporting people or goods by using kinetic energy. A representative example of a vehicle is an automobile.

For safety and convenience of a user who uses the vehicle, various sensors and devices are provided in the vehicle, and the functions of the vehicle are diversified.

The function of the vehicle can be divided into a convenience function for the convenience of the driver and a safety function for the safety of the driver and / or the pedestrian.

First, the convenience function has the motive for development related to driver convenience, such as providing infotainment (information + entertainment) function to the vehicle, supporting partial autonomous driving function, assisting driver's vision such as night vision or blind spot . For example, an active cruise control (ACC), a smart parking assist system (SPAS), a night vision (NV), a head up display (HUD) (AVM), adaptive headlight system (AHS), and so on.

The safety function is to secure the safety of the driver and / or pedestrians. The lane departure warning system (LDWS), the lane keeping assist system (LKAS), the autonomous emergency braking, and AEB) functions.

In order to support and enhance the function of such a vehicle, a head up display (HUD) is being developed.

 A head up display (HUD) projects a virtual image onto the windshield of the vehicle.

Depending on the devices installed in the vehicle, the contents of this virtual image may vary and may also vary depending on the information the driver needs (e.g., cruise control, active cruise control, navigation, cruising speed, check control messages, etc.).

Providing information necessary for driving within the visible area of the driver, the driver can concentrate on the road traffic situation. And it reduces the need for the driver to alternate between the instrument cluster and the road. Accordingly, the head-up display contributes to driver's fatigue reduction and driving safety.

The head-up display (HUD) projects light output from a light source such as an LED array to a thin film transistor (TFT) that generates an image, so that an image generated in a thin film transistor (TFT) So as to appear floating in a space on the front road surface. Therefore, the driver can check necessary traveling information while looking ahead.

On the other hand, when the head-up display (HUD) is installed in the vehicle, the area in which the image is displayed is fixed. However, since the driver's view (driver-view) varies depending on the driver, there is a problem that distortion occurs in the image that the driver sees, even if there is no distortion in the output image.

The present invention is directed to solving the above-mentioned problems and other problems.

Another object of the present invention is to provide a vehicle control apparatus and a control method thereof that can solve the problem that different distortions occur in an image projected on a windshield depending on the driver's eye position.

An embodiment of the present invention for realizing the above-described problems relates to a control method of a vehicle control apparatus. The control method includes: sensing an eye position of a driver; Selecting one distortion correction function corresponding to the sensed eye position from among a plurality of distortion correction functions stored in the memory; And correcting an image output toward the windshield of the vehicle using the selected distortion correction function.

In one embodiment, as the sensed eye position changes, the selected distortion correction function may be changed from one to another.

In one embodiment, the control method further comprises storing the plurality of distortion correction functions in the memory, wherein the step of storing the plurality of distortion correction functions in the memory comprises: Outputting an image including cells; Measuring a distortion of the grid cells using a camera arranged to face the image at a driver's seat of the vehicle; Generating a distortion correction function corresponding to a photographing position of the camera based on the distortion of the measured grid cells; And storing a plurality of distortion correction functions generated at different shooting positions in the memory.

In one embodiment, measuring the distortion of the grid cells comprises: extracting at least one graphic form formed by at least three adjacent grid cells of the grid cells; And a step of measuring a distortion of the extracted figure. In the step of generating a distortion correction function corresponding to a photographing position of the camera, a distortion correction function for normalizing the distortion of the extracted figure is generated, A different distortion correction function may be generated depending on the photographing position of the camera.

In one embodiment, the control method further includes changing at least one of a position and a size of an area in which the corrected image is output based on the sensed eye position when the sensed eye position is out of a predetermined range As shown in FIG.

In one embodiment, the control method includes: searching for an area where a probability that the driver's eye position is out of a predetermined range is greater than a reference, based on GPS information; And changing at least one of a position and a size of an area where the corrected image is output when the vehicle enters the searched area.

In one embodiment, the control method further comprises: if at least one of the entire region of the corrected image is edited when the sensed eye position is out of a predetermined range, And outputting the windshield to the windshield.

In one embodiment, at least one of the size and position of the at least one region may vary according to the sensed eye position.

In a temporary example, the method may further include outputting the pre-correction image or the post-correction image selectively toward the windshield of the vehicle based on the user input.

In one embodiment, the control method further comprises: receiving the plurality of distortion correction functions from a server; And updating the received plurality of distortion correction functions to the memory.

Meanwhile, in an embodiment according to the present invention, the vehicle control apparatus includes a control section, and the control section can perform at least one of the control methods described above using at least one of the display and the memory disposed in the vehicle.

Effects of the vehicle control apparatus and the control method thereof according to the present invention will be described as follows.

Since the HUD image is corrected in real time by a plurality of pre-stored distortion correction functions, the augmented reality matching the real world can be provided by the HUD.

Since a plurality of distortion correction functions are stored in a lookup table and the HUD image is corrected in real time, it has a small amount of calculation and excellent correction ability against distortion occurring nonlinearly according to the eye position.

If at least a part of the HUD image is not included in the driver's view due to the eye position, the driver can continuously receive information through the HUD since the position of editing or outputting the HUD image is changed.

 Further scope of applicability of the present invention will become apparent from the following detailed description. It should be understood, however, that the detailed description and specific examples, such as the preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the invention will become apparent to those skilled in the art.

1 is a block diagram for explaining a control apparatus for a vehicle related to the present invention;
2 is a conceptual diagram for explaining a head-up display disposed in a vehicle
3 is a flowchart for explaining a control method performed in the vehicle control apparatus of FIG.
4 is a flowchart for describing in more detail a method for storing distortion correction functions in memory.
5 is a conceptual diagram for explaining a method of generating distortion correction functions using a camera installed in a windshield direction in a driver's seat
6 is an exemplary diagram for explaining a method of generating a distortion correction function using an image photographed by a camera
7 is a flowchart for explaining a method of adjusting a region in which an image is output according to a driving environment differently
Figs. 8A and 8B are illustrations for explaining the method of Fig. 7
9 is a flowchart for explaining a method of outputting different images according to the positions of eyes
10A and 10B are illustrations for explaining the method of FIG. 9

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals are used to designate identical or similar elements, and redundant description thereof will be omitted. The suffix "module" and " part "for the components used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role. In the following description of the embodiments of the present invention, a detailed description of related arts will be omitted when it is determined that the gist of the embodiments disclosed herein may be blurred. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. , ≪ / RTI > equivalents, and alternatives.

Terms including ordinals, such as first, second, etc., may be used to describe various elements, but the elements are not limited to these terms. The terms are used only for the purpose of distinguishing one component from another.

The singular expressions include plural expressions unless the context clearly dictates otherwise.

In the present application, the terms "comprises", "having", and the like are used to specify that a feature, a number, a step, an operation, an element, a component, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

The vehicle control apparatus described herein may be an apparatus for electronically controlling at least one component provided in a vehicle, for example, an electronic control unit (ECU).

The vehicle control device may be a mobile phone, a smart phone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a slate PC, mobile phones such as tablet PCs, ultrabooks and wearable devices as well as fixed terminals such as digital TVs, desktop computers, digital signage, and the like.

1 is a block diagram for explaining a vehicle control apparatus related to the present invention.

The vehicle control apparatus 100 includes a wireless communication unit 110, an input unit 120, a sensing unit 140, an output unit 150, an interface unit 160, a memory 170, a control unit 180, 190), and the like. The components shown in Fig. 1 are not essential for implementing the vehicle control device, and thus the vehicle control device described in this specification can have more or fewer components than those listed above.

More specifically, the wireless communication unit 110 among the above-described components can communicate with the vehicle control device 100 and the wireless communication system, between the vehicle control device 100 and another vehicle control device, or between the vehicle control device 100 and the outside And one or more modules that enable wireless communication between the servers. In addition, the wireless communication unit 110 may include one or more modules for connecting the onboard controller 100 to one or more networks.

The wireless communication unit 110 may include at least one of a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short distance communication module 114, and a location information module 115 .

The input unit 120 includes a camera 121 or an image input unit for inputting a video signal, a microphone 122 for inputting an audio signal, an audio input unit, a user input unit 123 for receiving information from a user A touch key, a mechanical key, and the like). The voice data or image data collected by the input unit 120 may be analyzed and processed by a user's control command.

The sensing unit 140 may include at least one sensor for sensing at least one of the information in the on-vehicle control device 100, the surrounding information on the on-vehicle control device 100, and the user information. For example, the sensing unit 140 may include a proximity sensor 141, an illumination sensor 142, a touch sensor, an acceleration sensor, a magnetic sensor, A G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor, a finger scan sensor, an ultrasonic sensor, A microphone 226, a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation detection sensor, A thermal sensor, a gas sensor, etc.), a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the vehicle control apparatus 100 disclosed in this specification can combine and utilize the information sensed by at least two of the sensors.

The output unit 150 includes at least one of a display unit 151, an acoustic output unit 152, a haptic tip module 153, and a light output unit 154 to generate an output related to visual, auditory, can do. The display unit 151 may have a mutual layer structure with the touch sensor or may be integrally formed to realize a touch screen. Such a touch screen may function as a user input unit 123 for providing an input interface between the vehicle control apparatus 100 and the user and may provide an output interface between the vehicle control apparatus 100 and the user.

The interface unit 160 serves as a path to various kinds of external devices connected to the vehicle control device 100. The interface unit 160 is connected to a device having a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, And may include at least one of a port, an audio I / O port, a video I / O port, and an earphone port. In the vehicle control apparatus 100, corresponding to the connection of an external device to the interface unit 160, it is possible to perform appropriate control relating to the connected external device.

In addition, the memory 170 stores data supporting various functions of the on-vehicle control device 100. The memory 170 may store a plurality of application programs (application programs or applications) driven by the onboard control device 100, data for operation of the onboard control device 100, and commands. At least some of these applications may be downloaded from an external server via wireless communication. At least a part of these application programs may exist on the on-vehicle control apparatus 100 from the time of shipment for the basic functions (e.g., call incoming, outgoing, message reception, and origination functions) of the on- have. On the other hand, the application program is stored in the memory 170, installed on the on-vehicle control device 100, and can be driven by the control unit 180 to perform the operation (or function) of the on-vehicle control device.

In addition to the operations related to the application program, the control unit 180 typically controls the overall operation of the on-vehicle control device 100. [ The control unit 180 may process or process signals, data, information, and the like input or output through the above-mentioned components, or may drive an application program stored in the memory 170 to provide or process appropriate information or functions to the user.

In addition, the controller 180 may control at least some of the components illustrated in FIG. 1A in order to drive an application program stored in the memory 170. FIG. Furthermore, the control unit 180 may operate at least two or more of the components included in the vehicular controller 100 in combination with each other for driving the application program.

The power supply unit 190 receives external power and internal power under the control of the controller 180 and supplies power to the respective components included in the vehicular controller 100. The power supply unit 190 includes a battery, which may be an internal battery or a replaceable battery.

At least some of the respective components may operate in cooperation with one another to implement an operation, control, or control method of a vehicle control apparatus according to various embodiments described below. Further, the operation, control, or control method of the on-vehicle control device may be implemented on the on-vehicle control device by driving at least one application program stored in the memory 170. [

Hereinafter, the display unit 151 enumerated above will be described in detail before explaining various embodiments implemented through the vehicle control apparatus 100 described above.

The display unit 151 displays (outputs) information to be processed by the onboard control device 100. For example, the display unit 151 may display execution screen information of an application program driven by the on-vehicle control device 100, or UI (User Interface) and GUI (Graphic User Interface) information based on the execution screen information have.

In another example, the display unit 151 can display the vehicle-related information. Here, the vehicle-related information may include vehicle control information for direct control to the vehicle, or vehicle driving assistance information for the driver to the driver. Further, the vehicle-related information may include vehicle state information indicating the current state of the vehicle or vehicle driving information related to the driving of the vehicle.

The display unit 151 may be a liquid crystal display (LCD), a thin film transistor-liquid crystal display (TFT LCD), an organic light-emitting diode (OLED), a flexible display display, a 3D display, and an electronic ink display.

In addition, the display unit 151 may exist in two or more depending on the implementation of the onboard control device 100. In this case, the on-vehicle control device 100 may have a plurality of display portions that are spaced apart from one another, disposed integrally with each other, or disposed on different surfaces.

The display unit 151 may include a touch sensor that senses a touch with respect to the display unit 151 so that a control command can be received by a touch method.

Specifically, the display unit 151 includes a display and a touch sensor, and the display and the touch sensor can operate organically under the control of the controller 180. For example, when a touch is made to the display unit 151, the touch sensor senses the touch, and the controller 180 generates a control command corresponding to the touch based on the touch. The controller 180 can sense the touch applied to the touch sensor and perform control corresponding to the sensed touch even when the display is off. The content input by the touch method may be a letter or a number, an instruction in various modes, a menu item which can be designated, and the like.

In this way, the display unit 151 can form a touch screen together with the touch sensor. In this case, the touch screen can function as a user input unit 123 (see FIG. 1A).

Meanwhile, the display unit 151 may include a cluster so that the driver can check the vehicle state information or the vehicle driving information while driving. Clusters can be located on the dashboard. In this case, the driver can confirm the information displayed in the cluster while keeping the line of sight ahead of the vehicle.

Meanwhile, according to the embodiment, the display unit 151 may be implemented as a Head Up Display (HUD). When the display unit 151 is implemented as an HUD, information can be output through a transparent display provided in the windshield. Alternatively, the display unit 151 may include a projection module to output information through an image projected on the windshield.

On the other hand, according to the embodiment, the display unit 151 may include a transparent display. In this case, the transparent display may be attached to the windshield.

The transparent display can display a predetermined screen while having a predetermined transparency. Transparent displays can be made of transparent TFEL (Thin Film Elecroluminescent), transparent OLED (Organic Light-Emitting Diode), transparent LCD (Liquid Crystal Display), transmissive transparent display, transparent LED (Light Emitting Diode) Or the like. The transparency of the transparent display can be adjusted.

At least some of the above-described components may operate in cooperation with one another to implement the operation, control, or control method of the vehicular controller 100 according to the various embodiments described below. The operation, control, or control method of the on-vehicle control device 100 may be implemented on the on-vehicle control device 100 by driving at least one application program stored in the memory 170. [

In the following, various embodiments may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

Hereinafter, embodiments related to a control method that can be implemented in the vehicle control apparatus configured as above will be described with reference to the accompanying drawings. It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. For example, the operations performed by the vehicle control apparatus can be modified to be performed by the head-up display.

In the following description of the drawings, the drawings drawn on the left side are referred to in the order of clockwise or top-down.

2 is a conceptual diagram for explaining a head-up display disposed in a vehicle.

A head-up display (hereinafter referred to as "HUD") of a vehicle is a device designed to display driving information on the windshield of a vehicle. Initially, it was introduced to secure the front view of an airplane pilot. It is also introduced in vehicles.

Under the control of the vehicle control unit, the HUD displays various information related to driving such as image information for guiding the route and text information for guiding the speed on a windshield glass or augmented reality (AR) ), Thereby preventing the driver's line of sight from being dispersed.

Hereinafter, an image output by the HUD is referred to as a 'HUD image'.

The output of the HUD image is implemented in such a way that the projected image is reflected by the mirror through a display module such as a projector, thereby outputting the HUD image at a position visible to the driver.

As shown in FIG. 2, the HUD image is output from the display module 210, and the path of the output image is changed by the mirror 220 and output toward the windshield 230 of the vehicle. At this time, the HUD image may be projected on the windshield 230 or may be projected on a separate screen (not shown) located between the mirror 220 and the windshield 230.

Since the HUD image is reflected on the transparent windshield, the driver can check the road while confirming the HUD image. In other words, the augmented reality can be realized by the HUD image.

The augmented reality shows virtual reality related information about real objects, and HUD images are superimposed on the background of reality inside windshields inside the vehicle. Through this, the driver can view information such as the address of the building, the trend of the real estate, the facilities provided in the building, and the usage history thereof through the HUD video in the background of the actual building.

In order to realize more perfect augmented reality, the background of the real world and the information displayed by the HUD image must be perfectly matched. When distortion occurs in the HUD image, information about the second building different from the first building is displayed in the first building, which may cause inconvenience.

On the other hand, the windshield 230 has a windshield angle, and each portion of the windshield 230 may have different curvatures depending on the position. Here, the windshield angle refers to the angle formed by the windshield with respect to the gravity direction. Due to the windshield angle and curvature, distortion may occur in the HUD image projected onto the windshield. This distortion is a problem in creating a perfect augmented reality.

In order to solve such a problem, the present invention proposes a vehicle control device and / or a head-up display (HUD) capable of correcting the distortion of the HUD image in real time.

3 is a flowchart for explaining a control method performed by the vehicle control apparatus of FIG.

First, the control unit stores a plurality of distortion correction functions in a memory (S310).

Depending on the position of the driver's eyes, different distortions occur in the HUD image. For example, when the vehicle is turning a curve, the driver is influenced by the centrifugal force exerted on the outside of the curve. The driver's eye position moves from the center of the steering wheel to the left or right, . The HUD image itself is output without distortion. However, due to driver's eye position, windshield angle, and curvature of the windshield, the driver feels distorted in the HUD image or uses distorted information.

When the vehicle is identified, the curvature of the windshield angle and the windshield has a fixed value, but the driver's eye position has different values depending on the type of the driver, the position of the seat, and the driving situation.

Here, the eye position means position information related to the eye. For example, the position information includes the position of the driver ' s eyes and the direction the eyes point at the position of the eyes.

The distortion correction function is a function for correcting the distortion of the HUD image corresponding to the eye position of the driver changing in real time. Specifically, the distortion of the HUD image generated due to the driver's eye position can be corrected by a distortion correction function corresponding to the driver's eye position.

The driver's eye position can be represented by a two-dimensional coordinate system or a three-dimensional coordinate system. For example, the driver's eye position can be expressed by the y-axis with respect to the direction of gravity and the x-axis perpendicular to the y-axis and extending along the vehicle width direction of the vehicle.

In this case, the plurality of distortion correction functions may be stored in a memory as a lookup table having the driver's eye position as a variable (e.g., x1, y1). The control unit can quickly search for any one of the distortion correction functions corresponding to the driver's eye position using the lookup table.

The plurality of distortion correction functions may be stored in memory by the manufacturer at the product launch stage.

In addition, the control unit may receive a plurality of distortion correction functions from the server and update the received plurality of distortion correction functions in the memory. The vehicle control apparatus can be mounted on various vehicles, and the distortion correction functions to be applied to the vehicle must be different. Thus, the on-vehicle control device may be installed in the vehicle, and then periodically or irregularly communicate with the server and update the distortion correction functions corresponding to the mounted vehicle.

A method of generating the distortion correction function will be described later in detail with reference to FIG.

In operation S350, the control unit senses the eye position of the driver and selects any distortion correction function corresponding to the eye position of the sensed driver among the plurality of distortion correction functions stored in the memory in operation S350.

The control unit can detect the driver's eye position using a camera disposed toward the driver's seat. More specifically, the driver's eye can be searched for in the image captured by the camera, and the position of the searched eye can be represented in a predetermined coordinate system. For example, when the predetermined coordinate system is two-dimensional, the eye position of the driver can be calculated with coordinates having the x-axis value and the y-axis value.

The control unit selects one of the plurality of distortion correction functions using the calculated coordinates. For example, in a state where the first distortion correction function corresponding to the first coordinate range and the second distortion correction function corresponding to the second coordinate range are stored, the calculated coordinates are stored in the first and second coordinate ranges A first function or a second function may be selected depending on whether it is included.

Next, the controller corrects the image output toward the windshield of the vehicle using the selected distortion correction function (S370). As the sensed eye position is changed, the distortion correction function is changed from one to another.

Since the HUD image is corrected in real time by a plurality of distortion correction functions stored in the memory, the amount of computation is small and accurate information can be provided to the vehicle in operation.

On the other hand, if the driver's eyes are not detected, the control unit can control the HUD so that the HUD image is not output. That is, the HUD image is unnecessarily output and the battery is prevented from being consumed.

When an image corrected by the distortion correction function is a post-correction image, the image before correction can be referred to as a pre-correction image. The driver may feel inconvenienced in outputting the corrected image regardless of his or her intention.

In order to solve such inconvenience, the vehicle control apparatus according to the present invention can selectively output the pre-correction image or the post-correction image toward the windshield of the vehicle based on the user input. For example, the correction activation button is implemented in a hardware or software manner, and the execution of the correction function is turned on / off each time a user input is input to the correction activation button, so that the pre-correction image or the post-correction image can be outputted .

Furthermore, the control unit may simultaneously output the pre-correction image and the post-correction image, and may output any one of the images selected by the user.

Figure 4 is a flow chart for describing in more detail a method for storing distortion correction functions in memory. 5 and 6 together, a method for generating distortion correction functions will be described in more detail.

In order to obtain a plurality of distortion correction functions, the distortion of the HUD image corresponding to different eye positions must first be specified. In order to measure the distortion of the HUD image, the vehicle has a HUD 510 and a camera 520.

The control unit outputs an image including the grid cells to the windshield of the vehicle (S312).

The HUD 510 outputs a predetermined image in the direction in which the windshield of the vehicle is located. The predetermined image is for measuring distortion and includes a plurality of regularly arranged grid cells. Although circular grid cells are shown in the figure, the shapes of the grid cells can be modified in various ways.

Next, the distortion of the grid cells is measured using a camera arranged in the driver's seat of the vehicle so as to face the image (S314).

The camera 520 corresponds to the eyes of the driver, and the distortion of the HUD image can be measured by the image captured by the camera 520. In order to specify the position of the driver's eyes, the camera 520 is disposed to face forward in the driver's seat of the vehicle.

The image captured by the camera 520 (hereinafter referred to as a 'camera image') includes the image output by the HUD.

The first camera image may be obtained at the first photographing position, and the second camera image may be obtained at the second photographing position. The controller may measure the distortion of the HUD image corresponding to the first photographing position using the first camera image and measure the distortion of the HUD image corresponding to the second photographing position using the second camera image.

The step of measuring the distortion of the HUD image includes extracting at least one figure formed by at least three adjacent grid cells among the grid cells included in the camera image and measuring the distortion of the extracted figure .

For example, as shown in FIG. 6, the distortion of the region including the four grid cells can be measured by extracting the rectangles formed by the four adjacent grid cells and measuring the squash distortion. By measuring the distortion for all the grid cells, it is possible to measure the distortion of the entire area of the HUD image.

Next, a distortion correction function corresponding to the photographing position of the camera is generated based on the distortion of the measured grid cells (S316).

A distortion correction function for normalizing the distortion of each extracted graphic is generated and a different distortion correction function may be generated according to the photographing position of the camera 520. [ For example, the first distortion correction function may be generated using the first camera image acquired at the first shooting position, and the second distortion correction function may be generated using the second camera image acquired at the second shooting position have.

The distortion correction function is made to correct the position where each grid cell should be displayed. In other words, according to the distortion correction function, the position of each grid cell is corrected so that the plurality of graphic shapes formed by the grid cells are not distorted, and all the four angles are at right angles.

Specifically, the distortion correction function may be a matrix for dividing the HUD image into a plurality of regions (or pixels), and correcting a position where each region should be output. It is possible to obtain a post-correction image in which the position of at least one region is changed by correcting the pre-correction image using the distortion correction function. Thus, the distortion that has occurred according to the pre-correction image is not found in the post-correction image.

Next, the control unit stores a plurality of distortion correction functions generated at different shooting positions in a memory (S318).

The shooting position can be artificially varied, or it can be varied by centrifugal force or other naturally occurring forces as the vehicle travels. For example, the user may capture an image while changing the photographing position of the camera 520, and the control unit may obtain a distortion correction function corresponding to each photographing position using the captured image.

On the other hand, the image captured by the camera 520 may include not only the image but also roads existing outside the windshield.

On the road, a background image 530 including grid cells may be installed so as to calculate a photographing position of the camera 520. In this case, the camera image may include at least a part of the background image 530 as well as the image output from the HUD 510.

The photographing position of the camera 520 can be calculated by the background image 530 included in the camera image. The control unit calculates the photographing position of the camera 520 using the grid cells included in the background image 530. [ For example, when the predetermined coordinate system is two-dimensional, the x-axis coordinate value and the y-axis coordinate value can be calculated as the photographing position of the camera 520. The photographing position corresponds to the eye position of the driver.

Accordingly, the vehicle can acquire camera images in various environments while driving the track on which the background image 530 is installed, and can generate distortion correction functions accordingly.

As described above, the output position of each region (or each pixel) of the HUD image is corrected by the distortion correction function that varies depending on the driver's eye position. Thus, an augmented reality in which a distortion occurring to a driver is corrected in real time and perfectly matches with the real world can be provided.

Meanwhile, at least one of the position and the size of the area in which the HUD image is output may vary depending on the eye position, depending on the driving environment of the vehicle.

FIG. 7 is a flowchart for explaining a method of controlling an image output region according to a driving environment, and FIGS. 8A and 8B are illustrations for explaining the method of FIG.

The control unit can sense the driving environment of the vehicle using a sensor provided in the vehicle (S710). Driving information is generated as the driving environment is detected.

Here, the driving information means all kinds of information related to the vehicle sensed through the sensor.

For example, the driving information may include information related to the vehicle itself, such as driving speed, weight, collision probability, characteristics of the road (kind of road such as unpaved road, highway, intersection, curvature in curve section, And information sensed in the surroundings of the vehicle.

The traveling information may include position information of the vehicle such as GPS, various information analyzed from the image captured by the image sensor, possibility of collision between the object detected by the radar or the rider and the object, (Pitch, roll, yaw) about three axes pierced in the vertical direction and the vertical direction, respectively.

Next, the control unit can predict the driver's eye position based on the sensed driving information (S730). For example, if the driver's eye position is detected at the first coordinate (x1, y2) at the time t1, the second coordinate (x2, y2) is calculated as the eye position of the driver to be positioned at the time t2 based on the detected driving information Can be predicted. Where t1 is the current time and t2 is the future time.

The control unit may determine at least one of the position and the size of the area in which the HUD image is output based on the sensed eye position (e.g., the first coordinate) or the predicted eye position (e.g., the second coordinate) S750).

Specifically, the control unit can distinguish between a steady state in which the eye position is within a predetermined range and an abnormal state in which the eye position is out of the predetermined range.

As shown in FIG. 8A, the HUD image 820 in the normal state can be output to the predetermined area 810. FIG. In a normal state, the user can confirm the whole of the HUD image 820 displayed in the predetermined area 810. However, in the abnormal state, the user can not confirm a part of the HUD image 820. [ A portion of the HUD image 820 is perceived as being cut off to the driver. This is because the display area (810- > 810 ') that the driver can see depends on the eye position.

The control unit may change at least one of the position and the size of the region in which the post-correction video is output based on the eye position when the eye position of the driver is out of the predetermined range. As a result, the HUD image can be outputted in the field of view of the driver.

The control unit can calculate the display area 810- > 810 'in which the driver can confirm the HUD image based on the eye position of the driver. The control unit controls at least one of the position and the size of the area in which the HUD image is output based on the calculated display area 810 '.

For example, by controlling the angle between the display module 210 (see FIG. 2) of the HUD and the mirror 220 (see FIG. 2), the control unit can control the position of the area in which the HUD image is output, Size can be changed.

As another example, the area in which the HUD image can be output can be fixed. In this case, the control unit may calculate the overlapping area between the area where the HUD image can be output and the display area where the driver can confirm the HUD image, and output the HUD image at the calculated area (820-> 820 ").

As the size of the area in which the HUD image is output is changed, the size of the object included in the HUD image may be changed. For example, if the size of the region increases, the size of the object included in the HUD image may also increase.

On the other hand, the control unit searches for an area where the probability that the driver's eye position is out of a predetermined range is greater than a reference based on the GPS information, and when the vehicle enters the search area, Can be changed. At this time, it is possible to predict at least one of the position and the size of the area where the post-correction image is output based on the predicted eye position. Since at least one of the at least one of the eyes is changed after the eye position of the driver is changed, at least one of the eye positions is predicted in advance and a more accurate augmented reality can be provided.

Meanwhile, the control unit may change the type of the HUD image according to the position of the eyes.

FIG. 9 is a flowchart for explaining a method of outputting different images according to the positions of eyes, and FIGS. 10A and 10B are illustrations for explaining the method of FIG.

First, the control unit determines whether the eye position is out of a predetermined range (S910). If it is not within the predetermined range, the distortion correction function is selected based on the eye position (S350), and the distortion of the HUD image is corrected using the selected distortion correction function (S370).

On the other hand, if the predetermined range is exceeded, a replacement image is selected (S930), and the distortion is corrected using the distortion correction function on the selected replacement image. If it is outside the predetermined range, at least a part of the HUD image appears to be cut off.

The substitute image may be, for example, a blur effect on the HUD image. As another example, the substitute image may include notification information that the eye position is out of a predetermined range, so that it can sit in the correct posture or repeat the HUD setting.

Referring to FIG. 10A, when the eye position is located within a predetermined range, the display area 1010 that the driver can check according to the eye position coincides with the area 1020 where the HUD image is output.

If the eye position deviates from the predetermined range, a discrepancy occurs between the display area 1010 'that the driver can check and the area 1020 where the HUD image is output, and the entire area of the HUD image is displayed in the display area 1010' At least some not included are not included in the driver ' s field of view.

In this case, the control unit may replace the entire HUD image with a predetermined alternative image, or may notify the driver that the eye position is out of a predetermined range by applying a predetermined display effect to the HUD image. In this way, the driver can correct the posture or set the wrongly set HUD so that the entire HUD image can be confirmed at his / her eye position.

In another example, when the eye position is out of a predetermined range, the control unit may edit at least one area of the entire area of the HUD image and output the edited image toward the windshield of the vehicle.

As shown in FIG. 10B, if the eye position is out of a predetermined range, a mismatch occurs between the display area 1030 that the driver can check and the area 1040 where the HUD image is output. The controller edits the HUD image so that the objects included in the HUD image can be displayed in the display area 1030 at all. Specifically, the entire area of the HUD image is divided into an object-free area including no object and an object-containing area including the object, and the HUD is controlled such that an object-containing area is displayed in the display area 1030.

On the other hand, at least one of the size and position of at least one area to be edited varies depending on the driver's eye position.

The present invention described above can be embodied as computer-readable codes on a medium on which a program is recorded. The computer readable medium includes all kinds of recording devices in which data that can be read by a computer system is stored. Examples of the computer readable medium include a hard disk drive (HDD), a solid state disk (SSD), a silicon disk drive (SDD), a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, , And may also be implemented in the form of a carrier wave (e.g., transmission over the Internet). Also, the computer may include a control unit 180 of the terminal. Accordingly, the above description should not be construed in a limiting sense in all respects and should be considered illustrative. The scope of the present invention should be determined by rational interpretation of the appended claims, and all changes within the scope of equivalents of the present invention are included in the scope of the present invention.

Claims (10)

Sensing a driver's eye position;
Selecting one distortion correction function corresponding to the sensed eye position from among a plurality of distortion correction functions stored in the memory; And
And correcting the image output toward the windshield of the vehicle using the selected distortion correction function. The method according to claim 1,
And the selected distortion correction function is changed from one to another as the sensed eye position is changed. The method according to claim 1,
Further comprising storing the plurality of distortion correction functions in the memory,
Wherein the step of storing the plurality of distortion correction functions in the memory comprises:
Outputting an image including grid cells to a windshield of the vehicle;
Measuring a distortion of the grid cells using a camera arranged to face the image at a driver's seat of the vehicle;
Generating a distortion correction function corresponding to a photographing position of the camera based on the distortion of the measured grid cells; And
And storing a plurality of distortion correction functions generated at different shooting positions in the memory. The method of claim 3,
Wherein measuring the distortion of the grid cells comprises:
Extracting at least one graphic form formed by at least three adjacent grid cells among the grid cells; And
And measuring distortion of the extracted figure,
The step of generating the distortion correction function corresponding to the photographing position of the camera generates the distortion correction function for normalizing the distortion of the extracted figure so that different distortion correction functions are generated according to the photographing position of the camera Of the vehicle. The method according to claim 1,
Further comprising changing at least one of a position and a size of an area in which the corrected image is output based on the sensed eye position when the sensed eye position is out of a predetermined range, / RTI > The method according to claim 1,
Searching for an area where the probability that the driver's eye position is out of a predetermined range is greater than a reference, based on GPS information; And
Further comprising changing at least one of a position and a size of an area where the corrected image is outputted when the vehicle enters the searched area. The method according to claim 1,
Further comprising editing at least one region of the entire area of the corrected image when the detected eye position is out of a predetermined range and outputting the edited image of at least one region to the windshield of the vehicle A control method of a control device. 8. The method of claim 7,
Wherein at least one of a size and a position of the at least one region varies depending on the sensed eye position. The method according to claim 1,
And outputting the pre-correction image or the post-correction image selectively to the windshield of the vehicle based on the user input. The method according to claim 1,
Receiving the plurality of distortion correction functions from a server; And
Further comprising updating the received plurality of distortion correction functions in the memory.

Images