KR20130036934A - Mobile terminal for performing head up display function, operating method thereof, and routine informing device interacting thereof - Google Patents

Abstract

Disclosed are a mobile terminal mounted on a vehicle and performing a head up display (HUD) function and a black box function. The mobile terminal forms a communication link with a camera, a display unit, a route guide device, and receives the route guide information in accordance with a communication unit for receiving route guide information from the route guide device and a standing angle of the mobile terminal. And a controller configured to process a display signal and display the display on the display unit, and a black box function to store an image photographed by the camera.

Description

A mobile terminal mounted on a vehicle and capable of performing a front display device function, the mobile terminal driving method, and a route guidance device interoperating with the mobile terminal. interacting

The present invention relates to a head up display (HUD) related technology, and more particularly, a mobile terminal capable of performing a front display function for route guidance information mounted on a vehicle and received from a route guidance apparatus; A method of driving the mobile terminal and a route guidance device interoperating with the mobile terminal.

As terminals, such as personal computers, laptops, mobile phones, and smart phones, are diversified in function, multimedia with complex functions such as taking pictures or videos, playing music or video files, receiving games and broadcasting, etc. It is implemented in the form of a multimedia player.

For functional support and enhancement of the terminal, it may be considered to improve the structural and / or software part of the terminal. Recently, as various terminals including a mobile terminal provide complex and various functions, the menu structure is also complicated, and there is an increasing interest in a technology capable of performing a front display function for traffic-related information by executing an application in the mobile terminal. have.

The technical problem to be solved by the present invention is a mobile terminal capable of performing a front display function for the route guidance information received from the route guidance device according to the angle mounted on the vehicle or a black box function and a driving method of the mobile terminal To provide.

Another technical problem to be solved by the present invention is a movement mounted to a vehicle to generate an augmented reality image based on an external vehicle image received from an image acquisition device, and to perform a front display function on the generated augmented reality image. It is to provide a terminal and a driving method of the mobile terminal.

Another technical problem to be solved by the present invention is to provide a route guidance device that can change the display form of the route guidance information displayed on the display unit when the route guidance information is transmitted to the front display device in conjunction with the front display device It is.

Technical problems to be achieved by the present invention are not limited to the above-mentioned technical problems, and other technical problems not mentioned above will be clearly understood by those skilled in the art from the following description. Could be.

In order to solve the above technical problem, a mobile terminal according to an embodiment of the present invention mounted on a vehicle to perform a head up display (HUD) function and a black box function includes a camera; A display unit; A communication unit for establishing a communication link with the route guidance device and receiving route guidance information from the route guidance device; And a front display device function of processing the received route guidance information into a front display signal according to a standing angle of the mobile terminal and displaying the image captured by the camera. It may include a control unit.

In order to solve the above technical problem, a mobile terminal according to an embodiment of the present invention mounted on a vehicle to perform a front display device function includes a display unit; A communication unit configured to receive an external vehicle image from an image acquisition device; And a control unit configured to obtain augmented reality information corresponding to a subject of the external vehicle image and to process the augmented reality image matching the augmented reality information to a subject of the external vehicle image as front image information for display on the display unit. It may include.

In order to solve the above technical problem, a mobile terminal driving method according to an embodiment of the present invention, which is mounted on a vehicle and may perform a head up display (HUD) function, forms a communication link with a route guidance device. Making; Sensing a standing angle of the mobile terminal; According to the standing angle of the mobile terminal, the black box function to receive the route guidance information from the route guidance device to process the front display signal to display the display on the display unit or to store the image taken by the camera It may include the step of performing.

In order to solve the above technical problem, a mobile terminal driving method according to another embodiment of the present invention, which is mounted on a vehicle and performs a front display device function, forms a communication link with an image capturing apparatus, and sets the vehicle from the image capturing apparatus. Receiving an external image; Acquiring augmented reality information corresponding to a subject of the received external vehicle image; And processing the augmented reality image obtained by matching the augmented reality information to the subject of the vehicle external image as the front display image information and displaying the same on the display unit.

Each of the mobile terminal driving methods according to the above-described embodiments of the present invention can be implemented by executing a computer program for implementing the mobile terminal driving method stored in a computer-readable recording medium.

According to an aspect of the present invention, there is provided a route guidance device including a communication unit forming a communication link with a front display device; A display unit displaying route guidance information; And a controller configured to provide at least a part of the route guidance information to the front display apparatus when the front display device is activated, and to change a display state of the route guide information on the display unit.

According to the present invention, the user may be provided with a mobile terminal and a method of driving the mobile terminal capable of performing a front display function or a black box function on the route guidance information received from the route guidance device according to the angle mounted on the vehicle. have.

According to the present invention, a user can generate an augmented reality image based on an exterior image of a vehicle received from an image capturing apparatus, and perform a front display function on the generated augmented reality image and a driving method of the mobile terminal. Can be provided.

According to the present invention, the user may be provided with a route guidance apparatus that can change the display form of route guidance information displayed on the display unit when the route guidance information is transmitted to the front display apparatus in association with the front display apparatus.

1 is a block diagram of a system for providing a front display function according to the present invention.
Figure 2 shows that the mobile terminal according to the present invention is mounted on the vehicle coupled to the cradle.
FIG. 2B illustrates a side view of a state in which the mobile terminal illustrated in FIG. 2 is mounted in a vehicle.
3A to 3C illustrate a result in which vehicle driving information, route guidance information, etc., which are processed into signals for front display by the mobile terminal according to the present invention performing the front display function, are reflected on the windshield of the vehicle.
4 illustrates that the vehicle driving information processed by the front display signal is displayed on the display unit in the mobile terminal according to the present invention which performs the front display function.
5 illustrates vehicle driving information processed by the mobile terminal according to the present invention performing the front display function and the vehicle driving information reflected on the windshield of the vehicle.
6A and 6B illustrate an augmented reality image processed as a signal for front display by a mobile terminal performing a front display function according to an exemplary embodiment of the present invention is reflected on the windshield of the vehicle.
7 is a block diagram of a mobile terminal according to the present invention.
8 is a flowchart illustrating a method of providing a front display function of a mobile terminal according to an embodiment of the present invention.
9 illustrates an example of a standing angle at which the mobile terminal according to the present invention performs the front display function and the black box function.
10 is a flowchart illustrating a process of performing a front display function by a mobile terminal according to the present invention.
11 is a view illustrating a screen for setting route guidance information for front display of a front display application executed in a mobile terminal according to the present invention.
12 is a flowchart illustrating another example of a process of performing a front display function by a mobile terminal according to the present invention.
13 is a flowchart illustrating another example of a method of performing a front display function of a mobile terminal according to the present invention.
FIG. 14 illustrates that the route guidance display state of the route guidance apparatus is changed according to the method of performing the front display function illustrated in FIG. 13.
15 is a flowchart illustrating an example of a method of performing a black box function of a mobile terminal according to the present invention.
FIG. 16 illustrates an example of a method of generating an augmented reality image using an external vehicle image received from an image capturing apparatus in a mobile terminal and performing a front display function using the generated augmented reality image.
FIG. 17 illustrates another example of a method of generating an augmented reality image using an external vehicle image received from an image acquisition device in a mobile terminal and performing a front display function using the generated augmented reality image.
18 is a block diagram showing a schematic configuration of a route guidance device according to the present invention.
19 is a flowchart illustrating a method of driving a route guidance device according to the present invention.
20 illustrates an example in which a display state of route guidance information is changed according to the driving method of the route guidance apparatus illustrated in FIG. 19.
FIG. 21 illustrates another example in which the display state of the route guide information is changed according to the driving method of the route guide apparatus shown in FIG. 19.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings which illustrate preferred embodiments of the present invention and the contents described in the accompanying drawings.

In the present specification, when one component 'transmits or transmits' data or a signal to another component, the component may directly transmit or transmit the data or signal to the other component, and at least one It means that the data or signal can be transmitted to the other component through another component. In addition, the suffix "module" and " part "for constituent elements used in the following description are given or mixed in consideration of ease of specification, and do not have their own meaning or role.

The foregoing objects, features and advantages of the present invention will become more apparent from the following detailed description taken in conjunction with the accompanying drawings. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Like numbers refer to like elements throughout. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

1 is a configuration diagram of a system 10 for providing a front display function according to the present invention. Referring to FIG. 1, the system 10 includes a mobile terminal 100, a route guidance device 200, an image acquisition device 300, and a service server 400 connected to each other via a wired / wireless communication network. .

The mobile terminal 100 may be mounted on a vehicle to perform a front display device function. That is, the image displayed on the display unit 151 of the mobile terminal 100 may be reflected on the windshield of the vehicle, and the vehicle driver may obtain information by checking the image reflected on the windshield. The mobile terminal 100 described herein may include a mobile phone, a smart phone, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), and the like. However, the scope of the present invention is not limited thereto.

On the other hand, the mobile terminal 100 is preferably mounted on the vehicle at an angle at which the image reflected on the windshield is well visible to the driver. For example, the mobile terminal 100 may be mounted on a holder that can change the angle with respect to the windshield. In addition, the vehicle itself may be provided with a structure in which the mobile terminal 100 is mounted at an angle at which an image reflected on the windshield is easily seen by a user.

2a shows that the mobile terminal 100 according to the present invention is coupled to a cradle 500 and mounted on a vehicle. FIG. 2B illustrates a side view of a state in which the mobile terminal 100 illustrated in FIG. 2A is mounted on a vehicle. 2A and 2B, the holder 500 includes an adsorption part 510, a connection part 520, and a coupling part 530. The adsorption part 510 serves to fix the cradle 500 to the vehicle windshield 20. The connection part 520 connects the adsorption part 510 and the coupling part 530. The coupling part 530 fixes the mobile terminal 100 by using a plurality of fixing means 532. The rotating part 534 provided at the connection part of the coupling part 530 with the connection part 520 enables the coupling part 530 to rotate. The shape of the holder on which the mobile terminal 100 is mounted is not limited to the structure and shape shown in FIGS. 2A and 2B.

Referring back to FIG. 1, the mobile terminal 100 receives route guide information from the route guide device 200 and outputs the received route guide information through the display unit 151. Then, the route guidance information is reflected on the windshield of the vehicle, and the vehicle driver may check the reflected route guidance information.

3A to 3C illustrate a result in which vehicle driving information, route guidance information, etc., which are processed as signals for front display by the mobile terminal 100 according to the present invention which performs the front display function, is reflected on the windshield of the vehicle.

In the reflective area 30 of the vehicle windshield shown in FIG. 3A, the speed limit of the currently running road and the speed of the current vehicle received from the route guidance device 200 are reflected by the mobile terminal 100 on the windshield of the vehicle. It can be seen that. It can be seen that the route guidance information received from the route guidance device 200 is reflected in the reflective region 30 of the vehicle windshield shown in FIG. 3B. Referring to the reflection area 30 of the vehicle windshield shown in FIG. 3C, it can be seen that the pedestrian attention information received from the route guidance device 200 is reflected.

4 illustrates that the vehicle driving information processed by the front display signal is displayed on the display unit 151 in the mobile terminal 100 according to the present invention which performs the front display function.

4 (a) shows that route guidance information is displayed on the route guidance apparatus 200. 4B illustrates a display unit 151 of the mobile terminal 100 in a state in which the route guidance information shown in FIG. 4A is transmitted to the mobile terminal 100 and then processed into a front display signal. Indicates what is shown in. Referring to FIG. 4B, it can be seen that the route guidance information displayed on the mobile terminal 100 is displayed in a state in which left and right are reversed. The route guidance information displayed in the inverted state is reflected on the vehicle windshield, so that the left and right are inverted again to provide the screen of FIG. 4A to the user through the vehicle windshield.

5 illustrates vehicle driving information processed by the mobile terminal 100 according to the present invention which performs the front display function, and vehicle driving information reflected on the windshield of the vehicle.

Referring to FIG. 5, it may be seen that the path information displayed on the display unit 151 of the mobile terminal 100 is inverted in left and right directions. However, the route guidance information displayed on the display unit 151 is reflected on the windshield of the vehicle and the left and right are reversed again.

Referring back to FIG. 1, the mobile terminal 100 receives an exterior image of the vehicle from the image acquisition apparatus 300 and receives an augmented reality image combined with the exterior image of the vehicle and a virtual image matching the exterior image of the vehicle. The generated augmented reality image may be processed into image information for front display and displayed on the display unit 151.

6A and 6B illustrate an augmented reality image processed as a signal for front display by the mobile terminal 100 performing the front display function according to an exemplary embodiment of the present invention is reflected on the windshield 30 of the vehicle.

In FIG. 6A, it can be seen that the virtual image is coupled to the vehicle exterior image and reflected on the vehicle windshield. In FIG. 6A, the building information, the gas station information, and the like displayed in the box may be a POI and is virtual information matched to the external image of the vehicle. The search for the virtual image matching the actual vehicle exterior image may be performed by the mobile terminal 100, or may be performed by the service server 400 at the request of the mobile terminal 100. Referring to FIG. 6B, it can be seen that the virtual image including the building information displayed in the box and the road covered by the building is combined with the real image and reflected on the windshield of the vehicle.

7 is a block diagram of a mobile terminal 100 according to an embodiment of the present invention. Referring to FIG. 7, the mobile terminal 100 includes a wireless communication unit 110, an A / V input unit 120, a user input unit 130, a sensing unit 140, an output unit 150, The storage unit 160, the interface unit 170, the controller 180, and the power supply unit 190 may be included. Since the components shown in FIG. 7 are not essential, the mobile terminal 100 may have more components or fewer components.

Hereinafter, the components will be described in order.

The wireless communication unit 110 may include one or more modules for enabling wireless communication between the mobile terminal 100 and the wireless communication system or between the mobile terminal 100 and the network in which the mobile terminal 100 is located. For example, the wireless communication unit 110 may include a broadcast receiving module 111, a mobile communication module 112, a wireless Internet module 113, a short range communication module 114, and a location information module 115 .

The broadcast receiving module 111 receives a broadcast signal and / or broadcast related information from an external broadcast management server through a broadcast channel.

The broadcast channel may include a satellite channel and a terrestrial channel. The broadcast management server may refer to a server for generating and transmitting broadcast signals and / or broadcast related information, or a server for receiving broadcast signals and / or broadcast related information generated by the broadcast management server and transmitting the generated broadcast signals and / or broadcast related information. The broadcast signal may include a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and a broadcast signal in which a data broadcast signal is combined with a TV broadcast signal or a radio broadcast signal.

The broadcast-related information may refer to a broadcast channel, a broadcast program, or information related to a broadcast service provider. The broadcast related information may also be provided through a mobile communication network. In this case, it may be received by the mobile communication module 112.

The broadcast related information may exist in various forms. For example, it may exist in the form of Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB) or Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H).

The broadcast receiving module 111 receives broadcasting signals using various broadcasting systems. In particular, the broadcasting receiving module 111 may be a Digital Multimedia Broadcasting-Terrestrial (DMB-T), a Digital Multimedia Broadcasting-Satellite (DMB-S) Only Digital Broadcast-Handheld (DVB-H), Integrated Services Digital Broadcast-Terrestrial (ISDB-T), and the like. Of course, the broadcast receiving module 111 may be adapted to other broadcasting systems that provide broadcast signals as well as the digital broadcasting system described above.

The broadcast signal and / or broadcast related information received through the broadcast receiving module 111 may be stored in the storage unit 160.

The mobile communication module 112 transmits and receives a radio signal to at least one of a base station, an external terminal, and a server on a mobile communication network. The wireless signal may include various types of data in response to a voice call signal, a video call call signal, or a text / multimedia message transmission / reception.

The wireless Internet module 113 refers to a module for wireless Internet access, and the wireless Internet module 113 can be embedded in the mobile terminal 100 or externally. WLAN (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), HSDPA (High Speed Downlink Packet Access), LTE (Long Term Evolution) .

The short range communication module 114 refers to a module for short range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The location information module 115 is a module for confirming or obtaining the location of the mobile terminal. The location information module 115 may obtain location information using a global navigation satellite system (GNSS). Here, the Global Positioning System (GNSS) is a term used to describe the radionavigation satellite systems in which certain types of radionavigation receivers orbiting the Earth are transmitting reference signals that can determine their position near the surface or ground surface . The Global Positioning System (GNSS) includes Global Positioning System (GPS) in the United States, Galileo in Europe, GLONASS (Global Orbiting Navigational Satelite System) in Russia, COMPASS in China and Japan QZSS (Quasi-Zenith Satellite System), which is operated by the Ministry of Land, Infrastructure and Transport.

As a representative example of the GNSS, the location information module 115 may be a Global Position System (GPS) module. The GPS module calculates information on a distance (distance) from three or more satellites to a point (object), information on a time when the distance information is measured, and then applies a trigonometric method to the calculated distance information, Dimensional position information according to latitude, longitude, and altitude with respect to a point (object) in the three-dimensional space. Further, a method of calculating position and time information using three satellites and correcting the error of the calculated position and time information using another satellite is also used. The GPS module continuously calculates the current position in real time, and calculates velocity information using the current position.

Referring to FIG. 7, the A / V input unit 120 is for inputting an audio signal or a video signal, and may include a camera 121 and a microphone 122. The camera 121 processes image frames such as still images or moving images obtained by the image sensor in the video communication mode or the photographing mode. The processed image frame can be displayed on the display unit 151. [

The image frame processed by the camera 121 may be stored in the storage unit 160 or transmitted to the outside through the wireless communication unit 110. [ The camera 121 may be equipped with two or more cameras according to the configuration of the terminal.

The microphone 122 receives an external sound signal through a microphone in a communication mode, a recording mode, a voice recognition mode, or the like, and processes it as electrical voice data. The processed voice data can be converted into a form that can be transmitted to the mobile communication base station through the mobile communication module 112 when the voice data is in the call mode, and output. Various noise reduction algorithms may be implemented in the microphone 122 to remove noise generated in receiving an external sound signal.

The user input unit 130 generates input data for a user to control the operation of the terminal. The user input unit 130 may include a key pad dome switch, a touch pad (static pressure / capacitance), a jog wheel, a jog switch, and the like.

The sensing unit 140 senses the movement of the mobile terminal 100 such as the open / close state of the mobile terminal 100, the position of the mobile terminal 100, the presence or absence of user contact, And senses the current state of the terminal 100 to generate a sensing signal for controlling the operation of the mobile terminal 100. The sensing signal may be transmitted to the controller 180 so that the controller 180 may perform a specific function.

The sensing unit 140 may include a touch sensor for detecting a touch of a user, a vibration sensor for detecting vibration generated based on the touch of the user, and a gyro for detecting rotation of the mobile terminal 100. Sensors, acceleration sensors, geomagnetic sensors, and the like. However, the scope of the present invention is not limited thereto.

For example, when the mobile terminal 100 is in the form of a slide phone, it may sense whether the slide phone is opened or closed. In addition, it may be responsible for a sensing function related to whether the power supply unit 190 is powered on, whether the interface unit 170 is connected to an external device, and the like. Meanwhile, the sensing unit 140 may include a proximity sensor.

The output unit 150 may include a display unit 151, an acoustic output unit 152, an alarm unit 153, and a haptic module 154, for example, for generating an output related to visual, auditory, have.

The display unit 151 displays and outputs information processed by the mobile terminal 100. For example, when the mobile terminal is in the call mode, a UI (User Interface) or a GUI (Graphic User Interface) associated with a call is displayed. When the mobile terminal 100 is in the video communication mode or the photographing mode, the photographed and / or received video or UI and GUI are displayed.

The display unit 151 may be a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a three-dimensional display 3D display).

Some of these displays may be transparent or light transmissive so that they can be seen through. This may be referred to as a transparent display. A typical example of the transparent display is a transparent LCD or the like. The rear structure of the display unit 151 may also be of a light transmission type. With this structure, the user can see an object located behind the terminal body through the area occupied by the display unit 151 of the terminal body.

There may be two or more display units 151 according to the embodiment of the mobile terminal 100. For example, the plurality of displays may be spaced apart or integrally disposed on one surface of the mobile terminal 100, or may be disposed on different surfaces.

When the display unit 151 and a sensor for detecting a touch operation (hereinafter, referred to as a touch sensor) form a mutual layer structure (hereinafter referred to as a touch screen), the display unit 151 may be configured in addition to an output device. Can also be used as an input device. The touch sensor may have the form of, for example, a touch film, a touch sheet, a touch pad, or the like.

The touch sensor may be configured to convert a change in a pressure applied to a specific portion of the display unit 151 or a capacitance generated in a specific portion of the display unit 151 into an electrical input signal. The touch sensor can be configured to detect not only the position and area to be touched but also the pressure at the time of touch.

If there is a touch input to the touch sensor, the corresponding signal (s) is sent to the touch controller. The touch controller processes the signal (s) and transmits the corresponding data to the controller 180. As a result, the controller 180 can know which area of the display unit 151 is touched.

A proximity sensor may be disposed in an inner region of the mobile terminal or in the vicinity of the touch screen that is wrapped by the touch screen. The proximity sensor refers to a sensor that detects the presence or absence of an object approaching a predetermined detection surface or a nearby object without mechanical contact using the force of an electromagnetic field or infrared rays. The proximity sensor has a longer life span than the contact sensor and its utilization is also high.

Examples of the proximity sensor include a transmission photoelectric sensor, a direct reflection photoelectric sensor, a mirror reflection photoelectric sensor, a high frequency oscillation proximity sensor, a capacitive proximity sensor, a magnetic proximity sensor, and an infrared proximity sensor.

And to detect the proximity of the cursor with the change of the electric field along the proximity of the cursor when the touch screen is electrostatic. In this case, the touch screen (touch sensor) may be classified as a proximity sensor.

The proximity sensor detects a proximity touch and a proximity touch pattern (e.g., a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, a proximity touch movement state, and the like). Information corresponding to the sensed proximity touch operation and proximity touch pattern may be output on the touch screen.

The audio output unit 152 may output audio data received from the wireless communication unit 110 or stored in the storage unit 160 in a call signal reception mode, a call mode or a recording mode, a voice recognition mode, a broadcast reception mode, The sound output unit 152 outputs an acoustic signal related to a function (e.g., a call signal reception sound, a message reception sound, etc.) performed in the mobile terminal 100. [ The audio output unit 152 may include a receiver, a speaker, a buzzer, and the like. Also, the sound output unit 152 may output sound through the earphone jack 116. The user can connect the earphone to the earphone jack 116 to hear the sound output.

The alarm unit 153 may output a signal for notifying occurrence of an event of the mobile terminal 100. Examples of events that occur in the mobile terminal include call signal reception, message reception, key signal input, touch input, and the like. The alarm unit 153 may output a signal for notifying the occurrence of an event in a form other than the video signal or the audio signal, for example, vibration. The video signal or the audio signal may also be output through the display unit 151 or the audio output unit 152.

The haptic module 154 generates various tactile effects that the user can feel. A typical example of the haptic effect generated by the haptic module 154 is vibration. The intensity and pattern of vibration generated by the haptic module 154 can be controlled. For example, different vibrations may be synthesized and output or sequentially output.

In addition to the vibration, the haptic module 154 may be configured to perform various functions such as an effect of stimulation by a pin arrangement vertically moving with respect to a contact skin surface, an effect of stimulation by air spraying force or suction force through a jet opening or a suction opening, A variety of tactile effects such as an effect of stimulation through contact of an electrode, an effect of stimulation by an electrostatic force, and an effect of reproducing a cold sensation using a heat absorbing or exothermic element can be generated.

The haptic module 154 may not only deliver the haptic effect through direct contact, but also implement the haptic effect through the muscle sense of the user's finger or arm. The haptic module 154 may include two or more haptic modules 154 according to the configuration of the portable terminal 100.

The storage unit 160 may store a program for the operation of the controller 180 and may temporarily store input / output data (e.g., a phone book, a message, a still image, The storage unit 160 may store data related to vibration and sound of various patterns outputted when a touch is input on the touch screen.

According to an embodiment of the present invention, the storage unit 160 stores a vehicle exterior image acquired from the image acquisition apparatus 300 and a virtual image matching the vehicle exterior image.

The storage unit 160 may be a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (for example, SD or XD memory) (Random Access Memory), SRAM (Static Random Access Memory), ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM A disk, and / or an optical disk. The mobile terminal 100 may operate in association with a web storage that performs a storage function of the storage unit 160 on the Internet.

The interface unit 170 serves as a path for communication with all external devices connected to the mobile terminal 100. The interface unit 170 receives data from an external device or receives power from the external device to transfer the data to each component in the mobile terminal 100 or to transmit data in the mobile terminal 100 to an external device.

For example, a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device having an identification module, an audio I / O port, A video input / output (I / O) port, an earphone port, and the like may be included in the interface unit 170.

The interface unit 170 may include a card slot on which the card type user identification module can be mounted. Then, the user identification module may be connected to the mobile terminal 100 through the card slot. At this time, a plurality of user identification modules may be connected to the interface unit 170.

When the mobile terminal 100 is connected to an external cradle, the interface unit 170 may be a path through which power from the cradle is supplied to the mobile terminal 100, or various commands A signal may be a path through which the signal is transmitted to the mobile terminal. The various command signals or the power source input from the cradle may be operated as a signal for recognizing that the mobile terminal is correctly mounted on the cradle.

The control unit 180 typically controls the overall operation of the mobile terminal. For example, voice communication, data communication, video communication, and the like. The control unit 180 may include a multimedia module 181 for multimedia playback. The multimedia module 181 may be implemented in the control unit 180 or may be implemented separately from the control unit 180. [

The controller 180 may perform a pattern recognition process for recognizing handwriting input or drawing input performed on the touch screen as characters and images, respectively.

Under the control of the control unit 180, the power supply unit 190 receives external power and internal power, and supplies power necessary for operation of the respective components.

The various embodiments described herein may be embodied in a recording medium readable by a computer or similar device using, for example, software, hardware, or a combination thereof.

According to a hardware implementation, the embodiments described herein may be implemented as application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays May be implemented using at least one of processors, controllers, micro-controllers, microprocessors, and electrical units for performing functions. In some cases, And may be implemented by the control unit 180.

According to a software implementation, embodiments such as procedures or functions may be implemented with separate software modules that perform at least one function or operation. The software code may be implemented by a software application written in a suitable programming language. Also, the software codes may be stored in the storage unit 160 and executed by the control unit 180. [

In the above, the configuration of the mobile terminal 100 according to the present invention has been described with reference to FIG. 7. Hereinafter, the front display technology implemented in the mobile terminal 100 according to the above-described embodiment of the present invention will be described in detail with reference to FIGS. 2 to 17.

8 is a flowchart illustrating a method of providing a front display function of a mobile terminal 100 according to an embodiment of the present invention. Hereinafter, a method of providing the front display function will be described with reference to necessary drawings.

The mobile terminal 100 senses the standing angle of the mobile terminal 100 (S100). The standing angle sensing may be performed by various sensors such as a tilt sensor, a gyro sensor, a gravity sensor, and the like mounted on the mobile terminal 100. When the standing angle is sensed, the mobile terminal 100 determines whether the sensed standing angle is included in the first range or the second range (S110).

If the sensed standing angle is included in the first range, the mobile terminal 100 performs a front display function (S120). When the sensed standing angle is included in the second range, the mobile terminal 100 performs a black box function using the mounted camera (S130). The first range and the second range may be changed by the user of the mobile terminal 100.

9 illustrates an example of a standing angle at which the mobile terminal according to the present invention performs the front display function and the black box function.

Referring to FIG. 9, when the standing angle of the horizontal plane of the mobile terminal 100 is a first range between 10 degrees and 45 degrees, the mobile terminal 100 performs a front display function, and 90 degrees to 135 degrees. If the second range between the mobile terminal 100 may perform a black box function.

10 is a flowchart illustrating a process in which the mobile terminal 100 performs the front display function according to the present invention. For reference, FIGS. 10 and 11 illustrate a process in which the mobile terminal 100 performs a front display function for route guidance information received from the route guidance apparatus.

When the mobile terminal 100 performs the front display function, the mobile terminal 100 first forms a communication link with the route guidance device 200 using the communication unit 120 (S200). The communication link may be a wireless communication link based on a Wi-Fi communication link, a Bluetooth communication link, or a wired communication link such as a USB communication link.

When a communication link with the route guidance device 200 is established, the mobile terminal 100 receives route guidance information from the route guidance device 200 (S210). Then, the mobile terminal 100 filters the route guidance information for front display among the received route guidance information (S220). Here, the path guidance information for front display may be predetermined by the user.

11 is a view illustrating a path guide information setting screen for front display of a HUD application executed in a mobile terminal according to the present invention. Referring to FIG. 11, it can be seen that a user can select route guidance information such as lane information, direction information, vehicle speed, and distance to a destination by executing an environment setting function of the HUD application.

When the route guidance information for front display is filtered, the mobile terminal 100 processes the filtered route guidance information into a front display signal (S230). Here, processing to the front display signal means inverting the image left and right as described with reference to FIGS. 4 and 5.

When the route guidance information is processed into a front display signal, the mobile terminal 100 displays the processed front display signal on the display unit 151 (S240). Then, the front display signal is reflected on the vehicle windshield, and the vehicle driver may check the reflected front display signal.

12 is a flowchart illustrating another example of a process in which the mobile terminal 100 performs the front display function according to the present invention.

First, a communication link with the route guidance device 200 is formed (S300). Then, the mobile terminal 100 requests the route guidance device 200 to transmit route guidance information for forward display through the communication link (S310). That is, the mobile terminal 100 requests only predetermined route guidance information.

In response to the request, the route guidance information for front display is received from the route guidance apparatus 200 (S320). When the route guidance information is received, the mobile terminal 100 processes the received front guidance route guidance information into a front display signal (S330) and displays it on the display unit 151.

13 is a flowchart illustrating another example of a method for performing a front display function of a mobile terminal 100 according to the present invention.

The front display function of the mobile terminal 100 is activated (S400). Then, the mobile terminal 100 receives the route guidance information from the route guidance device 200 and performs a front display function (S410). While performing the front display function, the mobile terminal 100 controls the route guidance apparatus 200 to change the display state of route guidance information (S420).

FIG. 14 illustrates that the route guidance display state of the route guidance apparatus 200 is changed according to the method of performing the front display function illustrated in FIG. 13.

Referring to FIG. 14A, it can be seen that the path guide device 200 is currently executing a DMB function and a path guide function. In the state of FIG. 14A, the front display function is activated. Then, the mobile terminal 100 controls the route guide device 200 to block the output of the route guidance image displayed on the display unit 241, as shown in (b) of FIG. The route guidance image is processed to project route guidance information onto the vehicle windshield 30, as shown in FIG. At this time, the output state of the route guidance voice in the route guidance device 200 is preferably maintained as it is.

15 is a flowchart illustrating an example of a method of performing a black box function of the mobile terminal 100 according to the present invention.

When the vehicle is mounted on the vehicle and the standing angle falls within a predetermined range, the black box function of the mobile terminal 100 is activated (S500). Then, the mobile terminal 100 photographs an external image of the vehicle using the camera 121 (S510). Then, the mobile terminal 100 determines a method of storing the captured vehicle external image (S520).

The captured vehicle external image may be stored in the storage 160 of the mobile terminal 100 (S530, case 1). The captured vehicle external image may be transmitted to the route guide device 200 (S540), and stored in the route guide device 200 (S550, case2). The captured vehicle external image may be transmitted to the black box service server 400 (s560) and stored in the service server 400 (S570, case 3). In addition, the captured vehicle external image may be transmitted to the route guide device 200 and then transmitted to the service server 400 by the route guide device 200.

FIG. 16 illustrates a method for generating an augmented reality image using an external image of a vehicle received from an image acquisition apparatus 300 and performing a front display function using the generated augmented reality image in the mobile terminal 100 according to the present invention. An example is shown.

First, the mobile terminal 100 receives an external vehicle image from the image acquisition apparatus 300 through the communication unit 120 (S600). Then, the mobile terminal 100 searches for augmented reality information corresponding to a subject of the external image of the vehicle from the augmented reality information data previously stored in the storage unit 160 (S610). The subject of the external image of the vehicle may include a structure such as a building, a road, and natural objects such as a forest and a river on the external image of the vehicle. The augmented reality information may be periodic information such as names of the structure and the natural object. Meanwhile, unlike FIG. 16, the vehicle external image may be photographed by a camera mounted on the route guide device 200 and received from the route guide device 200.

When the augmented reality information corresponding to the subject of the external vehicle image is completed, the mobile terminal 100 matches the retrieved augmented reality information to the subject of the external vehicle image (S620). Next, the mobile terminal 100 processes the subject of the matched vehicle external image and the augmented reality information into a front display signal (S630), and then displays the processed front display signal on the display unit 151. (S640). Then, the augmented reality image may be reflected to the vehicle windshield and provided to the vehicle driver. FIG. 17 illustrates an augmented reality image using an external vehicle image received from the image acquisition apparatus 300 in the mobile terminal 100 according to the present invention. And another example of a method of performing a front display function using the generated augmented reality image.

First, the mobile terminal 100 receives an external vehicle image from the image acquisition apparatus 300 through the communication unit 120 (S700). Then, the mobile terminal 100 transmits the received vehicle exterior image to the service server 400 and requests a search for augmented reality information corresponding to a subject of the vehicle exterior image (S710).

Based on the search request, the mobile terminal 100 receives augmented reality information retrieved from the service server 400 (S720), and matches the augmented reality information to a subject of the external vehicle image (S730). Thereafter, the mobile terminal 100 processes the subject of the matched vehicle external image and the augmented reality information into a front display signal (S740), and then displays the processed front display signal on the display unit 151. (S750). Then, the subject and the augmented reality information of the vehicle external image may be reflected to the windshield of the vehicle and provided to the vehicle driver.

18 is a block diagram showing a schematic configuration of a route guidance device 200 according to the present invention. When the front display device connected to the route guide device 200 is activated, at least some of the route guide information may be provided to the front display device, and the output state of the route guide information displayed on its own may be changed.

For example, when the front display apparatus is activated and starts to perform its function, the route guidance apparatus 200 may block the display of the route guidance information itself. In this case, the path guidance apparatus 200 may continue to output the path guidance sound. Then, the vehicle driver may be provided with the route guidance image by the front display apparatus, and the route guidance sound may be provided through the route guide apparatus 200.

Also, when the front display apparatus is activated while the route guidance function and other functions are activated, the route guidance apparatus 200 displays an area where information corresponding to the other function is displayed on the route guidance apparatus 200. It may be enlarged and reduced in an area in which the route guidance information is displayed.

Referring to FIG. 18, the route guidance device 200 may include a communication unit 210, an input unit 220, a sensing unit 230, an output unit 240, a storage unit 250, a power supply unit 260, and a control unit 270. It may include. Since the components shown in FIG. 18 are not essential, the vehicle route guidance apparatus 200 may be implemented to have more components or fewer components. Hereinafter, the components will be described in order.

The communication unit 210 may include one or more modules that enable communication with the mobile terminal 100 and / or an external network according to the present invention. For example, the communication unit 100 may include a location data module, a wireless internet module, a broadcast transmission / reception module, a short range communication module, a wired communication module, and the like.

The position data module is a module for checking or obtaining position data of the route guidance device 200. Location data module (A method of obtaining this location data may be a method of obtaining location data through a global navigation satellite system (GNSS).

The wireless internet module is an apparatus that accesses the wireless internet to acquire or transmit data. The wireless Internet that can be accessed through the wireless internet module may be a wireless LAN (WLAN), a wireless broadband (WBRO), a world interoperability for microwave access (Wimax), a high speed downlink packet access (HSDPA), or the like.

The broadcast transmission / reception module is an apparatus for receiving broadcast signals through various broadcast systems. The broadcast system that can be received through the broadcast transmission / reception module 115 includes: Digital Multimedia Broadcasting Terrestrial (DMBT), Digital Multimedia Broadcasting Satellite (DMBS), Media Forward Link Only (MediaFLO), Digital Video Broadcast Handheld (DVBH), and ISDBT ( Integrated Services Digital Broadcast Terrestrial). The broadcast signal received through the broadcast transmission / reception module 115 may include traffic data, living data, and the like.

The short-range communication module is a module for short-range communication. Bluetooth, radio frequency identification (RFID), infrared data association (IrDA), ultra wideband (UWB), ZigBee, and the like can be used as the short distance communication technology.

The wired communication module serves to provide an interface with another electronic device connected to the route guidance device 200. For example, the wired communication module 119 may be a USB module capable of communicating through a USB port.

The input unit 220 is a module that generates input data for controlling the operation of the route guidance device 200. The input unit 220 may generate input data by converting a physical input from the outside into a specific electrical signal. The input unit 220 may include a user input module, a microphone, an image acquisition module, and the like.

The user input module receives a control input for controlling the operation of the route guidance device 200 from the user. The user input module may include a key pad dome switch, a touch pad (constant voltage / capacitance), a jog wheel, a jog switch, and the like.

The microphone is a device for receiving a user's voice and sound generated from inside and outside the vehicle. The image acquisition module is an apparatus for acquiring images inside and outside the vehicle. For example, the image acquisition module may include a camera.

The sensing unit 230 detects a current state of the path guide device 200 and generates a sensing signal for controlling the operation of the path guide device 200. The sensing unit 230 may include a motion sensing module, an optical sensing module, and the like.

The motion sensing module may detect a motion in the three-dimensional space of the path guide device 200. The motion sensing module may include a three-axis geomagnetic sensor and a three-axis acceleration sensor. The motion data acquired through the motion sensing module may be combined with the position data obtained through the position data module 111 to calculate a more accurate trajectory of the vehicle to which the route guide device is attached.

The light sensing module is a device for measuring ambient illuminance of the path guidance device 200. The brightness of the display unit 241 may be changed to correspond to the ambient brightness by using the illumination data acquired through the light sensing module.

The output unit 240 is a device for the route guidance device 200 to output data. The output unit 240 may include a display unit 241, a sound output unit 242, and the like.

The display unit 241 is a device for outputting data that can be visually recognized by the route guidance device 200. Meanwhile, as described above, when the display unit 241 is a touch screen, the display unit 241 may simultaneously function as the output unit 240 and the input unit 220. In addition, unlike the display unit 241 illustrated in FIG. 18, the display unit 241 may be implemented as an independent display unit instead of a module included in the output unit 240.

The sound output unit 242 outputs audio data that can be recognized acoustically. The sound output unit 242 outputs an audio signal related to a function (eg, a path guide function) performed by the path guide device 200. The sound output unit 242 may include a receiver, a speaker, a buzzer, and the like.

The storage unit 250 may store a program for operating the route guide device 200, and may temporarily store data (path data and image data) input / output in relation to the route guide device 200. In addition, the storage unit 250 may store map data used for the route guidance service. The storage unit 250 may be embedded in or detachable from the path guide device 200, and may be a flash memory type, a hard disk type, or a multimedia card micro type. ), Card type memory (e.g. SD or XD memory), RAM (Random Access Memory, RAM), SRAM (Static Random Access Memory), ROM (ReadOnly Memory, ROM), EEPROM (Electrically Erasable Programmable ReadOnly Memory) It may include a storage medium of at least one type of PROM (Programmable ReadOnly Memory), magnetic memory, magnetic disk, optical disk. The route guide device 200 may operate in association with a web storage that performs a storage function of the storage unit 250 on the Internet.

The power supply unit 260 receives external power and internal power to supply power required for the operation of each component of the path guide device 200 or another device connected to the path guide device 200.

The controller 270 typically controls the overall operation of the route guidance device 200. In addition, the controller 270 may output a control signal for controlling another device connected to the route guidance device 200. The controller 270 may be configured with other modules (ie, the communication unit 210, the input unit 220, the sensing unit 230, the output unit 240, the storage unit 250, the power unit 260, etc.) and information described above. And / or an input / output unit (ie, an interface unit) for transmitting and receiving a control signal.

19 is a flowchart illustrating a method of driving the route guidance device 200 according to the present invention. Hereinafter, a driving method of the route guidance device 200 will be described with reference to necessary drawings.

When the route guidance function of the route guide device 200 is activated, the route guide device 200 displays route guide information on the display unit 241 (S800), and the route guide sound is output to the sound output unit 242. Output through

While the route guidance information is output through the display unit 241 and the sound output unit 242, a communication link with the front display device is formed through the communication unit 210. Then, when the front display function of the front display device is activated, the route guide device 200 receives a request for providing route information from the front display device through the communication unit 210 (S810). On the other hand, the front display device may be a front display-only terminal, as described above with reference to Figures 1 to 17 may be a mobile phone, a smart phone or the like equipped with a front display application.

Then, the route guidance apparatus 200 provides at least a portion of route guidance information to the front display apparatus (S820), and changes the display state of the route guidance information on the display unit 241 (S830). For example, as illustrated in FIG. 20, the route guidance apparatus 200 may block the route guidance information from being displayed on the display unit 241.

20 illustrates an example in which a display state of route guidance information is changed according to the driving method of the route guidance apparatus illustrated in FIG. 19. Referring to FIG. 20A, the route guidance apparatus 200 displays route guidance information on the display unit 241 and outputs route guidance sounds through the sound output unit 242 during the route guidance function. .

In the state of FIG. 20A, at least a part of the route guidance information is transmitted to the front display apparatus according to a request from the front display apparatus, and then the route guide apparatus 200 is illustrated in FIG. 20B. As described above, it can be seen that the route guidance information blocks itself from being displayed on the display unit 241. However, even at this time, as shown in FIG. 20B, the path guide apparatus 200 may output the path guide sound to the sound output unit 242.

In addition, as illustrated in FIG. 21, the route guide apparatus 200 may enlarge a display area of information corresponding to another function displayed on the display unit 241 along with route guide information.

FIG. 21 illustrates another example in which the display state of the route guide information is changed according to the driving method of the route guide apparatus shown in FIG. 19.

Referring to FIG. 21A, it can be seen that the display unit 241 displays the DMB broadcast related information 241A and the path guide function related information 241B together. In the state of FIG. 21A, when the route guide information is provided to the front display device, the route guide device 200 displays a display area of the DMB broadcast related information as shown in FIG. 21B. 241A 'may be enlarged and the area 241B' on which the route guidance function related information is displayed may be reduced.

In addition, as shown in FIG. 14, when the route guidance information is transmitted to the front display apparatus, the route guidance apparatus 200 may set the entire display unit 241 as a display area of the DMB broadcasting related information. .

As described above, each of the methods for performing the front display function of the mobile terminal 100 according to the present invention may be implemented in a program form that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program recorded on the medium may be those specially designed and constructed for the present invention or may be those known to those skilled in the computer software.

Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of programs include high-level language code that can be executed by a computer using an interpreter or the like, as well as machine code as produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

100: mobile terminal 110: wireless communication unit
120: A / V input unit 130: user input unit
140: sensing unit 150: output unit
151: display 152: sound output unit
160: storage unit 170: interface unit
180: control unit 190: power unit
200: route guidance device 210: communication unit
220: input unit 230: sensing unit
240: output unit 241: display unit
242: sound output unit 250: storage unit
260: power supply unit 270: control unit
300: image acquisition device 400: service server

Claims (18)

A mobile terminal mounted in a vehicle and performing a head up display (HUD) function and a black box function,
camera;
A display unit;
A communication unit for establishing a communication link with the route guidance device and receiving route guidance information from the route guidance device; And
According to the standing angle of the mobile terminal, performing the front display device function to process the received route guidance information into a front display signal to display on the display unit or to perform a black box function for storing the image taken by the camera A mobile terminal comprising a control unit. The apparatus of claim 1,
And a standing angle at which the mobile terminal performs the front display device function or a standing angle at which the mobile terminal performs the black box function in response to a signal received through an input means. The apparatus of claim 1,
When the mobile terminal performs the front display device function, only predetermined route guide information among the route guide information received from the route guide device is selectively processed into a front display signal and displayed on the display unit. Mobile terminal. The apparatus of claim 1,
And performing the front display device function, controlling the route guide device to change the display state of the route guide information. The method of claim 1, wherein the mobile terminal,
When the mobile terminal performs the black box function, the mobile terminal further comprises a storage unit for storing the image taken by the camera. The apparatus of claim 1,
When the mobile terminal performs the black box function, the mobile terminal, characterized in that for transmitting the image taken by the camera to the route guide device through the communication unit. The apparatus of claim 1,
When the mobile terminal performs the black box function, the mobile terminal, characterized in that for transmitting the image taken by the camera to the black box service server through the communication unit. A mobile terminal mounted on a vehicle and performing a front display device function,
A display unit;
A communication unit configured to receive an external vehicle image from an image acquisition device; And
A controller configured to acquire augmented reality information corresponding to a subject of the external vehicle image and to process the augmented reality image matching the augmented reality information to the subject of the external vehicle image as image information for front display and display the image on the display unit; Mobile terminal comprising. 9. The mobile terminal of claim 8,
Further comprising a storage for storing the augmented reality information,
The control unit,
And augmented reality information corresponding to a subject of the vehicle external image among the augmented reality information stored in the storage unit. The method of claim 8, wherein the control unit,
And transmitting the received vehicle exterior image to the augmented reality service server through the communication unit to request retrieval of augmented reality information corresponding to the subject of the received vehicle exterior image. In the method of driving a mobile terminal mounted on a vehicle and capable of performing a head up display (HUD) function,
Establishing a communication link with the route guidance device;
Sensing a standing angle of the mobile terminal; And
According to the standing angle of the mobile terminal performs the front display function to receive the route guidance information from the route guidance device to process the signal for front display and display on the display unit;
A method of driving a mobile terminal, the method comprising: performing a black box function of storing an image photographed by a camera. A recording medium storing a computer program for executing the mobile terminal driving method according to claim 11. A mobile terminal driving method mounted on a vehicle and performing a front display device function,
Establishing a communication link with an image acquisition device and receiving an external vehicle image from the image acquisition device;
Acquiring augmented reality information corresponding to a subject of the received external vehicle image; And
And processing the augmented reality image obtained by matching the augmented reality information to the subject of the external image of the vehicle and converting the augmented reality image into image information for front display and displaying the same on the display unit. A recording medium storing a computer program for executing the mobile terminal driving method according to claim 13. A communication unit forming a communication link with the front display device;
A display unit displaying route guidance information; And
And a controller configured to provide at least a part of the route guide information to the front display device when the front display device is activated, and to change a display state of the route guide information on the display unit. 16. The apparatus of claim 15,
And when at least a portion of the route guidance information is provided to the front display device, outputting the route guidance information through the display. The method of claim 16, wherein the route guidance device,
Further comprising a sound output unit for outputting a path guide sound corresponding to the path guide information,
The control unit,
And the output state of the route guidance sound is maintained through the sound output unit even when the front display device is activated to block the output of the route guidance information. 16. The apparatus of claim 15,
When the front display device is activated while a first event, which is a route guidance service provided using the route guidance information, and a second event, which is a service different from the first event, are simultaneously provided through the display unit, And a display area of the information related to the second event is enlarged than when the front display device is not activated.

Images