JP2014002719A - Remote control device, display device and method for controlling the same - Google Patents

Abstract

Intuitive control of a UI screen provided from each mode of a display device operable in a plurality of operation modes.
A remote control device 200 that provides a remote control signal to an external display device having a plurality of operation modes includes a signal output unit 230 that outputs a remote control signal to the external display device that provides a UI screen, and a remote control device. A detection unit 210 for detecting the operation of the first display unit, a first button unit used for receiving a user command for selecting one of the pointing mode and the gesture mode in the operation mode of the external display device, and a UI screen An input unit 220 including a second button unit used to receive a user command for selecting an object displayed on the display unit; and when a user command is input through the first button unit, a pointing mode and And a control unit 240 that selects and controls any one of the gesture modes.
[Selection] Figure 6

Description

  The present invention relates to a remote control device, a display device, and a control method thereof, and more particularly to a remote control device that detects an operation, a display device, and a control method thereof.

  With the development of electronic technology, measures for controlling electronic devices in various ways have been developed. Conventionally, an electronic device has been controlled using a button provided in the electronic device or a remote controller that is a separate device from the electronic device.

  However, when controlling an electronic device using a remote controller that is a separate device from the electronic device as in the past, the user must check the buttons provided on the remote controller one by one for the desired operation. There was a problem that the operation of pressing had to be performed.

  For example, when using a pointer displayed on the screen in order to select specific content on the screen of the electronic device, the user repeatedly selects a four-way button on the remote controller several times and selects the pointer in the content area. And the specific content can be selected by pressing the selection button provided on the remote controller on the content area. That is, the user has to confirm the button provided on the remote controller many times and select the content through an operation of pressing several times.

  Accordingly, there is a need for a method for a user to search information displayed on the screen of the electronic device more conveniently from a long distance.

Korean Published Patent No. 2010-0000734

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to provide a user interface screen (hereinafter referred to as a UI screen) provided from each mode of a display device operable in a plurality of operation modes. To provide a remote control device, a display device, and a control method thereof.

  A remote control device for providing a remote control signal to an external display device having a plurality of operation modes including a pointing mode and a gesture mode according to an embodiment of the present invention provides the remote control signal to the external display device that provides a UI screen. A signal output unit that outputs a signal, a detection unit that detects an operation of the remote control device, and a user command for selecting one of the pointing mode and the gesture mode in the operation mode of the external display device An input unit including a first button unit used for receiving and a second button unit used for receiving a user command for selecting an object displayed on the UI screen; and When a user command is input via the button unit, the pointing mode is And a control unit that controls the signal output unit to output information related to a user command for selecting one of the gesture modes and information related to the detected motion, wherein the pointing mode includes the detection A mode for controlling an operation of an object displayed on the UI screen according to an operation of the remote control device detected by a unit, wherein the gesture mode is the UI according to a preset gesture of the remote control device. It may be a mode for controlling the display state of the screen.

  Note that the external display device sets the pointing mode as a default mode during a time period in which information on the user command for selecting one of the pointing mode and the gesture mode is not received from the remote control device. The external display device is in a non-default mode during a time period during which information about the user command to hold and select one of the pointing mode and the gesture mode is continuously received from the remote control device It is possible to switch to the gesture mode and hold the switched mode.

  The controller continuously outputs information on a user command for selecting one of the pointing mode and the gesture mode while the first button is pressed, and the first button When the pressing operation on the unit is released, the signal output unit may be controlled so as not to output information on the user command for selecting one of the pointing mode and the gesture mode.

  The detection unit may include at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor.

  A display device controlled by a remote control device for providing a remote control signal according to an embodiment of the present invention and having a plurality of operation modes including a pointing mode and a gesture mode is provided from the remote control device to the remote control device. A receiving unit for receiving a remote control signal including information relating to an operation of the user, information relating to a user command for selecting one of the pointing mode and the gesture mode, and information relating to item selection, and a pointing object performing an indicator function And a display unit that displays a UI screen having a display screen and a user command received from the remote control device for selecting one of the pointing mode and the gesture mode. The pointing mode is displayed on the UI screen according to the operation of the remote control device detected by the detection unit. This is a mode for controlling the movement of the pointer object, and the gesture mode may be a mode for controlling the display state of the UI screen according to a predefined gesture of the remote control device.

  The display unit may display a pointing object in a form changed according to the operation mode.

  The control unit changes the operation mode of the display device to the pointing mode while the receiving unit does not receive the information related to the user command for selecting either the pointing mode or the gesture mode. The operation mode of the display apparatus is determined to be the gesture mode while receiving information related to the user command for determining and selecting one of the pointing mode and the gesture mode, and the operation mode of the display apparatus is When in the pointing mode, the menu item included in the UI screen may be selected based on information regarding the operation of the remote control device and information regarding item selection received from the remote control device.

  The control unit may change and display at least one of the shape, hue, size, position, and direction of the pointing object according to an operation state that can be input on the user interface screen. You may control it.

  The pointing object is a hand-shaped pointer, and the control unit performs a flicking operation, a zoom in / out operation, and a panning operation on the user interface screen. Based on an operable form, the hand-shaped pointer is controlled to be displayed in a pointing form, a palm form, or a grip form. Good.

  A display system including a remote control device for providing a remote control signal according to an embodiment of the present invention and a display device having a plurality of operation modes including a pointing mode and a gesture mode controlled by the remote control device, Including a remote control device and a display device for outputting a remote control signal including information relating to operation of the remote control device, information relating to a user command for selecting one of the pointing mode and the gesture mode, and information relating to item selection The display device is configured to select, from the remote control device, information related to operation of the remote control device, information related to a user command for selecting one of the pointing mode and the gesture mode, and item selection. A receiving unit for receiving a remote control signal including information related to the display; a display unit for displaying a UI screen having a pointing object for performing an indicator function on the UI screen; the pointing mode received from the remote control device; A control unit that controls selection of either the pointing mode or the gesture mode based on information on a user command for selecting one of the gesture modes, and the pointing mode is controlled by the detection unit A mode for controlling an operation of the pointer object displayed on the UI screen according to the detected operation of the remote control device, wherein the gesture mode is a predetermined gesture of the remote control device; It may be a mode for controlling the display state of the UI screen I.

  Note that the display device holds the pointing mode as a default mode while information on the user command for selecting one of the pointing mode and the gesture mode is not received from the remote control device. And switching to the gesture mode as a non-default mode and holding the switched mode while continuously receiving information on the user command for selecting one of the pointing mode and the gesture mode It may be constituted as follows.

  The remote control device includes a signal output unit that outputs the remote control signal to the display device, and a first button unit that receives a user command for selecting one of the pointing mode and the gesture mode. While the input button provided and the first button unit are pressed, information on the user command for selecting one of the pointing mode and the gesture mode is output, and the first button unit And a control unit that controls the signal output unit so as not to output information on the user command for selecting either the pointing mode or the gesture mode when the pressing operation is released.

  Meanwhile, a remote control device control method for providing a remote control signal to an external display device having a plurality of operation modes including a pointing mode and a gesture mode according to an embodiment of the present invention detects an operation of the remote control device. Step, and when a user command for selecting one of the pointing mode and the gesture mode is input in the operation mode of the remote control device, information on the detected operation, the pointing mode, and the gesture Outputting information related to a user command for selecting one of the modes to the external display device, wherein the pointing mode is displayed on the UI screen according to the operation of the remote control device detected by the detection unit. To this Is a mode for controlling the operation of the ray pointers object, the gesture mode may be a mode for controlling the display state of the UI screen according to a preset gesture of the remote control device.

  In addition, the control method of the display device controlled by the remote control device that provides the remote control signal according to the embodiment of the present invention and having a plurality of operation modes including the pointing mode and the gesture mode is a pointing that performs an indicator function. A remote control signal including a step of displaying a UI screen having an object; information relating to operation of the remote control device; and information relating to a user command for selecting one of the pointing mode and the gesture mode. Receiving from the control device, and based on the information regarding the user command for selecting one of the pointing mode and the gesture mode received from the remote control device, the pointing mode Preliminary may include the steps of one of the choice of the gesture mode.

  As described above, according to the present invention, an intuitive remote control method that provides an interface experience similar to a touch method can be provided.

It is a schematic diagram which shows the structure of the display system which concerns on one Embodiment of this invention. It is a figure for demonstrating the operation | movement which concerns on one Embodiment of the display system shown in FIG. It is a figure for demonstrating the operation | movement which concerns on one Embodiment of the display system shown in FIG. It is a block diagram which shows the structure of the display apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the display apparatus which concerns on one Embodiment of this invention. It is a figure for demonstrating the detailed structure of the control part which concerns on one Embodiment of this invention. It is a figure which shows the software structure of the preservation | save part for supporting operation | movement of the control part which concerns on the above-mentioned various embodiment. It is a block diagram which shows the structure of the remote control apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the detailed structure of the remote control apparatus shown in FIG. It is a block diagram for demonstrating the form of the signal processing algorithm which concerns on one Embodiment of this invention. It is a figure which shows the external appearance of the remote control apparatus shown in FIG. 6 and FIG. It is a figure which shows the example provided with two or more mode switching button parts of a remote control device. It is a figure for demonstrating the operation form of the remote control apparatus which concerns on various embodiment of this invention. It is a figure for demonstrating the operation state in the pointing mode which concerns on one Embodiment of this invention. It is a figure for demonstrating the operation state which concerns on the mode switching which concerns on one Embodiment of this invention. It is a figure for demonstrating the operation state in gesture mode which concerns on one Embodiment of this invention. It is a figure for demonstrating the operation state in the gesture mode which concerns on another embodiment of this invention. It is a figure for demonstrating the operation state in the gesture mode which concerns on another embodiment of this invention. It is a flowchart for demonstrating the control method of the remote control apparatus which concerns on one Embodiment of this invention. It is a flowchart for demonstrating the control method of the remote control apparatus which concerns on one Embodiment of this invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a schematic diagram showing a configuration of a display system according to an embodiment of the present invention.

  As shown in FIG. 1, a display system according to an embodiment of the present invention includes a display device 100 and a remote control device 200.

  The display device 100 can be realized by a digital television as a device controlled by the remote control device 200, but can be applied to any device capable of remote control such as a personal computer.

  As a device for remotely controlling the display apparatus 100, the remote control apparatus 200 can receive a user command and transmit a signal corresponding to the input user command to the display apparatus 100.

  In particular, the remote control device 200 can detect the operation of the remote control device 200 in the XYZ space and transmit a signal related to the detected three-dimensional operation to the display device 100. Here, the three-dimensional operation may correspond to a command for controlling the display apparatus 100. That is. The user can transmit a predetermined command to the display device 100 by operating the remote control device 200 in space.

  In this case, the remote control device 200 can transmit the operation information itself corresponding to the detected operation (first embodiment). However, in some cases, the remote control device 200 displays the operation information corresponding to the detected operation. It is also possible to transmit it by converting it into a control signal for controlling (second embodiment). This may be a problem regarding whether the calculation for calculating the control signal from the detected operation information is performed in the display apparatus 100 or the remote control apparatus 200, but a detailed description thereof will be described in the embodiment. Will be described later. Meanwhile, the display apparatus 100 can provide a plurality of different operation modes.

  In detail, the display apparatus 100 is configured to control a pointing mode displayed on a UI screen provided through the display apparatus 100 according to the operation of the remote control apparatus 200 and the remote control apparatus 200 defined in advance. It is possible to operate in a gesture mode for controlling the display state of the UI screen according to the gesture. Here, the display state of the UI screen controlled in the gesture mode is changed by screen switching such as screen switching for executing a specific function such as menu screen display or page switching for changing the content content displayed on the screen. As described above, the screen display state is changed by the occurrence of a preset event, and it may be simply distinguished from the pointing mode for controlling the moving state of the pointing object. On the other hand, the pointing mode may be provided based on a global coordinate system in which the operation area of the remote control device 200 and the screen of the display device 100 match.

  Specifically, the operation area of the remote control device 200 corresponding to the entire area of the display device 100 screen may be set in advance. For example, in a preset reference state, an operation region between 30 degrees rotated to the left and 30 degrees rotated to the right may correspond to the entire display screen. Here, the preset reference state may be set according to a user command. For example, when the power source of the remote control device 200 is turned on and a user command for setting a reference state is input (for example, a centering button (not shown) provided in the remote control device 200 is input), the remote control device The current state of 200 is set as the reference state, and a pointing object corresponding to the current state of the remote control device 200 is set in a predetermined area of the screen of the display device 100 (for example, the center of the screen). Area).

  Next, the coordinate value of the pointing object corresponding to the rotation state of the remote control device 200 in which the user moves may be calculated around the preset absolute reference point. For example, the X and Y coordinate values corresponding to the rotation state of the remote control device 200 that the user moves with the center of the display screen as the absolute reference point may be calculated.

  For example, if the remote control device 200 is rotated 10 degrees to the right in the space, the pointing object corresponding to the rotation state of the remote control device 200 matches the (x1, y1) point (for example, the center point). To (x2, y2).

  However, in some cases, a relative coordinate system may be applied, and the relative coordinate system is a technique that has already been publicly known, and thus detailed description thereof is omitted.

  The remote control device 200 may include a mode switching button that receives a user command for switching the operation mode of the display device 100. In this case, the mode switching button may be realized to hold the switched mode when the pressed state is held. Detailed description thereof will be described later with reference to the drawings.

  On the other hand, the display device 100 and the remote control device 200 include BT (BlueTooth (registered trademark)), Zigbee (registered trademark), Wi-Fi (Wireless Fidelity), IR (Infrared), Serial Interface, USB (Universal Serial Bus), and the like. It is possible to communicate through various communication methods. For example, when the display apparatus 100 and the remote control apparatus 200 communicate through BT, they may be linked to each other through Bluetooth sparing. Detailed techniques related to Bluetooth sparing are obvious to those skilled in the art, and thus detailed description thereof is omitted.

  Hereinafter, for convenience of explanation, the operations according to the first and second embodiments described above will be described separately.

[In the case of the first embodiment]
FIG. 2A is a diagram for explaining an operation according to the first embodiment of the display system shown in FIG.

  As shown in FIG. 2A, the remote control device 200 receives a mode switching command (S11). For example, if the display device 100 operates in the pointing mode, a mode switching command for switching to the gesture mode may be received. If the display device 100 operates in the gesture mode, a mode switching command for switching to the pointing mode is received. May be received. The display apparatus 100 can provide a UI screen including a pointing object. The pointing object displayed on the UI screen in the pointing mode may have a function of pointing at least one content, and the pointer moves on the UI screen according to the rotation direction and the degree of rotation of the remote control device 200. become. For example, when the pointing object is a hand-shaped pointer, the pointing object may be a hand-shaped pointer. On the other hand, in the pointing mode, the display position of the pointing object displayed on the screen can be moved using the absolute coordinate method as described above. In the gesture mode, the display apparatus 100 may change the pointing-type hand-shaped pointer to a palm-shaped hand-shaped pointer for display. The display apparatus 100 may display a guide GUI (Graphic User Interface) for guiding gesture input on the screen together with a palm-shaped hand-shaped pointer. Here, the guide GUI may take another form depending on the type of the UI screen. For example, the guide GUI may be in the form of an arrow that guides a gesture direction that can be applied on the current UI screen.

  Next, when the remote control device 200 detects an operation (S12), the remote control device 200 generates operation information corresponding to the detected operation (S13). For example, motion information corresponding to “rotate 5 degrees to the right” can be generated.

  Next, the remote control device 200 transmits the information regarding the operation mode of the display device related to the mode switching command received at step S11 and the operation information generated at step S13 to the display device 100 (S14). Here, the information regarding the operation mode may be information indicating a pointing mode or a gesture mode.

  In this case, the display apparatus 100 generates a control signal for controlling the operation of the display apparatus 100 based on the received information regarding the operation mode and the operation information (S15).

  Specifically, when the received mode information is pointing mode information, the received operation information is generated by converting the received operation information into a control signal corresponding to the pointing mode, and the received information regarding the operation mode is the gesture mode The received operation information may be converted into a control signal corresponding to the gesture mode. For example, when the received information about the operation mode indicates the pointing mode and the received operation information is information that rotates 10 degrees to the right, the pointing object displayed on the screen is moved by the corresponding coordinates. A control signal may be generated. Alternatively, the received information regarding the operation mode indicates the gesture mode, the received operation information is information that rotates 10 degrees to the right, and a screen switching command is mapped to the gesture corresponding to the operation information. In this case, a control signal for switching the UI screen page displayed on the screen to the next UI screen page for display may be generated.

  Next, the display apparatus 100 performs an operation corresponding to the generated control signal (S16).

  FIG. 2B is a diagram for explaining an operation according to the second embodiment of the display system shown in FIG. 1.

  Of the steps shown in FIG. 2B, steps S21 to S23 are the same as those shown in FIG.

  Next, the remote control device 200 generates a control signal based on the mode information related to the mode switching command received in step S21 and the operation information generated in step S23 (S24). That is, if the received mode information is pointing mode information, the remote control device 200 generates the received operation information by converting the received operation information into a control signal corresponding to the pointing mode, and the received mode information is the gesture mode information. In this case, the received motion information can be generated by converting it into a control signal corresponding to the gesture mode. That is, the remote control device 200 may be realized in a form having an arithmetic function for generating a control signal based on the received mode information and operation information.

  Next, the remote control device 200 transmits the generated control signal to the display device 100 (S25).

  In this case, the display apparatus 100 performs an operation corresponding to the received control signal (S26). That is, unlike the embodiment shown in FIG. 2A, the display device 100 can perform an operation without performing a special calculation operation.

  For example, when a control signal corresponding to an operation for changing the UI page currently displayed to the next UI page is received, an operation corresponding to the received control signal may be performed.

  3A and 3B are block diagrams showing a configuration of a display device according to an embodiment of the present invention.

  Referring to FIG. 3A, the display apparatus 100 includes a receiving unit 110, a display unit 120, and a control unit 130. Hereinafter, the operation of the display apparatus 100 will be described by dividing it into the first embodiment and the second embodiment described above.

[Operation of Display Device 100 according to First Embodiment]
The display apparatus 100 can be realized by a digital television, a personal computer, a notebook personal computer, or the like, but is not limited thereto, and may be applied without limitation as long as the device has a display function and can be controlled remotely.

  The receiving unit 110 can communicate with the remote control device 200. Specifically, the receiving unit 110 includes various types such as BT (BlueTooth (registered trademark)), Zigbee (registered trademark), Wi-Fi (Wireless Fidelity), IR (Infrared), Serial Interface, and USB (Universal Serial Bus). Communication with the remote control device 200 can be performed through the communication method.

  Specifically, the receiving unit 110 can receive a remote control signal including information regarding the operation mode and operation information of the remote control device 100 from the remote control device 200. Here, the information on the operation mode may be realized in various forms as information on a user command for selecting either the pointing mode or the gesture mode. For example, the mode information may be realized in the form of a flag, but in the pointing mode, the flag may indicate a “0” value, and in the gesture mode, the flag may indicate a “1” value. That is, the remote control device 200 can generate flag information indicating a “0” value or a “1” value in accordance with a user command for mode switching. In this case, a user command for mode selection may be input via a first button unit (not shown) provided in the remote control device 200.

  On the other hand, the information regarding the operation mode may be transmitted every time the operation information is transmitted, but may be transmitted only when there is a mode switching command in some cases. In the latter case, the display apparatus 100 receives the operation information received after the information about the operation mode is transmitted until the information about the switched operation mode is retransmitted. Recognize as information.

  The receiving unit 110 can also receive information regarding item selection from the remote control device 200. For example, in the pointing mode, a selection signal for selecting an item where the pointing object is located may be received. In this case, the selection signal is transmitted via a second button unit (not shown) provided in the remote control device 200. May be entered.

  In some cases, the receiving unit 110 is realized to perform a transmission function, and can transmit information to the remote control device 200. For example, when the display apparatus 100 is powered off, a power off signal may be transmitted to the remote control apparatus 200 so that the remote control apparatus 200 is automatically powered off.

  The display unit 120 can provide various display screens that can be provided through the display apparatus 100.

  In particular, the display unit 120 can display a UI screen having a pointing object that performs an indicator function.

  The display unit 120 can display a GUI corresponding to the operation mode of the display apparatus 100 according to the control of the control unit 130.

  Specifically, the display unit 120 can display a pointing object such as a cursor, a mouse cursor, or a highlight display on a UI screen displayed in the pointing mode. In addition, the display unit 120 can display the moving position of the pointing object according to the operation information received from the remote control device 200 in the pointing mode.

  The display unit 120 can display a pointing object in a form changed according to the operation mode.

  The display unit 120 can display a guide GUI for guiding gesture input on the displayed UI screen when switching from the pointing mode to the gesture mode. Here, the guide GUI may be an arrow-shaped cue indicating the gesture direction, but is not limited thereto.

  Here, the display unit 120 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a flexible display. It may be realized by various displays such as (3D Display).

  The control unit 130 has a function of controlling the overall operation of the display apparatus 100.

  The control unit 130 can control the operation mode selection based on the information regarding the operation mode received from the remote control device 200.

  Specifically, the control unit 130 can control selection of either the pointing mode or the gesture mode based on information related to a user command for selecting either the pointing mode or the gesture mode.

  Specifically, the control unit 130 determines the operation mode of the display apparatus 100 as the pointing mode while not receiving the information related to the user command for selecting one of the pointing mode and the gesture mode. While the information regarding the user command for selecting any one of the gesture modes is received, the operation mode of the display apparatus 100 can be determined as the gesture mode.

  The control unit 130 may convert the operation information received from the remote control device 200 into a control signal for controlling the display device 100, and control the operation of the display device 100 according to the converted control signal. it can.

  Specifically, the control unit 130 converts the received operation information into a control signal form applied to the function of the display apparatus 100 using the control signal mapping table, and uses the converted control signal to display the display apparatus 100. Can control the function. In particular, the control unit 130 may control the display unit 120 so as to provide a corresponding form of GUI according to the operation mode of the display apparatus 100.

  For example, when a UI screen is displayed on the display unit 120, the display apparatus 100 can basically operate in the pointing mode and display a corresponding GUI. When operating in the gesture mode according to the received mode information, the display unit 120 can be controlled to display the corresponding GUI. Here, the GUI may be realized in the form of the above-described pointer, cursor, mouse cursor, or the like as a pointing object that performs various indicator functions on the UI screen.

  When the operation mode of the display device 100 is the pointing mode, the menu item included in the UI screen is selected based on the information regarding the operation of the remote control device 200 and the information regarding the item selection received from the remote control device 100. It's okay.

  On the other hand, when the mode information corresponding to the gesture mode is received from the remote control device 200, the control unit 130 can change and display the pointing object displayed on the screen. For example, when the pointing object is a hand-shaped pointer, the pointing mode displays a pointing shape with only the index finger open, and the gesture mode displays a palm shape with all fingers open. The shape of the hand can be displayed.

  The control unit 130 changes at least one of the shape, hue, size, position, and direction of the pointing object according to an operation state that can be input on the UI screen provided from the gesture mode. Can be displayed. For example, if the pointing object is a hand-shaped pointer and the screen can be zoomed in / out in gesture mode, the size of the hand-shaped pointer is enlarged and displayed in the zoomed-in state. In the out state, the size of the hand-shaped pointer can be reduced and displayed.

  In addition, the control unit 130 can control the display unit 120 to display a guide GUI having a direction in which a gesture input through the remote control device 200 guides in the gesture mode. For example, when a left / right flicking operation is possible in the gesture mode, a guide GUI in the left / right arrow direction can be displayed.

  Note that the control unit 130 is at least one of the moving direction and the moving degree of the pointing object displayed on the UI screen according to at least one of the rotating direction and the rotating degree of the remote control device 200 in the pointing mode. The display unit 120 can be controlled so that one of them is changed and displayed. In this case, as described above, the absolute coordinate system can be used.

  The control unit 130 systematically matches the UI screen in the XY axis direction when a gesture for rotating the remote control device 200 on the XY axis parallel to the screen of the display unit 120 is input in the gesture mode. The display unit 120 can be controlled so as to change and display the other UI screen.

  When a gesture for rotating the remote control device 200 on the Z axis perpendicular to the screen of the display unit 120 is input in the gesture mode, the control unit 130 displays a UI screen according to the rotation state of the remote control device 200. The content selected above may be controlled to be zoomed in or zoomed out for display.

[Operation of Display Device According to Second Embodiment]
Of the operation of the display device according to the second embodiment, detailed description of the same configuration as that of the display device according to the first embodiment will be omitted.

  The receiving unit 110 can communicate with the remote control device 200. Specifically, the receiving unit 110 includes various types such as BT (BlueTooth (registered trademark)), Zigbee (registered trademark), Wi-Fi (Wireless Fidelity), IR (Infrared), Serial Interface, and USB (Universal Serial Bus). Communication with the remote control device 200 can be performed through the communication method.

  Specifically, the receiving unit 110 can receive a control signal generated based on the mode information input via the remote control device 200 and the operation information of the remote control device 200. That is, unlike the first embodiment, the receiving unit 110 may receive a control signal from the remote control device 200 having a form capable of controlling the display device 100 without any special calculation.

  The display unit 120 can provide various display screens that can be provided via the display apparatus 100. The content displayed on the display unit 120 is the same as that of the first embodiment, and thus detailed description thereof is omitted. To do.

  The control unit 130 can control the operation of the display device 100 based on the control signal received from the remote control device 200. Specifically, the control unit 130 may perform an operation of controlling the movement state of the pointing object according to the control signal received in the pointing mode, and may perform a screen switching operation corresponding to the control signal received in the gesture mode. . Since the control operation of the control unit 130 for the display unit 120 is the same as that of the first embodiment, detailed description thereof is omitted.

  3B is a block diagram illustrating a detailed configuration of the display device illustrated in FIG. 3A.

  In the following, for convenience of explanation, the explanation will be focused on the detailed configuration of the display device corresponding to the first embodiment. In the case of the second embodiment, the display device operates in response to a control signal transmitted from the remote control device, and thus detailed description thereof is omitted.

  As illustrated in FIG. 3B, the display apparatus 1000 includes a receiving unit 110, a display unit 120, a control unit 130, a storage unit 140, a video receiving unit 150, a video processing unit 160, a user interface unit 170, and a UI. A processing unit 180 is included. In FIG. 3B, description will be given assuming that the display apparatus 100 shown in FIG. 3A is realized by a digital television. Note that, in the configuration illustrated in FIG. 3B, detailed description of portions that overlap with the components illustrated in FIG. 3A is omitted.

  The storage unit 140 can be realized by a memory, an HDD (Hard Disk Drive), or the like as a storage medium for storing various data and programs necessary for operating the display apparatus 1000.

  In particular, the storage unit 140 can store control information corresponding to operation information received from the remote control device 200 in the gesture mode. Specifically, the storage unit 140 may store the control signal corresponding to the operation information received from the remote control device 200 in the gesture mode in the form of a table. For example, for operation information corresponding to a gesture received from the remote control device 200 that flicks from left to right (for example, an operation to rotate the remote control device 200 to the left), a UI screen switching command is supported. The control information corresponding to the content zoom-in / out command is saved by matching the control information corresponding to the content zoom-in / out command with respect to the motion information corresponding to the back-and-forth movement gesture (for example, the motion of rotating the remote control device 200 back and forth) be able to.

  The video receiving unit 150 has a function of receiving broadcast content through an antenna, a cable, or the like, or receiving video content from an external device or an external communication network.

  Specifically, the image receiving unit 150 can receive various image contents through a network or an airwave. Here, the content may be various types of content such as already produced content such as VOD content and broadcast content.

  In this case, the video receiving unit 150 can be realized in various forms. For example, when the display apparatus 100 is realized by a digital television, the video reception unit 110 may be realized by a set-top box, a tuner, an external input port, a network communication module, or the like.

  The video processing unit 160 has a function of performing signal processing on various signals received via the video receiving unit 150. Accordingly, the video processing unit 160 may include signal processing elements such as a demo duration, a decoder, an A / D converter, and a scaler.

  The user interface unit 170 has a function of receiving various user signals.

  Here, the user interface unit 170 can be realized in various forms according to the device type of the display apparatus 100. For example, when the display device 100 is realized by a digital television, the user interface unit 170 may be realized by the above-described receiving unit 110 that receives a remote control signal.

  The user interface unit 170 may be realized in a touch screen form in which an input panel or display including a plurality of keys and a touch pad form a mutual layer structure.

  The UI processing unit 180 has a function of generating various UI elements that are displayed in an overlapping manner with the video output to the display unit 120 under the control of the control unit 130. Here, the UI processing unit 180 can generate a 2D or 3D UI element.

  Note that the UI processing unit 180 may perform operations such as 2D / 3D switching of UI elements, transparency, hue, size, form and position adjustment, highlighting, and animation effects under the control of the control unit 130.

  FIG. 4 is a diagram for explaining a detailed configuration of the control unit 130 according to an embodiment of the present invention.

  As shown in FIG. 4, the control unit 130 includes a system memory 131, a main CPU 132, an image processing unit 133, a network interface 134, a storage unit interface 135, and first to nth interfaces 136-1 to 136-1. 136-n, an audio processing unit 137, and a system bus 138 are included.

  The system memory 131, the main CPU 132, the image processing unit 133, the network interface 134, the storage unit interface 135, the first to n-th interfaces 136-1 to 136-n and the audio processing unit 137 are connected via the system bus 138. They are connected to each other and can transmit and receive various data and signals.

  The first to nth interfaces 136-1 to 136-n support interfacing between various components including the display unit 120 and each component in the control unit 130. Note that at least one of the first to n-th interfaces 136-1 to 136-n is connected through a button provided on the main body portion of the display apparatus 100 or the external input port 1 to the external input port n. Alternatively, it may be realized by an input interface that receives various signals from an external device.

  The system memory 151 includes a ROM 131-1 and a RAM 131-2. The ROM 131-1 stores a command word set for system boot. When a turn-on command is input and power is supplied, the main CPU 132 copies an O / S (Operating System) stored in the storage unit 140 to the RAM 131-2 according to the command stored in the ROM 131-1, and stores the O / S. Run S to boot the system. When the boot is completed, the main CPU 132 performs various operations by copying various application programs stored in the storage unit 140 to the RAM 131-2 and executing the application programs copied to the RAM 131-2.

  As described above, the main CPU 132 can perform various operations by executing the application program stored in the storage unit 140.

  The storage unit interface 135 is connected to the storage unit 140 and transmits / receives various programs, contents, data, and the like.

  For example, when a control signal is received from the remote control device 200, the main CPU 132 accesses the storage unit 140 via the storage unit interface 135, reads the control information mapped to the control signal, and then reads the control information. The operation of the display device 100 can be controlled using the information.

  The image processing unit 133 may include a decoder, a renderer, a scaler, and the like. Accordingly, the stored content is decoded, the decoded content data is rendered to form a frame, and the size of the configured frame is scaled according to the screen size of the display unit 120. The image processing unit 133 provides the processed frame to the display unit 120 for display.

  In addition, the audio processing unit 137 refers to a component that processes audio data and provides it with sound output means such as a speaker (not shown). The audio processing unit 137 can perform audio signal processing such as decoding audio data stored in the storage unit 140 or audio data received from the outside, performing noise filtering, and then amplifying the data to an appropriate decibel. In the above example, when the content to be played back is video content, the audio processing unit 137 can process the audio data demultiplexed from the video content and output it in synchronization with the image processing unit 133. Can be provided to a speaker (not shown).

  The network interface 134 is a part connected to an external device through a network. If the application that provides the information providing service according to the present invention is driven, the main CPU 132 can communicate with the remote control device 200 via the network interface 134.

  On the other hand, the operation of the control unit 130 described above may be realized by executing various programs stored in the storage unit 140.

  FIG. 5 is a diagram illustrating a software structure of the storage unit 140 for supporting the operation of the control unit 130 according to various embodiments described above. As illustrated in FIG. 5, the storage unit 140 includes a base module 510, a device management module 520, a communication module 530, a presentation module 540, a web browser module 550, and a service module 560.

  The base module 510 means a basic module that processes a signal transmitted from each hardware included in the display apparatus 100 and transmits the signal to an upper layer module.

  The base module 510 includes a storage module 511, a location infrastructure module 512, a security module 513, a network module 514, and the like.

  The storage module 511 is a program module that manages a database (DB) and a registry. The location infrastructure module 512 is a program module that supports location infrastructure services in conjunction with hardware such as a GPS chip. The security module 153 is a program module that supports hardware authentication (certification), request permission (permission), security storage (secure storage), and the like, and the network module 514 is a module for supporting network connection. , DNET module, UpnP module and the like. For example, a network connection with an external server (not shown) or the like can be performed via the network module 514.

  The device management module 520 is a module for managing information on external inputs and external devices and using the information. The device management module 520 may include a sensing module 521, a device information management module 522, a remote control module 523, and the like. For example, information on the remote control device 200 or the like can be managed via the device management module 520.

  The sensing module 521 is a module that analyzes various sensor data. For example, the sensing module 521 may include a face recognition module, a voice recognition module, a motion recognition module, an NFC recognition module, and the like.

  The device information management module 522 is a module that provides information for various devices, and the remote control module 523 is a program module that performs an operation of remotely controlling peripheral devices such as a telephone, a printer, a camera, and an air conditioner.

  The communication module 530 is a module for performing communication with the outside. The communication module 530 includes a messenger program, an SMS (Short Message Service) & MMS (Multimedia Message Service) program, a messaging module 531 such as an e-mail program, a telephone information aggregator (Call Info Aggregator) program module, a VoIP module, and the like. A telephone module 532 may be included.

  The presentation module 540 is a module for configuring a display screen. The presentation module 540 includes a multimedia module 541 for reproducing and outputting multimedia contents, and a UI & graphic module 542 for performing UI and graphic processing. The multimedia module 541 may include a player module, a camcorder module, a sound processing module, and the like. Thereby, various multimedia contents are reproduced to generate and reproduce a screen and sound. The UI & graphic module 542 includes an image composition module 542-1 that combines images, a coordinate combination module 542-2 that generates a combination of coordinates on a screen on which an image is displayed, and X11 that receives various events from hardware. Modules 542-3, 2D / 3DUI tool kits 542-4, etc. that provide tools for configuring UIs in 2D or 3D form may be included. For example, a UI screen corresponding to the pointing mode or the gesture mode can be configured via the presentation module 540.

  The web browser module 550 refers to a module that performs web browsing and accesses a web server. The web browser module 550 may include various modules such as a web view module that configures a web page, a download agent module that performs a download, a bookmark module, a web kit module, and the like.

  In addition, the service module 560 refers to an application module for providing various services. For example, the service module 560 may include various modules such as a navigation service module that provides a map, current location, landmark, route information, a game module, an advertisement application module, and the like.

  The main CPU 132 in the control unit 130 accesses the storage unit 140 via the storage unit interface 135, copies various modules stored in the storage unit 140 to the RAM 131-2, and operates according to the operation of the copied module. I do.

  As an example, in the case of the GUI display operation corresponding to the above-described pointing mode or gesture mode, the main CPU 132 configures the GUI screen using the image combination module 542-1 in the presentation module 540. Then, the display position of the GUI screen is determined using the coordinate combination module 542-2, and the display unit 120 is controlled to display the GUI screen at the position.

  Alternatively, when a user operation corresponding to the message reception operation is performed, the main CPU 132 causes the messaging module 541 to execute, accesses the message management server, and then receives a message stored in the user account. The main CPU 132 configures a screen corresponding to the received message using the presentation module 540 and then displays the screen on the display unit 120.

  In addition, the main CPU 132 can drive the telephone module 532 when performing a telephone call operation.

  As described above, the storage unit 140 may store programs having various structures, and the control unit 130 performs operations according to the various embodiments described above using various programs stored in the storage unit 140. It can be carried out.

  FIG. 6 is a block diagram showing the configuration of the remote control device according to one embodiment of the present invention.

[Operation of Remote Control Device According to First Embodiment]
As shown in FIG. 6, the remote control device 200 includes a detection unit 210, an input unit 220, a signal output unit 230, and a control unit 240.

  The remote control device 200 has a function of providing a remote control signal to the external display device 100 having a plurality of operation modes including a pointing mode and a gesture mode.

  The detection unit 210 detects a three-dimensional operation of the remote control device 200. Specifically, the detection unit 210 may include at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor. Various sensors included in the detection unit 210 can detect the three-dimensional operation of the remote control device 200 through one or a combination of two or more. Here, the three-dimensional operation may be a rotation state of the remote control device 200.

  The acceleration sensor is a sensor that specifies a spatial operation of the remote control device 200. That is, the acceleration sensor can detect at least one of a change in acceleration and a change in each acceleration that occur when the user moves the remote control device 200. Here, the acceleration sensor may be realized by a three-axis acceleration sensor that measures an increase / decrease in linear velocity with respect to three axes orthogonal to each other. Thereby, it is possible to acquire the motion acceleration related information of the remote control device 200 using the acceleration sensor and the information related to the tilt using the gravitational acceleration component without moving.

  The gyro sensor is an inertial sensor that measures the rotational angular velocity of the remote control device 200. That is, it means a sensor that can know the direction and speed of rotation using the inertial force of the rotating object. The gyro sensor may be realized by a three-axis angular velocity sensor that measures the increase / decrease of the rotation angle with respect to three axes orthogonal to each other.

  A geomagnetic sensor is a sensor that measures an azimuth. That is, the geomagnetic sensor means a sensor that measures a azimuth angle by detecting a magnetic field formed in the north-south direction of the earth. Here, the geomagnetic sensor may be realized by a triaxial geomagnetic sensor that measures the strength and direction of magnetism with respect to three axes orthogonal to each other. The north direction measured by the geomagnetic sensor may be a magnetic north. However, even if the geomagnetic sensor measures the direction of magnetic north, it can output the direction of true north through an internal calculation.

  In addition, the detection unit 210 may optionally further include a distance sensor. Here, the distance sensor is a sensor that measures the distance between the remote control device 200 and the display device 100. That is, the distance between the position where the user uses the remote control device 200 and the display device 100 can be detected.

  The input unit 220 has a function of inputting various user commands.

  Specifically, the input unit 220 includes a first button unit (not shown) used to receive a user command for selecting one of a pointing mode and a gesture mode in the operation mode of the external display apparatus 100. A second button part (not shown) used to receive a user command for selecting an object displayed on the UI screen may be included.

  In this case, when there is a pressing operation on the first button part (not shown), the information on the operation mode includes information indicating that the external device is in the gesture mode, and the pressing operation on the first button part (not shown). When is released, the information regarding the operation mode may include information indicating that the external device is in the pointing mode.

  For example, the mode information may be realized in various forms as information indicating the pointing mode or the gesture mode. For example, the mode information may be realized in the form of a flag, but in the pointing mode, the flag may indicate a “0” value, and in the gesture mode, the flag may indicate a “1” value. Flag information indicating a “0” value or a “1” value may be generated according to whether or not a first button (not shown) is operated.

  The signal output unit 230 outputs a remote control signal to the external display device 100. In this case, the signal output unit 230 may be a BT (BlueTooth (registered trademark)), Zigbee (registered trademark), Wi-Fi (Wireless Fidelity), IR (Infrared), Serial Interface, USB (Universal Serial Bus), or the like. Information regarding the remote control device can be transmitted to the display device 100 through various communication methods.

  In addition, the signal output unit 230 can output signals corresponding to various user commands input via the input unit 220 described later to the display apparatus 100.

  The control unit 240 controls the overall operation of the remote control device 200. For example, the control unit 140 can be realized by a CPU (Central Processing Unit) or an MCU (Microcontroller Unit).

  Specifically, the control unit 240 can control the operations of the detection unit 210, the input unit 220, and the signal output unit 230 described above.

  In particular, when a user command is input via the first button unit (not shown), the control unit 240 detects and detects information related to the user command for selecting either the pointing mode or the gesture mode. The signal output unit 230 can be controlled to output information related to the operation.

  Specifically, the control unit 240 continuously outputs information about a user command for selecting one of the pointing mode and the gesture mode while the first button unit (not shown) is pressed. Controlling the signal output unit 230 so as not to output information related to a user command for selecting one of the pointing mode and the gesture mode when the pressing operation on the first button unit (not shown) is released. Can do.

  In this case, the external display apparatus 100 maintains the pointing mode as a default mode during a time period in which information about a user command for selecting one of the pointing mode and the gesture mode is not received from the remote control device 200. can do.

  In addition, the external display apparatus 100 may perform non-default (Non-default) during a time period in which information about a user command for selecting one of the pointing mode and the gesture mode is continuously received from the remote control apparatus 200. ) The mode can be switched to the gesture mode and the switched mode can be held.

[Operation of Remote Control Device According to Second Embodiment]
In the case of the second embodiment, the operations of the detection unit 210, the input unit 220, and the signal output unit 230 are similar to those of the first embodiment, and thus detailed description thereof is omitted.

  However, the information related to the user command for selecting either the pointing mode or the gesture mode input via the input unit 220 is not transmitted to the display device 100, but is used to generate a control signal in the remote control device 200. May be used. Accordingly, the information related to the user command for selecting one of the pointing mode and the gesture mode input via the input unit 220, that is, the information related to the operation mode is in the form of a flag as in the first embodiment. It is not necessary to realize this, and it may be realized in various forms that can identify that a user command for mode switching is input. However, in some cases, a flag form is used as in the first embodiment. It is also possible to realize with.

  The control unit 240 uses the information related to the user command for selecting one of the pointing mode and the gesture mode input via the input unit 220 and the information related to the operation detected via the detection unit 210. A control signal for controlling 100 can be generated.

  Specifically, the controller 240 can generate a control signal for controlling the external display apparatus 100 by analyzing the information through a predefined signal processing algorithm.

  For example, the control unit 260 may analyze motion information through a gesture recognition algorithm based on information about a user command for selecting a gesture mode, and output a control signal corresponding to the analyzed gesture to the display apparatus 100. The signal output unit 230 can be controlled.

  The control unit 240 analyzes motion information through a signal processing algorithm based on information about a user command for selecting a pointing mode, and calculates an absolute coordinate value for controlling the position of the pointing object. The absolute coordinate value can be transmitted to the display apparatus 100 to control the moving state of the pointing object.

  When the selection button unit 222 is selected, the control unit 240 transmits a corresponding signal to the display device 100, and when the selection button unit 222 is selected, the item where the pointing object is located is displayed on the display device 100. Can be selected. That is, in the second embodiment, the remote control device 200 displays information related to an operation detected via the detection unit 210 based on information related to a user command for selecting either the pointing mode or the gesture mode. The control signal for controlling the display apparatus 100 can be generated by switching so as to correspond to the above. Thereby, unlike the first embodiment, the remote control device 200 operates in the pointing control mode or the gesture control mode.

  This will be described later with reference to FIG. 7 showing the detailed configuration of the remote control device.

  FIG. 7 is a block diagram showing a detailed configuration of the remote control device shown in FIG.

  The detection unit 210 is a component for sensing the operation of the remote control device 200. The detection unit 210 may include various sensors such as an acceleration sensor 211, an angular velocity sensor 212, a geomagnetic sensor 213, a touch sensor 214, and the like.

  The acceleration sensor 211 is a sensor that can measure acceleration and the direction of acceleration when an operation occurs. Specifically, the acceleration sensor 211 outputs a sensing value corresponding to the acceleration of operation of the remote control device 200 to which the sensor is attached and a sensing value corresponding to the gravitational acceleration that is changed according to the gradient. The control unit 260 can use the output value of the acceleration sensor 211 to determine the degree of inclination using the gravitational acceleration component in a state where the remote control device 200 does not move with the operation acceleration.

The angular velocity sensor 212 measures the force of Coriolis ( http://search.naver.com/?where=nexearch&query=coriolis&ie=utf8&sm=tab_prc&qdt=0:Coriolis ) acting in the velocity direction when the rotational motion is performed. , A sensor for detecting angular velocity. The control unit 260 can detect the rotation of the remote control device 200 even if the measurement value of the angular velocity sensor 212 is used.

  The geomagnetic sensor 213 is a sensor that detects the earth's magnetism and the magnetism of surrounding magnetic component objects using a biaxial or triaxial fluxgate. The control unit 260 can measure the direction and intensity of magnetism using the geomagnetic value detected from the geomagnetic sensor 213, and can calculate the azimuth angle based on it. Thereby, it can be determined in which direction the remote control device 200 is rotating.

  The touch sensor 214 can detect a user's touch operation. The touch sensor 214 may be realized by an electrostatic type or a reduced pressure type. The electrostatic touch sensor uses a dielectric material coated on the surface of the remote control device 200 and is excited by the user's human body when a part of the user's body is touched on the surface of the remote control device 200. It means a sensor that detects fine electricity and calculates touch coordinates. The pressure-reducing touch sensor includes two electrode plates built in the remote control device 200. When the user touches, the touch sensor detects that the upper and lower plates at the touched point are touched and a current flows, and the touch coordinates are obtained. It means a touch sensor of a calculation method. In addition, an infrared detection method, a surface ultrasonic conduction method, an integral tension measurement method, a piezo effect method, and the like may be used to detect a touch operation, and a detailed description thereof will be omitted.

  The input unit 220 has a function of inputting various user commands.

  Specifically, the input unit 220 includes a mode switching button unit 221, a selection button unit 222, a four-direction input unit 223, a touch input unit 224, and a power button unit 225.

  The mode switching button unit 221 may receive a user command for selecting one of the pointing mode and the gesture mode. An implementation form of the mode switching button unit 221 will be described later with reference to the drawings.

  The selection button unit 222 may receive a selection command.

  In particular, the selection button unit 222 may receive a user command for selecting an item on which the pointing object displayed on the screen of the display apparatus 100 is displayed in the pointing mode. That is, when the selection button unit 251 is pressed while the pointing object displayed on the screen of the display apparatus 100 is positioned on the specific item, the object may be selected and a corresponding function may be executed. For example, when the content is an icon interface of a specific application, the application execution screen may be displayed.

  Note that the selection button unit 222 may include functions such as an enter key and a confirmation key according to the characteristics of the provided UI screen.

  The four-direction input unit 223 is disposed outside the selection button unit 222 and has a function of inputting a user command for a four-way operation.

  On the other hand, the selection button unit 222 and the four-direction input unit 223 may be realized in a form including at least one of a touch sensor and an OJ (Optical Joystick) sensor using an optical technique.

  The touch input unit 224 may be realized in a form including a plurality of touch areas to which different functions are mapped. Specifically, the touch input unit 224 may include a plurality of touch areas to which different functions such as a channel switching function, a volume adjustment function, and a menu function are mapped.

  The power button unit 225 has a function of inputting a user command for power ON / OFF.

  Although not shown, the remote control device 200 has a centering button unit (not shown) for mapping the reference state of the remote control device 200 and the display position of the pointing object when the power button unit 225 is powered on. May be further included. Alternatively, the various buttons described above can be realized so as to function as a centering button unit (not shown). For example, when the power supply of the remote control device 200 is turned on, the remote control device 200 can perform a pairing operation with the display device 100. After the pairing is completed, when the user presses a centering button unit (not shown), a pointing object is displayed at the center of the display screen of the display device 100, and the current state of the remote control device 200 is displayed. May be matched to the position of the pointing object displayed at the center of the display screen. However, it is only one embodiment that the center of the display screen becomes the reference point, and the reference point can be set at another position on the display screen.

  On the other hand, the operation of the control unit 240 described above may be performed by a program stored in the storage unit 250.

  That is, the storage unit 250 includes O / S software for driving the remote control device 200, a signal processing algorithm (in the case of the second embodiment) for performing an operation on the detection signal detected from the detection unit 210, and the like. As described above, various data may be stored. The control unit 240 controls the overall operation of the remote control device 200 using various programs stored in the storage unit 250.

  Specifically, the control unit 240 includes a RAM 241, a ROM 242, a main CPU 243, first to nth interfaces 244-1 to 244-n and a bus 245.

  The RAM 241, the ROM 242, the main CPU 243, the first to n-th interfaces 244-1 to 244-n and the like are connected to each other via a bus 245 and can transmit and receive various data and signals.

  The first to n-th interfaces 244-1 to 244-n are connected not only to the various components shown in FIG. 7 but also to other components so that the main CPU 243 can access them.

  The main CPU 243 accesses the storage unit 250 and performs booting using the O / S stored in the storage unit 250. Various operations are performed using various programs and data stored in the storage unit 250.

  Specifically, the ROM 242 stores a command word set for system booting. When a turn-on command is input and power is supplied, the main CPU 243 copies the O / S stored in the storage unit 250 to the RAM 241 in accordance with the command stored in the ROM 242, and executes the O / S to execute the system. Let it boot. When booting is completed, the main CPU 243 copies various programs stored in the storage unit 250 to the RAM 241 and executes the programs copied to the RAM 241 to perform various operations.

  As described above, the control unit 240 can perform various operations by copying the program stored in the storage unit 250 to the RAM 241 and executing the program.

  FIG. 8 is a block diagram for explaining a form of a signal processing algorithm according to an embodiment of the present invention.

  As shown in FIG. 8, a block 1 (711) for receiving sensor measurement values, a preprocessing execution block 2 (712), a posture estimation block 3 (713), a motion distinction block 4 (714), and a calibration An execution block 5 (715), a user intention analysis block 6 (716), a gain function application block 7 (717), and a gesture recognition block 8 (718) are included. The signal processing algorithm as shown in FIG. 7 may be executed by the above-described CPU or MCU.

  Block 1 (711) receives various sensor values sensed from the detection unit 220. For example, the block 1 (711) may receive a sensing value sensed through at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor.

  Block 2 (712) includes a process of performing preprocessing on the received sensor value, and specifically includes a physical quantity conversion, a sensor axis conversion function, and a low-pass filtering function for the sensor value.

  For example, a sensor value converted into a digital value is converted into an actual physical quantity applicable to a signal processing algorithm (physical quantity conversion). In addition, the operation | work which matches the acceleration, an angular velocity, and the individual axis | shaft of a geomagnetic sensor to one defined axis | shaft can be performed (sensor axis conversion). Note that electrical noise and unintended high-frequency operation of the sensor can be removed through low-pass filtering (low-pass filtering).

  A block 3 (713) estimates the posture or posture angle (Eular Angle (Roll, Pitch, Yaw (traveling direction (heading))) from each sensor value. In this case, an estimation method using a Kalman filter may be applied.

  Block 4 (714) is a block for discriminating operating states using sensor signals, and can determine whether the remote control device 200 is not moving, moves slowly, moves quickly, or the like.

  The block 5 (715) proceeds with the calibration work. If it is determined in the block 4 (714) that the remote control device 200 is not moving, that is, if it is determined to be zero-rate, then the angular velocity is determined. After obtaining the average of the sensor output values, the value can be subtracted from the output value of the angular velocity sensor to compensate for the offset value of the angular velocity sensor.

  Block 6 (716) plays a role of analyzing and classifying whether the user wants to move or stop the remote control device 200, or which one to click.

  The block 7 (717) converts the yaw angle and the pitch angle output from the block 3 (713) into the X and Y coordinates of the display device 100, respectively. The position of the mouse cursor can be designated using such converted coordinates.

  Block 8 (718) can also perform gesture recognition for the specified gesture using the signal output from block 2 (712).

  Thereafter, the coordinates of the pointing object can be designated by matching the X and Y coordinates output from the block 7 (717) with the X and Y coordinates of the pointing object on the UI screen of the display apparatus 100.

  In addition, it is possible to respond by generating a preset event on the UI screen using Euler angles (roll, pitch, yaw) output from block 8 (718) or X, Y coordinates output from block 7 (717). The operation to be performed can be performed.

  On the other hand, the arithmetic operation based on the signal processing algorithm described above may be processed in the control unit 260 of the remote control device 200 or the control unit 130 of the display device 100.

  9A and 9B are views showing the external appearance of the remote control device 200 shown in FIGS. 6 and 7. FIG. 9A is a perspective view of the remote control device 200, and FIG. ) Shown in FIG. 9B, and FIG. 9C is a view seen from the “c” direction shown in FIG. 9A. FIG. 10 is a diagram illustrating an example in which a plurality of mode switching buttons 231 of the remote control device 200 are provided.

  Referring to FIG. 9 (a), the remote control device 200 considers the posture when the hand is naturally bent so as to minimize the feeling of fatigue on the hand during use. It is desirable to form it so that it protrudes further from the lower side of the front surface, and as the upper part goes to the rear side, it is formed so as to be gradually inclined downward with a gentler curvature.

  In this case, as shown in FIG. 9 (c), the front and left / right sides of the remote control device 200 are rounded so as to improve the grip feeling. Thereby, the user can use the remote control device 200 in a state in which the remote control device 200 is relaxed and wrapped in the hand and the hand or the wrist is not overwhelmed.

  When the user wraps the remote control device 200 with his / her hand, the remote control device 200 switches modes over the front surface or the front surface and both side surfaces of the remote control device 200 in which the middle finger F2 and the ring finger F3 of the five fingers are positioned. The button part 231 may be arranged, and the selection button part 251 may be arranged at the upper part of the remote control device 200 where the thumb F1 is located or in front of the upper part (see FIG. 11).

  Such an arrangement of the mode switching button unit 221 and the selection button unit 222 is a finger that can naturally perform an operation of pressing the mode switching button unit 221 and the selection button unit 222 in a state where the remote control device 200 is wrapped with a hand. It may be set in consideration of the positions of F1 to F3.

  On the other hand, in the present embodiment, the mode switching button unit 221 is formed as a single unit. However, the present invention is not limited to this, and the remote control device 2000 includes a plurality of mode switching button units as shown in FIG. 2210 and 2211 can also be provided. In this case, it is desirable that the mode switching button portions 2210 and 2211 are arranged vertically so that the plurality of mode switching button portions 2210 and 2211 are operated by the middle finger and the ring finger, respectively, of the five fingers.

  In the case where a plurality of mode switching button units 2210 and 2211 are provided, each mode switching button unit 2210 and 2211 may be operated alone or simultaneously to generate different predetermined signals. It can also be realized.

  The four-direction input 223 and the touch input unit 224 may be formed on the upper part of the remote control device 200. Preferably, the four-direction input unit 223 is disposed so as to surround the selection button unit 222, and the touch input unit 224 is selected. You may arrange | position so that the button part 222 may be enclosed. That is, the four-direction input unit 223 and the touch input unit 224 may be arranged while the range is expanded as the selected power supply unit 222 is located on the innermost side and gradually goes outward from the selected power supply unit 222.

  Such an arrangement of the four-direction input unit 223 and the touch input unit 224 can be easily operated with the thumb that is relatively free of movement among the five fingers in a state where the remote control device 200 is wrapped. It is taken into consideration.

  As shown in FIG. 9B, the touch input unit 224 has a channel adjustment area set on the right side, a volume adjustment area set on the left side, and a plurality of different functions mapped such as a menu area on the upper side. Touch areas may be included.

  The power button unit 225 may be disposed on the rear side of the upper portion of the remote control device 200 so as to be set at a position where finger approach is relatively easy as compared to the above button units. Such an arrangement of the power button unit 225 can minimize the case where the power button unit 225 is pressed by mistake while the remote control device 200 is used and the power is turned off.

  As described above, in the present embodiment, a method of using the remote control device 200 will be described based on a user who can use all five fingers, but is not limited thereto, and is most suitable according to the user. It is also possible to operate each button portion of the remote control device 200 using a finger.

  FIG. 11 is a diagram for explaining an operation mode of the remote control device according to various embodiments of the present invention.

  As shown in FIG. 11A, the user can select the pointing mode of the display apparatus 100 through an operation of releasing the pressed state of the mode switching button 221. As described above, when the display apparatus 100 is in the pointing mode, a pointer in the form of pointing as illustrated may be displayed on the UI screen provided from the display apparatus 100.

  As shown in FIG. 11B, the gesture mode of the display device 100 can be selected through a grip operation that holds the pressed state of the mode switching button 221. As described above, when the display apparatus 100 is in the gesture mode, a palm-shaped pointer may be displayed on the UI screen provided from the display apparatus 100 as illustrated.

  In this case, on the UI screen, a flicking operation, a zoom in / out operation, and the like may be possible through a gesture operation of the remote control device 200. For example, a flicking operation for switching the displayed UI screen to another UI screen, a flicking operation for displaying a new menu screen, a zoom-in / out operation for performing a zoom-in / out function on the displayed content, and the like are possible. It may be.

  As described above, the display apparatus 100 can basically operate in the pointing mode, and can operate in the gesture mode when there is a grip operation for holding the pressed state of the mode switching button unit 221. .

  As illustrated in FIG. 11C, the user can perform a panning operation through an operation of holding the pressed state of the mode switching button unit 221 and the selection button unit 222. In this case, a grip-type pointer may be displayed on the UI screen provided from the display apparatus 100 as illustrated. For example, when there is an operation of rotating the remote control device 200 in a specific direction, the content image enlarged and displayed on the screen may be moved and displayed in the direction. That is, when the remote control device 200 is rotated in the vertical and horizontal directions with the specific content selected, the displayed content image may be moved in a direction corresponding to the rotation direction of the remote control device 200. Accordingly, it is possible to provide an intuitive recognition such as grasping and moving content to the user through a grip-type pointer.

  FIG. 12 is a diagram for explaining an operation state in the pointing mode according to the embodiment of the present invention.

  As shown in FIG. 12, when the display apparatus 100 is in the pointing mode, the display apparatus 100 can display a GUI corresponding to the pointing mode. That is, the pointing hand 10 may be displayed on the screen.

  When the user rotates the remote control device 200 to the right side with the pointing type pointer 10 positioned on the content A 121, the pointing type pointer 10 displayed on the screen is displayed on the content B 122 positioned on the right side of the content A 121. Can be moved to.

  Specifically, the pointing-type pointer 10 displayed on the screen can be moved to correspond to the rotation direction and the rotation degree of the remote control device 200, and can be moved based on the absolute coordinate system. Here, the absolute coordinate system is a system in which the pointer moves in the absolute coordinate area, and the pointer displayed on the screen moves around a preset reference point. This is different from the relative coordinate system based on the position.

  FIG. 13 is a diagram for explaining an operation state related to mode switching according to an embodiment of the present invention.

  As shown in the first drawing of FIG. 13, when the display device 100 is in the pointing mode, the pointer 10 in the pointing form indicating the pointing mode may be displayed on the screen of the display device 100.

  Next, when the display device 100 is switched to the gesture mode, the pointer 10 displayed on the screen of the display device 100 is changed to a palm-shaped pointer 11 indicating the gesture mode, and the guide GUI 21 guides the gesture input direction. -1, 21-2 may be displayed. For example, as shown in the figure, when new UI pages are systematically arranged on the left and right sides, a guide GUI 21-1 and a guide GUI 21-2 are displayed in the left and right arrow directions indicating that a left and right side flicking gesture is possible. May be.

  When the remote control device 200 is rotated to the left, the palm-shaped pointer 11 is changed to the pointer 12 whose angle has been changed in the same manner as that of the actual user's left-direction flickering hand, and is displayed. The guide GUI 21-1 that guides the user may be displayed in a highlighted form. For example, normally, when a flicking operation is performed on a touch screen, the angle of the user's hand is changed. Similarly, the pointer form may be changed to the same form and displayed. This makes it possible to provide the user with an intuitive recognition such as performing a flicking operation on the touch screen.

  The moving direction of the UI screen displayed on the screen may be controlled according to the rotation direction of the remote control device 200. For example, as shown in the figure, when the remote control device 200 is rotated to the left, the UI screen may be moved to the left and displayed. That is, the content displayed at the center may be moved to the left and displayed, and the content that was not visible on the right may be displayed.

  FIG. 14 is a diagram for explaining an operation state in the gesture mode according to the embodiment of the present invention.

  As shown in FIG. 14, in a state where the specific content is displayed on the screen, the user selects the first button unit (for example, the mode switching button unit 221 shown in FIG. 4B) provided in the remote control device 200. When the pressing operation is performed, the display apparatus 100 operates in the gesture mode, and the guide GUI 22-1 to the guide GUI 22-4 that notify that the gesture input is possible are displayed on the screen.

  Next, when the user rotates the remote control device 200 in a specific direction while pressing the mode switching button unit 221, a preset menu may be displayed in an area corresponding to the rotation direction. For example, as shown in the figure, when the remote control device 200 is rotated to the right, the content list may be displayed in the left area.

  Note that when the user releases the pressing operation of the mode switching button, the display apparatus 100 operates in the pointing mode. In this case, the pointing pointer 14 may be moved according to the rotation state of the remote control device 200. For example, as shown in the drawing, when the user rotates the remote control device 200 in a state where the pointing pointer 14 is positioned on the specific content C123, the pointing pointer 14 moves the content D124 below depending on the degree of rotation. May be moved to.

  Thereafter, when the selection button provided in the remote control device 200 is pressed with the pointing-type pointer 14 positioned on the content D124, the selected content D124 is executed and displayed on the screen.

  FIG. 15 is a diagram for explaining an operation state in the gesture mode according to another embodiment of the present invention.

  As shown in FIG. 15, in the pointing mode, with the pointing pointer 15 positioned on the content E125, the selection button (for example, the selection button unit 232 in FIG. 9B) provided on the remote control device 200 is pressed. When the operation is performed, the selected content E125 is displayed on the screen of the display device 100.

  Next, when the user presses a mode switching button (for example, the mode switching button 221 in FIG. 9B) provided in the remote control device 200, the pointing pointer 15 displayed on the screen is in the gesture mode. The guide GUI 23-1 to the guide GUI 23-2 may be displayed to guide that the gesture input is possible.

  When the user presses the mode switching button and the remote control device 200 is moved to a position away from the user, the content displayed on the screen may be zoomed out (Zoom Out) and displayed. Here, the movement to the distant position may be a state in which the remote control device 200 is rotated in a direction toward the display device 100.

  When the user presses the mode switching button and moves the remote control device 200 to a position closer to the user, the content displayed on the screen may be zoomed in (Zoom In) and displayed. Here, the movement to the close position may be a state in which the remote control device 200 is rotated in the opposite direction of the display device 100.

  FIG. 16 is a diagram for explaining an operation state in the gesture mode according to still another embodiment of the present invention.

  As shown in FIG. 16, in the pointing mode, the second button unit (for example, the selection button unit 222 of FIG. 9B) provided in the remote control device 200 with the pointer 15 in the pointing form positioned on the content E <b> 125. ) Is pressed, the selected content E125 is displayed on the screen of the display device 100.

  Next, when the user presses the mode switching button (for example, the mode switching button 231 in FIG. 9B) and holds the selection button pressing operation state, the pointing pointer 15 can be panned. The guide GUI 24-1 to the guide GUI 24-4 may be displayed by changing to the grip-type pointer 17 indicating that there is, and guiding that gesture input is possible.

  Note that when the user presses the mode switching button and the selection button and the remote control device 200 is rotated in the up / down / left / right directions, panning operation may be performed on the content displayed on the screen. For example, as shown in the figure, when the user presses the mode switching button and the selection button and the remote control device 200 is rotated to the right, the content displayed on the screen is also moved to the right and displayed. Good.

  However, in the above-described embodiment, it has been described that the panning operation is performed in a state where the user presses the mode switching button and the selection button at the same time. However, this is only an embodiment, and the pressing operation of the mode switching button is performed. Nonetheless, the panning operation may be performed even in a state where the pressing operation of the selection button is held.

  FIG. 17 is a flowchart for explaining a control method of the remote control device according to the embodiment of the present invention.

  As shown in FIG. 17, the remote control device 200 communicates with the display device 100 (S1710). For example, when the remote control device 200 and the display device 100 communicate via the Bluetooth method, when the remote control device 200 is powered on, the remote control device 200 searches for the display device 100 and performs pairing with the display device 100. be able to.

  After the pairing, the display apparatus 100 can operate in a pointing mode. Specifically, when a user command is input through the remote control device 200, a pointing object is displayed on the screen of the display device 100, and the position of the displayed pointing object and the current state of the remote control device 200 are mapped. May be.

  Next, the operation of the remote control device 200 is detected (S1720).

  Thereafter, when a user command for selecting one of the pointing mode and the gesture mode is input (S1730: Y), information on the operation detected from step S1720 and information on the user command input in step S1730 Is output to the display device 100 (S1740).

  Here, the pointing mode is a mode for controlling the operation of the pointing object displayed on the UI screen of the display device 100 according to the detected operation of the remote control device, and the gesture mode is the remote control device 200. This mode may be a mode for controlling the display state of the UI screen according to the predefined gesture.

  On the other hand, in the pointing mode, the movement state of the pointing object displayed on the screen of the display device 100 may be controlled according to the operation state of the remote control device 200. Here, the operation state may be the rotation state of the remote control device 200 as described above. In the gesture mode, the display state of the screen of the display device 100 may be controlled according to the operation state of the remote control device 200. For example, screen switching such as displaying a menu screen or displaying another UI screen may be performed.

  On the other hand, when a user command that is not mode switching is input, the remote control device 200 can transmit a control signal corresponding to the input command to the display device 100. For example, when a content selection command, a pointing object movement command, or the like is input, a control signal corresponding to the command can be transmitted to the display apparatus 100.

  FIG. 18 is a flowchart for explaining a control method of the remote control device according to the embodiment of the present invention.

  As shown in FIG. 18, the display apparatus 100 displays a UI screen having a pointing object for performing an indicator function according to a preset event (S1810). Here, the preset event may be a time when the connection with the remote control device 200 is performed, a time when a specific button is input on the connected remote control device 200, or the like.

  Next, a remote control signal including information related to the operation of the remote control device and information related to a user command for selecting any one of the pointing mode and the gesture is received from the remote control device 200 (S1820).

  Thereafter, based on the information related to the user command for selecting either the pointing mode or the gesture mode received from the remote control device 200, either the pointing mode or the gesture mode is selected and operated (S1830).

  Specifically, when the display apparatus 100 receives information on a user command for selecting the pointing mode from the remote control apparatus 200, the display apparatus 100 operates in the pointing mode, and displays the above-mentioned pointing-type pointer on the UI screen. When the information about the user command to display and select the gesture mode is received from the remote control device 200, it may operate in the gesture mode and display a palm-shaped hand-shaped pointer.

  When a flicking operation is input through the remote control device 200 in the gesture mode, a hand-shaped pointer can be displayed in a form corresponding to the action of the flicking hand.

  When the panning operation is possible in the gesture mode, the hand-shaped pointer may be changed to the grip form and displayed. Since the specific form for each hand-shaped pointer has already been described above, further explanation is omitted.

  The display apparatus 100 can display a guide GUI for guiding gesture input in the gesture mode.

  For example, when a flicking operation is possible, a left-right arrow guide GUI that guides left-right gesture input, and when a zoom-in / out operation is possible, a up-down arrow guide GUI that guides up-down gesture input, When a panning operation is possible, a guide GUI in the up / down / left / right arrow direction for guiding the up / down / left / right gesture input can be displayed.

  As described above, according to various embodiments of the present invention, an intuitive remote control device can be provided to a user.

  Meanwhile, the above-described methods according to various embodiments of the present invention may be realized only by software upgrade for an existing display device or user terminal device.

  In addition, a non-transitory readable medium (non-transitory computer readable medium) in which a program for sequentially performing the control method according to the present invention is stored may be provided.

  Non-temporary readable medium means not a medium that stores data for a short time, such as a register, cache, or memory, but a medium that can store data semipermanently and can be read by a device. To do. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, a DVD, a hard disk, a Blu-ray disc, a USB, a memory card, or a ROM.

  In the above block diagram shown for the display device and the remote control device, the bus (Bus) is not shown. However, communication between each component in the display device and the remote control device may be performed by the bus. Each device may further include a processor such as a CPU or a microprocessor that performs the various steps described above.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments. It is obvious that a person having ordinary knowledge in the technical field to which the present invention belongs can come up with various changes or modifications within the scope of the technical spirit described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 110 Reception part 130 Control part 120 Display part 150 Image | video receiving part 160 Image | video processing part 170 User interface part 140 Storage part 180 UI processing part 131 System memory 132 Main CPU
133 Image processing unit 134 Network interface 137 Audio processing unit 135 Storage unit interface 136-1 First interface 136-2 Second interface 136-3 Third interface 136-n nth interface 231 BT chip 232 Wi-Fi chip 233 IR communication Chip 221 Mode switching button section 222 Selection button section 223 Four-direction input section 224 Touch input section 225 Power button section 243 Main CPU
244-1 first interface 224-n nth interface 250 storage unit 210 detection unit 211 acceleration sensor 212 angular velocity sensor 213 geomagnetic sensor 214 touch sensor 550 web browser module 520 sensing module 522 device information management module 523 remote control module 240 control unit 220 Input unit 230 Signal output unit

Claims (15)

In a remote control device for providing a remote control signal to an external display device having a plurality of operation modes including a pointing mode and a gesture mode,
A signal output unit for outputting the remote control signal to the external display device for providing a UI screen;
A detection unit for detecting the operation of the remote control device;
A first button used to receive a user command for selecting one of the pointing mode and the gesture mode in the operation mode of the external display device; and an object displayed on the UI screen. An input unit including a second button unit used to receive a user command to select
When a user command is input through the first button unit, the information related to the user command for selecting either the pointing mode or the gesture mode and the information related to the detected motion are output. A control unit for controlling the signal output unit,
The pointing mode is a mode for controlling the operation of the object displayed on the UI screen in accordance with the operation of the remote control device detected by the detection unit, and the gesture mode is a preset mode of the remote control device. A remote control device that is a mode for controlling a display state of the UI screen according to a set gesture. The external display device maintains the pointing mode as a default mode during a time period in which information about the user command for selecting one of the pointing mode and the gesture mode is not received from the remote control device. ,
During a time period in which information on the user command for selecting one of the pointing mode and the gesture mode is continuously received from the remote control device, the external display device is set as the non-default mode in the gesture mode. The remote control device according to claim 1, wherein the switched mode is maintained. The controller is
While the first button portion is pressed, information on a user command for selecting either the pointing mode or the gesture mode is continuously output, and the pressing operation on the first button portion is released. The remote control according to claim 1, wherein the signal output unit is controlled so as not to output information on the user command for selecting one of the pointing mode and the gesture mode. apparatus. The detector is
The remote control device according to claim 1, comprising at least one of an acceleration sensor, a gyro sensor, and a geomagnetic sensor. In a display device controlled by a remote control device that provides a remote control signal and having a plurality of operating modes including a pointing mode and a gesture mode,
A receiver that receives from the remote control device a remote control signal including information related to the operation of the remote control device, information related to a user command for selecting one of the pointing mode and the gesture mode, and information related to item selection. When,
A display unit for displaying a UI screen having a pointing object for performing an indicator function;
A control unit that controls selection of either the pointing mode or the gesture mode based on information related to a user command for selecting either the pointing mode or the gesture mode received from the remote control device Including and
The pointing mode is a mode for controlling the operation of the pointing object displayed on the UI screen according to the operation of the remote control device detected by the detection unit of the remote control device, and the gesture mode is The display device is a mode for controlling a display state of the UI screen according to a predefined gesture of the remote control device. The display unit is
The display device according to claim 5, wherein the pointing object is changed according to the operation mode. The controller is
While the receiving unit has not received information on the user command for selecting one of the pointing mode and the gesture mode, the operation mode of the display device is determined as the pointing mode,
6. The operation mode of the display device is determined to be the gesture mode while receiving information on the user command for selecting one of the pointing mode and the gesture mode. Display device.   When the operation mode of the display device is the pointing mode, the menu item included in the UI screen is selected based on information regarding the operation of the remote control device and information regarding item selection received from the remote control device. The display device according to claim 5, wherein the display device is a display device. The controller is
According to an operation state that can be input on the UI screen, at least one of the shape, hue, size, position, and direction of the pointing object is controlled to be displayed. The display device according to claim 6. The pointing object is
A hand-shaped pointer,
The controller is
Of the flicking operation, zoom-in / out operation, and panning operation, the hand-shaped pointer is set to a pointing form based on the form that can be operated on the UI screen. The display apparatus according to claim 6, wherein the display device is controlled to be changed to a palm shape or a grip shape. In a display system comprising a remote control device for providing a remote control signal and a display device having a plurality of operation modes including a pointing mode and a gesture mode controlled by the remote control device,
A remote control device that outputs a remote control signal including information related to operation of the remote control device, information related to a user command for selecting one of the pointing mode and the gesture mode, and information related to item selection;
Including a display device,
The display device includes:
A receiver that receives from the remote control device a remote control signal including information related to the operation of the remote control device, information related to a user command for selecting one of the pointing mode and the gesture mode, and information related to item selection. When,
A display unit for displaying a UI screen having a pointing object for performing an indicator function on the UI screen;
A control unit that controls selection of either the pointing mode or the gesture mode based on information related to a user command for selecting either the pointing mode or the gesture mode received from the remote control device Including and
The pointing mode is a mode for controlling the operation of the pointing object displayed on the UI screen according to the operation of the remote control device detected by the detection unit of the remote control device, and the gesture mode is The display system is a mode for controlling a display state of the UI screen according to a predefined gesture of the remote control device. The display device is configured to hold the pointing mode as a default mode while information about the user command for selecting one of the pointing mode and the gesture mode is not received from the remote control device And
While continuously receiving information on the user command for selecting one of the pointing mode and the gesture mode, the mode is configured to switch to the gesture mode as a non-default mode and hold the switched mode. The display system according to claim 11. The remote control device is:
A signal output unit for outputting the remote control signal to the display device;
An input unit including a first button unit that receives a user command for selecting one of the pointing mode and the gesture mode;
While the first button portion is pressed, information on the user command for selecting either the pointing mode or the gesture mode is output, and the pressing operation of the first button portion is released. And a control unit that controls the signal output unit so as not to output information on the user command for selecting one of the pointing mode and the gesture mode. The display system described. In a control method of a remote control device for providing a remote control signal to an external display device having a plurality of operation modes including a pointing mode and a gesture mode,
Detecting the operation of the remote control device;
When a user command for selecting one of the pointing mode and the gesture mode is input in the operation mode of the remote control device, information on the detected operation and any of the pointing mode and the gesture mode Outputting information on a user command for selecting one of the two to the external display device,
The pointing mode is a mode for controlling the operation of the pointing object displayed on the UI screen according to the operation of the remote control device detected by the detection unit of the remote control device, and the gesture mode is the gesture mode A control method for a remote control device, characterized in that it is a mode for controlling the display state of the UI screen according to a predefined gesture of the remote control device. In a control method of a display device controlled by a remote control device that provides a remote control signal and having a plurality of operation modes including a pointing mode and a gesture mode,
Displaying a UI screen having a pointing object for performing an indicator function;
Receiving from the remote control device a remote control signal including information regarding the operation of the remote control device and information regarding a user command for selecting one of the pointing mode and the gesture mode;
Selecting one of the pointing mode and the gesture mode based on the information about the user command for selecting either the pointing mode or the gesture mode received from the remote control device; A display device control method comprising:

Images