WO2024117306A1 - Display device and operation method therefor - Google Patents

Abstract

A display device and an operation method therefor are disclosed. The operation method for the display device, according to at least one of various embodiments of the present disclosure, may comprise the steps of: receiving a video play request signal; decoding video data; calculating an offset related to a video delay by receiving information about the decoded video data; transmitting the calculated offset related to the video delay to a graphics thread; and controlling that a screen is output by synchronizing a video and graphics.

Description

Display device and method of operation thereof

This disclosure relates to a display device and a method of operating the same.

Recently, the functions of terminals have been diversifying, for example, data and voice communication, taking photos and videos using a camera, recording voice, playing music files through a speaker system, etc., and outputting images or videos on a display. .

Some terminals add electronic game play functions or perform multimedia player functions.

As the functions of these terminals become more diverse, they are implemented as multimedia devices (Multimedia players) with complex functions such as taking photos or videos, playing music or video files, playing games, and receiving broadcasts. there is.

Accordingly, the processing of multimedia data that processes video and graphics together is increasing in terminals, but the processing time for high-end video is gradually increasing compared to graphics, so there are frequent cases of out-of-sync, making it inconvenient for users to watch videos. There is something to do.

The present disclosure seeks to provide a method of synchronization according to the occurrence of physical delay between hardware for video processing and hardware for processing graphics.

The present disclosure seeks to provide a display device and a method of operating the same that handle the physical delay between the above-described hardware configurations in software.

A method of operating a display device according to at least one embodiment among various embodiments of the present disclosure includes receiving a video playback request signal; decoding video data; Receiving information of decoded video data and calculating an offset regarding video delay; Passing the calculated offset related to the video delay to the graphics thread; and controlling to output a screen by synchronizing video and graphics.

According to the method of operating a display device according to at least one embodiment among various embodiments of the present disclosure, the method may further include storing an offset related to the calculated video delay.

According to a method of operating a display device according to at least one embodiment among various embodiments of the present disclosure, the method includes receiving an event signal; And it may further include checking the characteristics of the received event signal.

According to a method of operating a display device according to at least one embodiment among various embodiments of the present disclosure, when the confirmed characteristic of the received event signal is the first characteristic, reading the stored offset and transmitting it to the graphics thread ; And it may further include controlling to output a screen according to the event signal by synchronizing video and graphics.

According to a method of operating a display device according to at least one embodiment among various embodiments of the present disclosure, the first characteristic is a frame added to video processing in addition to the number of frames used when calculating the offset for the calculated video delay. This may indicate an unsolicited event signal.

According to a method of operating a display device according to at least one embodiment among various embodiments of the present disclosure, when the confirmed characteristic of the received event signal is a second characteristic, recalculating an offset related to the video delay; storing an offset for the recalculated video delay and passing it to the graphics thread; And it may further include controlling to output a screen according to the event signal by synchronizing video and graphics.

According to a method of operating a display device according to at least one embodiment among various embodiments of the present disclosure, the second characteristic may include additional frames for video processing in addition to the number of frames used when calculating the offset for the calculated video delay. This may indicate the required event signal.

A display device according to at least one embodiment among various embodiments of the present disclosure includes a video decoder that decodes video data when a video playback request signal is received; a video processor that receives information from decoded video data and calculates an offset regarding video delay; a graphics processor that processes graphics data for the decoded video data; and a control unit that transmits the calculated offset related to the video delay to a graphics thread of the graphics processor and controls a screen in which video and graphics are synchronized to be output.

According to at least one embodiment of the various embodiments of the present disclosure, the display device may further include a memory that stores an offset related to the calculated video delay.

According to a display device according to at least one embodiment among various embodiments of the present disclosure, when an event signal is received, the control unit may check characteristics of the received event signal.

According to the display device according to at least one embodiment among the various embodiments of the present disclosure, when the confirmed characteristic of the received event signal is the first characteristic, the control unit reads the stored offset and controls the graphics of the graphics processor. By transmitting it to a thread, the video and graphics can be synchronized and the screen according to the event signal can be controlled to be output.

According to the display device according to at least one embodiment among the various embodiments of the present disclosure, the first characteristic is that no additional frames are required for video processing other than the number of frames used when calculating the offset for the calculated video delay. It can indicate an event signal that does not occur.

According to the display device according to at least one embodiment among the various embodiments of the present disclosure, when the confirmed characteristic of the received event signal is the second characteristic, the control unit determines the offset related to the video delay in the video processor. Control to calculate, control to store the offset related to the video delay recalculated by the video processor in the memory, and transmit it to the graphics thread of the graphics processor, so that the video and graphics are synchronized to output the screen according to the event signal. You can control it.

According to the display device according to at least one embodiment among the various embodiments of the present disclosure, the second characteristic is such that additional frames are required for video processing in addition to the number of frames used when calculating the offset for the calculated video delay. Can indicate event signals.

According to at least one of the various embodiments of the present disclosure, the screen can be output in sync with the video and graphics despite the physical delay that occurs between hardware configurations, which has the effect of increasing the user's satisfaction with the display device. .

According to at least one of the various embodiments of the present disclosure, synchronization between video and graphics can be performed using a software method without adding a separate hardware device, thereby suppressing the occurrence of additional costs and increasing system efficiency. There is an effect.

Figure 1 shows a block diagram of the configuration of a display device according to an embodiment of the present invention.

Figure 2 is a block diagram of a remote control device according to an embodiment of the present invention.

Figure 3 shows an example of the actual configuration of a remote control device according to an embodiment of the present invention.

Figure 4 shows an example of utilizing a remote control device according to an embodiment of the present invention.

Figure 5 is a diagram for explaining the landscape mode and portrait mode of a stand-type display device according to an embodiment of the present disclosure.

Figure 6 is a block diagram of a display device according to another embodiment of the present disclosure.

7 and 8 are flowcharts illustrating a method of operating a display device according to an embodiment of the present disclosure.

9 to 11 are diagrams to explain a method of synchronizing a video area and a graphics area in a video processing process of a display device according to an embodiment of the present disclosure.

12 and 13 are diagrams to explain the video processing operation of the display device according to an embodiment of the present disclosure.

Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes “module” and “part” for components used in the following description are given or used interchangeably only for the ease of preparing the specification, and do not have distinct meanings or roles in themselves.

The display device according to an embodiment of the present invention is, for example, an intelligent display device that adds a computer support function to the broadcast reception function, and is faithful to the broadcast reception function while adding an Internet function, etc., such as a handwriting input device and a touch screen. Alternatively, it can be equipped with a more convenient interface such as a spatial remote control. In addition, by supporting wired or wireless Internet functions, you can connect to the Internet and a computer and perform functions such as email, web browsing, banking, or gaming. A standardized general-purpose OS (Operating System) can be used for these various functions.

Accordingly, in the display device described in the present invention, for example, various applications can be freely added or deleted on a general-purpose OS kernel, so various user-friendly functions can be performed. More specifically, the display device may be, for example, a network TV, HBBTV, smart TV, LED TV, OLED TV, etc., and in some cases, may also be applied to a smartphone.

Figure 1 shows a block diagram of the configuration of a display device 100 according to an embodiment of the present invention.

Referring to FIG. 1, the display device 100 includes a broadcast receiver 130, an external device interface 135, a memory 140, a user input interface 150, a controller 170, a wireless communication interface 173, and a display. It may include (180), a speaker (185), and a power supply circuit (190).

The broadcast receiver 130 may include a tuner 131, a demodulator 132, and a network interface 133.

The tuner 131 can select a specific broadcast channel according to a channel selection command. The tuner 131 can receive a broadcast signal for a specific selected broadcast channel.

The demodulator 132 can separate the received broadcast signal into a video signal, an audio signal, and a data signal related to the broadcast program, and can restore the separated video signal, audio signal, and data signal to a form that can be output.

The external device interface 135 may receive an application or application list within an adjacent external device and transfer it to the controller 170 or memory 140.

The external device interface 135 may provide a connection path between the display device 100 and an external device. The external device interface 135 may receive one or more of video and audio output from an external device connected wirelessly or wired to the display device 100 and transmit it to the controller 170. The external device interface 135 may include a plurality of external input terminals. The plurality of external input terminals may include an RGB terminal, one or more High Definition Multimedia Interface (HDMI) terminals, and a component terminal.

An image signal from an external device input through the external device interface 135 may be output through the display 180. A voice signal from an external device input through the external device interface 135 may be output through the speaker 185.

External devices that can be connected to the external device interface 135 include a set-top box (STB), Blu-ray player, DVD player, game console, sound bar, smartphone, PC, USB memory, and home theater. It can be any one, but this is just an example.

The network interface 133 may provide an interface for connecting the display device 100 to a wired/wireless network including an Internet network. The network interface 133 may transmit or receive data with other users or other electronic devices through a connected network or another network linked to the connected network.

Additionally, some of the content data stored in the display device 100 may be transmitted to a selected user or selected electronic device among other users or other electronic devices pre-registered in the display device 100.

The network interface 133 can access a certain web page through a connected network or another network linked to the connected network. In other words, you can access a certain web page through a network and transmit or receive data with the corresponding server.

And, the network interface 133 can receive content or data provided by a content provider or network operator. That is, the network interface 133 can receive content and information related thereto, such as movies, advertisements, games, VODs, and broadcast signals, provided from a content provider or network provider through a network.

Additionally, the network interface 133 can receive firmware update information and update files provided by a network operator, and can transmit data to the Internet, a content provider, or a network operator.

The network interface 133 can select and receive a desired application from among applications open to the public through a network.

The memory 140 stores programs for processing and controlling each signal in the controller 170, and can store signal-processed video, voice, or data signals.

In addition, the memory 140 may perform a function for temporary storage of video, voice, or data signals input from the external device interface 135 or the network interface 133, and may store information about a predetermined image through a channel memory function. You can also store information.

The memory 140 may store an application or application list input from the external device interface 135 or the network interface 133.

The display device 100 can play content files (video files, still image files, music files, document files, application files, etc.) stored in the memory 140 and provide them to the user.

The user input interface 150 may transmit a signal input by the user to the controller 170 or transmit a signal from the controller 170 to the user. For example, the user input interface 150 may be used remotely according to various communication methods such as Bluetooth, Ultra Wideband (UWB), ZigBee, Radio Frequency (RF) communication, or Infrared (IR) communication. Control signals such as power on/off, channel selection, and screen settings can be received and processed from the control device 200, or control signals from the controller 170 can be processed to be transmitted to the remote control device 200.

Additionally, the user input interface 150 can transmit control signals input from local keys (not shown) such as power key, channel key, volume key, and set value to the controller 170.

The video signal processed by the controller 170 may be input to the display 180 and displayed as an image corresponding to the video signal. Additionally, the image signal processed by the controller 170 may be input to an external output device through the external device interface 135.

The voice signal processed by the controller 170 may be output as audio to the speaker 185. Additionally, the voice signal processed by the controller 170 may be input to an external output device through the external device interface 135.

In addition, the controller 170 may control overall operations within the display device 100.

In addition, the controller 170 can control the display device 100 by a user command or internal program input through the user input interface 150, and connects to the network to display an application or application list desired by the user on the display device ( 100) You can make it available for download.

The controller 170 allows channel information selected by the user to be output through the display 180 or speaker 185 along with the processed video or audio signal.

In addition, the controller 170 controls a video signal or audio signal from an external device, such as a camera or camcorder, input through the external device interface 135 according to an external device video playback command received through the user input interface 150. so that it can be output through the display 180 or speaker 185.

Meanwhile, the controller 170 can control the display 180 to display an image, for example, a broadcast image input through the tuner 131, an external input image input through the external device interface 135, Alternatively, an image input through the network interface unit or an image stored in the memory 140 may be controlled to be displayed on the display 180. In this case, the image displayed on the display 180 may be a still image or a moving image, and may be a 2D image or a 3D image.

In addition, the controller 170 can control the playback of content stored in the display device 100, received broadcast content, or external input content, which may include broadcast video, external input video, audio files, It can be in various forms such as still images, connected web screens, document files, etc.

The wireless communication interface 173 can communicate with external devices through wired or wireless communication. The wireless communication interface 173 can perform short range communication with an external device. For this purpose, the wireless communication interface 173 includes Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), UWB, ZigBee, Near Field Communication (NFC), and Wi-Fi (Wireless- Short-distance communication can be supported using at least one of (Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies. This wireless communication interface 173 is between the display device 100 and a wireless communication system, between the display device 100 and another display device 100, or between the display device 100 through wireless area networks. It can support wireless communication between the network and the display device 100 (or external server). Local area wireless networks may be wireless personal area networks.

Here, the other display device 100 is a wearable device (e.g., a smart watch) capable of exchanging data with (or interoperable with) the display device 100 according to the present invention. It can be a mobile terminal such as smart glass, head mounted display (HMD), or smart phone. The wireless communication interface 173 may detect (or recognize) a wearable device capable of communicating around the display device 100.

Furthermore, if the detected wearable device is a device authenticated to communicate with the display device 100 according to the present invention, the controller 170 sends at least a portion of the data processed by the display device 100 to the wireless communication interface 173. It can be transmitted to a wearable device through . Accordingly, a user of a wearable device can use data processed by the display device 100 through the wearable device.

The display 180 converts video signals, data signals, and OSD (On Screen Display) signals processed by the controller 170 or video signals and data signals received from the external device interface 135 into R, G, and B signals, respectively. A driving signal can be generated by conversion.

Meanwhile, the display device 100 shown in FIG. 1 is only one embodiment of the present invention. Some of the illustrated components may be integrated, added, or omitted depending on the specifications of the display device 100 that is actually implemented.

That is, as needed, two or more components may be combined into one component, or one component may be subdivided into two or more components. In addition, the functions performed by each block are for explaining embodiments of the present invention, and the specific operations or devices do not limit the scope of the present invention.

According to another embodiment of the present invention, unlike shown in FIG. 1, the display device 100 does not have a tuner 131 and a demodulator 132 but has a network interface 133 or an external device interface 135. You can also receive and play video through the device.

For example, the display device 100 is divided into an image processing device such as a set-top box for receiving broadcast signals or contents according to various network services, and a content playback device for playing content input from the image processing device. It can be implemented.

In this case, the method of operating a display device according to an embodiment of the present invention to be described below includes not only the display device 100 as described with reference to FIG. 1, but also an image processing device such as the separated set-top box or the display 180. ) and a content playback device having an audio output unit 185.

Next, with reference to FIGS. 2 and 3, a remote control device according to an embodiment of the present invention will be described.

Figure 2 is a block diagram of a remote control device according to an embodiment of the present invention, and Figure 3 shows an example of the actual configuration of the remote control device 200 according to an embodiment of the present invention.

First, referring to FIG. 2, the remote control device 200 includes a fingerprint reader 210, a wireless communication circuit 220, a user input interface 230, a sensor 240, an output interface 250, and a power supply circuit ( 260), memory 270, controller 280, and microphone 290.

Referring to FIG. 2, the wireless communication circuit 220 transmits and receives signals to and from any one of the display devices 100 according to the embodiments of the present invention described above.

The remote control device 200 has an RF circuit 221 capable of transmitting/receiving signals to and from the display device 100 in accordance with RF communication standards, and is capable of transmitting/receiving signals to and from the display device 100 in accordance with IR communication standards. An IR circuit 223 that can be used may be provided. Additionally, the remote control device 200 may be provided with a Bluetooth circuit 225 capable of transmitting and receiving signals to and from the display device 100 according to the Bluetooth communication standard. In addition, the remote control device 200 is provided with an NFC circuit 227 capable of transmitting and receiving signals with the display device 100 according to the NFC (Near Field Communication) communication standard, and displays the display device 100 according to the WLAN (Wireless LAN) communication standard. A WLAN circuit 229 capable of transmitting and receiving signals to and from the device 100 may be provided.

In addition, the remote control device 200 transmits a signal containing information about the movement of the remote control device 200 to the display device 100 through the wireless communication circuit 220.

Meanwhile, the remote control device 200 can receive a signal transmitted by the display device 100 through the RF circuit 221 and, if necessary, turn on/off the display device 100 through the IR circuit 223. Commands for turning off, changing channels, changing volume, etc. can be sent.

The user input interface 230 may be composed of a keypad, button, touchpad, or touch screen. The user can input commands related to the display device 100 into the remote control device 200 by manipulating the user input interface 230. If the user input interface 230 has a hard key button, the user can input a command related to the display device 100 to the remote control device 200 through a push operation of the hard key button. This will be explained with reference to FIG. 3 .

Referring to FIG. 3, the remote control device 200 may include a plurality of buttons. The plurality of buttons include a fingerprint recognition button (212), power button (231), home button (232), live button (233), external input button (234), volume control button (235), voice recognition button (236), It may include a channel change button 237, a confirmation button 238, and a back button 239.

The fingerprint recognition button 212 may be a button for recognizing the user's fingerprint. In one embodiment, the fingerprint recognition button 212 is capable of a push operation and may receive a push operation and a fingerprint recognition operation.

The power button 231 may be a button for turning on/off the power of the display device 100.

The home button 232 may be a button for moving to the home screen of the display device 100.

The live button 233 may be a button for displaying a real-time broadcast program.

The external input button 234 may be a button for receiving an external input connected to the display device 100.

The volume control button 235 may be a button for adjusting the volume of the sound output by the display device 100.

The voice recognition button 236 may be a button for receiving the user's voice and recognizing the received voice.

The channel change button 237 may be a button for receiving a broadcast signal of a specific broadcast channel.

The confirmation button 238 may be a button for selecting a specific function, and the back button 239 may be a button for returning to the previous screen.

Figure 2 will be described again.

If the user input interface 230 has a touch screen, the user can input commands related to the display device 100 through the remote control device 200 by touching a soft key on the touch screen. Additionally, the user input interface 230 may be provided with various types of input means that the user can operate, such as scroll keys or jog keys, and this embodiment does not limit the scope of the present disclosure.

The sensor 240 may include a gyro sensor 241 or an acceleration sensor 243, and the gyro sensor 241 may sense information about the movement of the remote control device 200.

For example, the gyro sensor 241 can sense information about the operation of the remote control device 200 based on the x, y, and z axes, and the acceleration sensor 243 measures the moving speed of the remote control device 200. Information about such things can be sensed. Meanwhile, the remote control device 200 may further include a distance measurement sensor and can sense the distance from the display 180 of the display device 100.

The output interface 250 may output a video or audio signal corresponding to a manipulation of the user input interface 230 or a signal transmitted from the display device 100.

The user can recognize whether the output interface 250 is manipulating the user input interface 230 or controlling the display device 100.

For example, the output interface 250 includes an LED 251 that turns on when the user input interface 230 is manipulated or a signal is transmitted and received with the display device 100 through the wireless communication unit 225, and a vibrator 253 that generates vibration. ), a speaker 255 that outputs sound, or a display 257 that outputs an image may be provided.

In addition, the power supply circuit 260 supplies power to the remote control device 200, and power waste can be reduced by stopping power supply when the remote control device 200 does not move for a predetermined period of time.

The power supply circuit 260 can resume power supply when a predetermined key provided in the remote control device 200 is operated.

The memory 270 may store various types of programs, application data, etc. necessary for controlling or operating the remote control device 200.

When the remote control device 200 transmits and receives signals wirelessly through the display device 100 and the RF circuit 221, the remote control device 200 and the display device 100 transmit and receive signals through a predetermined frequency band. .

The controller 280 of the remote control device can store and reference information about the display device 100 paired with the remote control device 200 and the frequency band in which signals can be wirelessly transmitted and received in the memory 270.

The controller 280 controls all matters related to the control of the remote control device 200. The controller 280 sends a signal corresponding to a predetermined key operation of the user input interface 230 or a signal corresponding to the movement of the remote control device 200 sensed by the sensor 240 through the wireless communication unit 225. 100).

Additionally, the microphone 290 of the remote control device 200 can acquire voice.

A plurality of microphones 290 may be provided.

Next, Figure 4 will be described.

Figure 4 shows an example of utilizing the remote control device 200 according to an embodiment of the present invention.

Figure 4(a) illustrates that the pointer 205 corresponding to the remote control device 200 is displayed on the display 180.

The user can move or rotate the remote control device 200 up and down, left and right. The pointer 205 displayed on the display 180 of the display device 100 corresponds to the movement of the remote control device 200. This remote control device 200 can be called a ‘space remote control’ because the corresponding pointer 205 is moved and displayed according to movement in 3D space, as shown in the drawing.

Figure 4(b) illustrates that when the user moves the remote control device 200 to the left, the pointer 205 displayed on the display 180 of the display device 100 also moves to the left in response.

Information about the movement of the remote control device 200 detected through the sensor of the remote control device 200 is transmitted to the display device 100. The display device 100 can calculate the coordinates of the pointer 205 from information about the movement of the remote control device 200. The display device 100 may display the pointer 205 to correspond to the calculated coordinates.

Figure 4(c) illustrates a case where the user moves the remote control device 200 away from the display 180 while pressing a specific button in the remote control device 200. As a result, the selected area in the display 180 corresponding to the pointer 205 can be zoomed in and displayed enlarged.

Conversely, when the user moves the remote control device 200 closer to the display 180, the selected area in the display 180 corresponding to the pointer 205 is zoomed out and displayed in a reduced size. You can.

Meanwhile, when the remote control device 200 moves away from the display 180, the selected area may be zoomed out, and when the remote control device 200 approaches the display 180, the selected area may be zoomed in. .

Additionally, when a specific button in the remote control device 200 is pressed, recognition of up-down, left-right movement may be excluded. That is, when the remote control device 200 moves away from or approaches the display 180, up, down, left, and right movements are not recognized, and only forward and backward movements can be recognized. When a specific button in the remote control device 200 is not pressed, only the pointer 205 moves as the remote control device 200 moves up, down, left, and right.

Meanwhile, the moving speed or direction of the pointer 205 may correspond to the moving speed or direction of the remote control device 200.

Meanwhile, a pointer in this specification refers to an object displayed on the display 180 in response to the operation of the remote control device 200. Therefore, in addition to the arrow shape shown in the drawing as the pointer 205, objects of various shapes are possible. For example, the concept may include a point, a cursor, a prompt, a thick outline, etc. In addition, the pointer 205 can be displayed in correspondence to one of the horizontal and vertical axes on the display 180, as well as to multiple points, such as a line or surface.

Figures 5(a) and 5(b) are diagrams for explaining the landscape mode and portrait mode of a stand-type display device according to an embodiment of the present disclosure.

Referring to Figures 5(a) and 5(b), a stand type display device 100 is shown.

A shaft 103 and a stand base 105 may be connected to the display device 100.

The shaft 103 may connect the display device 100 and the stand base 105. The shaft 103 may extend vertically.

The lower end of the shaft 103 may be connected to the edge of the stand base 105.

The lower end of the shaft 103 may be rotatably connected to the circumference of the stand base 105.

The display device 100 and the shaft 103 may rotate about a vertical axis with respect to the stand base 105.

The upper part of the shaft 103 may be connected to the rear of the display device 100.

The stand base 105 may serve to support the display device 100.

The display device 100 may be configured to include a shaft 103 and a stand base 105.

The display device 100 may rotate around a point where the top of the shaft 103 and the rear of the display 180 come into contact.

Figure 5(a) shows that the display 180 is operating in a landscape mode in which the horizontal length is greater than the vertical length, and Figure 5(b) indicates that the display 180 operates in a landscape mode in which the vertical length is greater than the horizontal length. It can be indicated that it operates in landscape mode with .

Users can move around holding a stand-type display device. In other words, unlike a fixed device, the stand-type display device 100 has improved mobility and the user is not limited by its placement location.

Hereinafter, embodiments of a video processing method in the display device 100 according to the present disclosure will be described.

For convenience of explanation, an example will be given of providing an application execution screen with both a video area and a graphics area upon receipt of an application execution request. However, the present disclosure is not limited to this and is applicable to the processing of all types of video in which a video area and a graphics area exist.

The display device 100 may include a hardware configuration for processing a video area and a hardware configuration for processing a graphics area in a video path.

However, unlike processing of the graphics area, the hardware configuration for processing the video area requires processing such as storing and copying video frames during the processing of the video area, resulting in physical delay.

For example, when a change request for the video area and the graphics area is received, the change can be requested at the same time at the application level, but at the hardware level, the physical Due to the delay, the video may be delayed more than the graphics, resulting in problems with output or playback.

Therefore, compensation operation, that is, synchronization, is necessary for the video area and the graphics area being out of sync due to physical delay occurring at the hardware level.

Figure 6 is a block diagram of the display device 100 according to another embodiment of the present disclosure.

The display device 100 may include a display 180, a remote control device 200, and a processing unit 600.

In the following description of the display device 100, any content that overlaps with the content described above with reference to FIGS. 1 to 5 will be referred to and the overlapping description will be omitted. For example, the following description of the configuration and function of the remote control device 200 (eg, remote control) refers to the description of at least one of FIGS. 2 to 4 described above, and no redundant description is provided here.

Referring to FIG. 6, the processing unit 600 may include a memory 610, a decoder 620, a first processor 630, a second processor 640, and a control unit 650.

The processing unit 600 may receive user input through the remote control device 200.

Additionally, the processing unit 600 may exchange data by communicating with an application processor (AP) (not shown).

The memory 610 may store various data, for example, data received or processed by the processing unit 600. The memory 610 may have the same configuration as the memory 140 shown in FIG. 1 or may have a separate configuration.

The memory 610 may store video delay (or offset) information calculated by the processing unit 600.

The processing unit 600 or control unit 650 processes overall data communication, user interface configuration and provision, and control operations with the remote control device 200 (or other external input devices (not shown)). , control, and storage operations can be performed. For example, the control unit 650 may have the same configuration as the controller 170 of FIG. 1 .

The decoder 610 may decode video data corresponding to the video playback request signal.

The first processor 630 may receive video data information regarding video data decoded by the decoder 620. In this sense, the first processor 630 may correspond to a video processor.

The first processor 630 may calculate the video delay based on the video data information received from the decoder 620. The control unit 650 can control the video delay calculated through the surface manager to be stored in the memory 610.

The second processor 640 may perform processing on the graphics area. In this sense, the second processor 640 may correspond to a graphics processor.

The second processor 640 refers to the video delay calculated under the control of the control unit 650 (or surface manager), and processes and outputs the graphic data so that it is synchronized with the delayed output video data. .

7 and 8 are flowcharts shown to explain the operation method of the display device 100 according to an embodiment of the present disclosure.

Figure 7 explains a synchronization processing method for the video area and the graphics area according to an embodiment of the present disclosure.

Referring to FIG. 7, a video playback request signal may be received (S101).

When a video playback request signal is received through step S101, video data regarding the video area corresponding to the request signal may be decoded (S103).

Video delay (i.e., offset) may be calculated based on information about the video data decoded in step S103 (S105).

The video delay calculated for the video data in step S105 may be transmitted to a graphics thread for the graphics area (S107).

When the video delay information is transmitted to the graphics thread through step S107, a screen in which the video area and the graphics area are synchronized may be output (S109).

Figure 8 explains a synchronization processing method for the video area and the graphics area according to another embodiment of the present disclosure.

In particular, Figure 8 may describe a processing method when an event occurs while video is being output.

The display device 100 may receive an event signal (S201).

The display device 100 may check the characteristics of the event signal received in step S201 (S203).

If the characteristic of the event signal confirmed in step S203 is the first characteristic, the display device 100 may read the previously stored video delay (offset) from the memory 610 (S205).

At this time, the first characteristic may indicate a case in which no additional frames are required for video processing other than the number of frames used when calculating the offset related to the previously calculated video delay.

The display device 100 may transmit the video delay offset read in step S205 to a graphics thread for graphics area processing (S207).

When the video delay offset is transmitted to the graphics thread in step S207, the display device 100 synchronizes the video area and the graphics area to synchronize, and outputs a screen corresponding to the event signal received in step S201. It can be controlled as much as possible (S209).

Meanwhile, when the characteristic of the event signal confirmed in step S203 is the second characteristic, the display device 100 does not read the previously stored video delay offset from the memory 610 and use it as is, but rather reads the video data related to the video area. The video delay offset (new offset) can be recalculated (S211).

In the above, the second characteristic may indicate a case in which additional frames are required for video processing, unlike the first characteristic described above.

The display device 100 may transmit the video delay offset recalculated in step S211 to a graphics thread for graphics area processing (S213).

The display device 100 synchronizes the video area and the graphics area to be in sync based on the video delay offset transmitted to the graphics thread through step S213, and configures a screen corresponding to the event signal received in step S201. It can be controlled to be output (S215).

With reference to FIGS. 9 to 13, the embodiment(s) of the present disclosure shown in FIGS. 7 to 8 will be described in detail as follows.

9 to 11 are diagrams to explain a method of synchronizing a video area and a graphics area in a video processing process of the display device 100 according to an embodiment of the present disclosure.

Referring to (a) of FIG. 9, one image may be composed of a video area and a graphics area, as described above, and at the time of output, the video area and the graphics area must be configured and output to be in sync with each other.

However, when the display device 100 requests video playback at the application level, for example, at time t1, the data for the video area is decoded through the video decoder 620 and then final processed by the video processor 630. There is a physical delay until it is output.

The video processor 630 reports that the video is ready for output (a type of event) at the point when processing of the decoded video data is completed and output to the video area is possible (i.e., at time t1') or when preparation for output to the video area is completed. ) can be transmitted to the application level.

At the application level, when the fact that the video area is ready to be output (at time t1') is received from the video processor 630, processing of the graphics area can be requested from the graphics processor 640.

The graphics processor 640 may process and output graphics data for the graphics area according to an application level processing request (at time t1').

Through the above-described process, the display device 100 can output one image, that is, an image in sync in which the video area and the graphics area are synchronized, as shown in (b) of FIG. 9.

In other words, when the display device 100 completes processing of the video area including physical delay, it performs processing of the graphics area without physical delay, creating an image in which the video area and the graphics area are in sync. can be output. In other words, considering the physical delay, processing of the video area is requested first at time t1, and processing of the graphics area is requested secondly at time t1', and the physical delay is processed through control at the time of the processing request. By doing so, a screen that is in sync can be created at the time of output.

However, referring to FIG. 10, when using the method of FIG. 9, processing of the next setting, etc. may be restricted at the application level before image processing according to control such as one setting (eg, operation) is completed.

Referring to Figures 10 (a) and (b), at the application level, for example, when image processing according to one setting is requested at time t1, Figure 9 shows the video area containing the physical delay. When processing is completed, the processing operation for the graphics area is performed to ensure synchronization. In this case, if the physical delay is long or the processing of the video area in which it is reflected is performed at a specific point in time (for example, at or after t2), ) If it is not processed before, image processing according to other settings that were originally requested at time t2 will not be possible or will inevitably be delayed.

For example, assuming that the processing speed for one video must operate at 30 fps, if the physical delay in the video area is longer than 33.3 ms, 30 fps cannot be satisfied. This assumption can occur frequently during processing of high-definition images. Therefore, considering the situation shown in FIG. 10, supplementation to the method shown in FIG. 9 may be required.

In Figures 11 (a) and (b), as a supplement to the method of Figure 9, the physical delay offset occurring during the processing of the video area is transmitted in advance to the graphics thread for graphics area processing, thereby processing the graphics area. A method of synchronizing the video area and the graphics area to be synchronized by being shifted (or delayed) and processed based on the offset is disclosed.

Referring to (b) of FIG. 11, at the application level at time t1, the physical delay offset that will occur in the video area is also transmitted to the graphics area, so processing for the video area starts from time t1, but processing for the graphics area is at t1. ' By starting at the point, the two areas can be processed to be in sync.

Meanwhile, according to another embodiment of the present disclosure, the image processing speed may be set differently depending on various circumstances such as the screen mode of the display device and the type or attribute of the content, and the display device 100 is set to one setting. If the physical delay occurring in the process of processing the calculated video area is greater than the time point for different settings, the method shown in FIG. 11 may be applied. Otherwise, the method shown in FIG. 9 may be applied.

12 and 13 are diagrams to explain the video processing operation of the display device 100 according to an embodiment of the present disclosure.

Referring to FIG. 12, a method of processing images by synchronizing video and graphics on the execution screen of an application downloaded and installed on the display device 100 will be described.

For convenience of explanation, the application is described as an example that is not a native application built in during manufacturing of the display device 100, but is not limited thereto.

The display device 100 may execute the application according to the user's request and provide an execution screen. For example, assume that the user moves to the Music Tab.

In this case, when the display of the display device 100 focuses on a specific music video thumbnail, the video may be played in a thumbnail state.

After the video is decoded in the decoder driver, information about the video may be transmitted to the video driver, that is, the video scaler. At this time, the information about the video may include, for example, information such as 1080p and 60Hz in the case of FIG. 12.

In a video scaler, it can be calculated by calculating the physical delay offset. As a video scale, the physical delay offset can be calculated based on video information transmitted from the above-described decoder driver. For example, the video scaler requires 4 frames of video data for video processing, and if the input v-sync of the video is 60Hz, 60Hz is 16.67ms per frame, so for the 4 frames of video data, ultimately, It can be calculated that a delay offset of 4 x 16.67 = 66.66ms occurs.

In this way, the physical delay offset calculated from the video scaler, that is, 66.66ms, can be reported to the Surface Manager, saved, and delivered to the graphics thread of the graphics driver.

Since the display device 100 shares the delay information of the video, when the thumbnail of the corresponding video is clicked, the video size can smoothly increase and move. In other words, it can be expanded to full screen. In this case, the video and graphics must be in sync. In order to sync with the video, the graphics thread applies the delay offset, that is, 66.66ms, received in advance every time the graphics are changed, so that a synchronized image is output during the application execution process. can do.

13, the application installed on the display device 100 is not directly executed as shown in FIG. 12, but a web browser is executed, and after moving to a site corresponding to the application, a specific This explains how to control the operation when playing a video.

After video decoding in the decoder driver, when video information is delivered to the video scaler, the video scaler can calculate the physical delay for video processing compared to graphics by referring to the received video information.

As described above, the video scaler requires 4 frames of video data for video processing. However, in Figure 13, it is assumed that the video input v-sync is 30Hz, but it is not limited to this. Here, 30Hz requires 33.33ms per frame, which means 4 frames of video are required, so it can be calculated that a delay offset of 4 x 33.33 = 133.33ms occurs. The delay offset calculated in this way (i.e., 133.33ms) can be passed to the graphics thread.

At this time, as shown in (a) to (b) of FIG. 5, the display device 100 adopts a display whose display panel can be rotated, and the display panel is rotated 90 degrees at the user's request, that is, the image is displayed. When output horizontally and then vertically, the video scaler can calculate additional delay according to the image rotation. Here, if 1 more frame of video data is needed to rotate the image, 1 frame at the aforementioned 30Hz causes a delay of 33.33ms, so the sum of the newly generated delay and the existing delay is the final operation delay offset (total 33.33). + 133.33 = 166.66ms) can be computed and passed to the graphics thread.

The display device 100 moves the video position by scrolling. For example, in order to move the screen smoothly when requesting a movement from top to bottom, the video and graphics must be synchronized during the operation, and for this, the graphics thread must synchronize with the video. To match the delay offset of 166.66ms received in advance, the delay offset of 166.66ms can be applied every time the graphics are changed.

Even if not specifically mentioned, the order of at least some of the operations disclosed in this disclosure may be performed simultaneously, may be performed in an order different from the previously described order, or some may be omitted/added.

According to an embodiment of the present invention, the above-described method can be implemented as processor-readable code on a program-recorded medium. Examples of media that the processor can read include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage devices.

The display device described above is not limited to the configuration and method of the above-described embodiments, and the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made. It may be possible.

According to the display device according to the present disclosure, the video area in which physical delay occurs and the corresponding graphics area can be easily synchronized in software without adding hardware, thereby producing an image in which the video area and the graphics area are in sync with each other. It can be provided to users and has industrial applicability.

Claims (14)

Receiving a video playback request signal; decoding video data; Receiving information of decoded video data and calculating an offset regarding video delay; Passing the calculated offset related to the video delay to the graphics thread; and Including controlling to output a screen by synchronizing video and graphics, How the display device operates. According to paragraph 1, Further comprising the step of storing an offset related to the calculated video delay, How the display device operates. According to paragraph 2, Receiving an event signal; and Further comprising the step of checking the characteristics of the received event signal, How the display device operates. According to paragraph 3, If the characteristics of the confirmed received event signal are the first characteristics, reading the stored offset and transmitting it to a graphics thread; and Further comprising controlling to synchronize video and graphics to output a screen according to the event signal, How the display device operates. According to paragraph 4, The first characteristic is, Indicates an event signal that does not require additional frames for video processing other than the number of frames used when calculating the offset for the calculated video delay, How the display device operates. According to paragraph 3, If the characteristics of the confirmed received event signal are the second characteristics, recalculating an offset related to the video delay; storing an offset for the recalculated video delay and passing it to the graphics thread; and Further comprising controlling to synchronize video and graphics to output a screen according to the event signal, How the display device operates. According to clause 6, The second characteristic is, Indicates an event signal that requires additional frames for video processing in addition to the number of frames used when calculating the offset for the calculated video delay, How the display device operates. a video decoder that decodes video data when a video playback request signal is received; a video processor that receives information from decoded video data and calculates an offset regarding video delay; a graphics processor that processes graphics data for the decoded video data; and Comprising a control unit that transmits the calculated offset related to the video delay to the graphics thread of the graphics processor and controls a screen in which video and graphics are synchronized to be output, Display device. According to clause 8, Further comprising a memory for storing an offset related to the calculated video delay, Display device. According to clause 9, The control unit, When an event signal is received, check the characteristics of the received event signal, Display device. According to clause 10, The control unit, If the confirmed characteristics of the received event signal are the first characteristics, the stored offset is read and transmitted to the graphics thread of the graphics processor, and the video and graphics are synchronized to control the screen according to the event signal to be output. Display device. According to clause 11, The first characteristic is, Indicates an event signal that does not require additional frames for video processing other than the number of frames used when calculating the offset for the calculated video delay, Display device. According to clause 10, The control unit, If the confirmed characteristics of the received event signal are the second characteristics, the video processor is controlled to recalculate the offset regarding the video delay, and the video processor is controlled to store the offset regarding the video delay recalculated in the memory. and transmitting it to the graphics thread of the graphics processor to synchronize video and graphics and control the screen to be output according to the event signal. Display device. According to clause 13, The second characteristic is, Indicates an event signal that requires additional frames for video processing in addition to the number of frames used when calculating the offset for the calculated video delay, Display device.

Images