RU2808678C1 - Method, equipment and electronic device for processing display window overlay - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: technical result is achieved due to the fact that in the process of overlaying the image of the next window on the main window, an image of the displayed content of the next window is captured at the image capture time interval; and the captured image is displayed at a display position corresponding to the next window in the main window, the method further includes the step of determining the image capturing time interval based on the displayed content of the next window; when the display level of the main window is higher than the display level of the next window, the main window still shows the captured image corresponding to the display content of the next window.

EFFECT: improving the display quality of overlapping windows by eliminating the flickering of the overlapping window.

7 cl, 10 dwg

Description

This application claims priority to Chinese Provisional Patent Application No. 202010062553.5 filed with the China National Intellectual Property Office (China Patent Office) on January 19, 2020, entitled "METHOD, APPARATUS AND ELECTRONIC DEVICE FOR PROCESSING DISPLAY WINDOW OVERLAYING", which is incorporated by reference herein in its entirety.

Field of technology to which the present invention relates

The present invention relates to the technical field of images and, in particular, to a method, apparatus and electronic device for processing display window overlay.

BACKGROUND OF THE INVENTION

As electronic device technology has advanced, various types of application programs (APP) have been installed into an electronic device. Such application programs include a first application program and a second application program. The first application program is a program that runs in the task manager of the electronic device. The first application does not by default create an independent display window, but it needs to establish a connection with the second application in the electronic device and create the next window upon request by interacting with the second application using instructions. The next window can then be overlaid on top and shown in the main window corresponding to the second application.

In the prior art, the first application program monitors in real time the display position change information of the main window corresponding to the second application program, and in real time adjusts the display position of the next window corresponding to the first application program according to the display position change information, so that The display position of the next window changes as the display position of the main window changes. Since the next window corresponds to the first application program and the main window corresponds to the second application program, and they are two different application programs in the electronic device, when the user clicks the main window corresponding to the second application program, the display level of the main window will be higher than the display level next window corresponding to the first application program, so that the main window is superimposed on top of the next window. When the monitoring detects that the display level of the main window is higher than the display level of the next window, the first application program will reset the display level of the next window to a higher level than the display level of the main window so that the user can view the next window again.

It can be seen that in the prior art, when the display level of the main window is higher than the display level of the next window, the main window will be superimposed on top of the next window. Although the first application subsequently resets the display level of the next window to a higher level than the display level of the main window, this visually appears to the user as a blinking process of the window, resulting in poor display quality of the next window.

Brief Disclosure of the Present Invention

According to embodiments of the present invention, they provide a method, apparatus and electronic device for display window overlay processing to solve the problem of poor display quality of the next window in the prior art.

According to a first feature of an embodiment of the present invention, there is provided a method for processing a display window overlay, comprising the steps of:

in the process of overlaying the image of the next window on the main window, an image of the displayed content of the next window is captured according to the image capture time interval;

The captured image is displayed in the display position corresponding to the next window in the main window.

According to a possible implementation, before capturing an image of the displayed content of the next window according to the image capturing time interval, the method further includes the step of:

Determines the image capture time interval based on the displayed content of the next window.

According to a possible implementation, determining the image capture time interval based on the displayed content of the next window involves the steps of:

if the displayed content of the next window includes non-video content, the image capture time interval is determined to be equal to the predetermined pop time interval; or,

If the displayed content of the next window includes video content, the image capture time interval is determined based on the frame rate of the video content.

According to a possible implementation, determining the image capture time interval based on the frame rate of the video content involves the steps of:

determining the display duration of each video image based on the frame rate of the video in the video content;

determining whether the display duration of each video image is longer than a predetermined popping time interval;

if the display duration of each video image is longer than the predetermined popping time interval, then the image capture time interval is determined to be equal to the predetermined popping time interval.

if the display duration of each video image is not longer than the predetermined popping time interval, then the display duration of each video image is rounded down to the nearest integer to obtain an image capture time interval; or,

If the display duration of each video image is no longer than the predetermined pop time interval, then the display duration of each video image is rounded down to the nearest multiple of 10 ms to obtain the image capture time interval.

According to a possible implementation, if the next window includes a plurality of sub-windows, each of which is used to play the corresponding sub-video content, the video frame rate of the video content is determined to be equal to the maximum video frame rate among the video frame rates of all sub-video contents corresponding to the plurality of sub-windows.

According to a possible implementation, a method for capturing an image of the displayed content of the next window according to an image capturing time interval includes the steps:

if it has been detected that the next window is in focus, then an image of the displayed content of the next window is captured according to the image capture time interval;

if the next window has focus, then the display level of the next window is higher than the display level of the main window.

According to a possible implementation, a method for capturing an image of the displayed content of the next window according to an image capturing time interval includes the step of:

if it has been observed that the number of sub-windows contained in the next window is greater than a predetermined value, then an image of the displayed content of the next window is captured according to the increased image capture time interval.

According to a possible implementation, the method further includes the step of:

If it has been detected that the main window is in focus, then image capture of the displayed content of the next window is stopped according to the image capture time interval.

According to a possible implementation, the method further includes the step of:

if it is detected that the display level of the next window is greater than the display level of the main window after it has been detected that the main window is in focus, image capturing of the display content of the next window is stopped according to the image capturing time interval.

According to a possible implementation, the main window is a browser display window in an electronic device; and/or,

The following window is the electronic device local service video display window.

According to a second feature of an embodiment of the present invention, there is provided a display window overlay processing apparatus, including:

an image capturing module configured to, in the process of overlaying the image of the next window on the main window, capture an image of the displayed content of the next window according to the image capturing time interval;

a display module configured to display the captured image at a display position corresponding to the next window in the main window.

According to a third feature of an embodiment of the present invention, a display window overlay processing apparatus is provided, including an application program for a next window and an application program for a main window;

wherein the application program for the next window is configured, in the process of overlaying the image of the next window on the main window, to capture an image of the displayed content of the next window according to the image capture time interval, and push out the captured image to the application program for the main window;

the application program for the main window is configured to display the captured image at a display position corresponding to the next window in the main window.

According to a possible implementation, the application program for the next window is further configured to:

before capturing an image of the display content of the next window according to the image capture time interval, determining the image capture time interval based on the display content of the next window.

According to a possible implementation, the application program for the following window is specifically configured to:

if the displayed content of the next window includes non-video content, the image capture time interval is determined to be equal to the predetermined pop time interval; or,

If the displayed content of the next window includes video content, the image capture time interval is determined based on the frame rate of the video content.

According to a possible implementation, the application program for the following window is specifically configured to:

determining the display duration of each video image based on the frame rate of the video in the video content;

determining whether the display duration of each video image is longer than a predetermined popping time interval;

if the display duration of each video image is longer than the predetermined popping time interval, then the image capture time interval is determined to be equal to the predetermined popping time interval.

if the display duration of each video image is not longer than the predetermined popping time interval, then the display duration of each video image is rounded down to the nearest integer to obtain an image capture time interval; or,

If the display duration of each video image is no longer than the predetermined pop time interval, then the display duration of each video image is rounded down to the nearest multiple of 10 ms to obtain the image capture time interval.

According to a possible implementation, if the next window includes a plurality of sub-windows, each of which is used to play the corresponding sub-video content, the video frame rate of the video content is determined to be equal to the maximum video frame rate among the video frame rates of all sub-video contents corresponding to the plurality of sub-windows.

According to a possible implementation, the application program for the next window is specifically configured to:

if it has been detected that the next window is in focus, then an image of the displayed content of the next window is captured according to the image capture time interval;

if the next window has focus, then the display level of the next window is higher than the display level of the main window.

According to a possible implementation, the application program for the following window is specifically configured to:

if it has been observed that the number of sub-windows contained in the next window is greater than a predetermined value, then an image of the displayed content of the next window is captured according to the increased image capture time interval.

According to a possible implementation, the application program for the next window is further configured to:

If it has been observed that the main window is in focus, it stops capturing an image of the displayed content of the next window according to the image capturing time interval, and stops pushing the captured image to the application program for the main window.

According to a possible implementation, the application program for the next window is further configured to:

if it has been detected that the display level of the next window is greater than the display level of the main window after it has been detected that the main window is in focus, then the image capture of the display content of the next window is stopped according to the image capture time interval, and the ejection of the captured image is stopped into the application program for the main window.

According to a possible implementation, the main window application is a browser on an electronic device; and/or the main window is a browser display window; and/or,

The application program for the next window is the local service of the electronic device, and the next window is the video display window of the local service.

According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device comprising a processor and a memory;

wherein the memory is configured to store program instructions;

the processor is configured to recall and execute the program instructions stored in the memory, and the electronic device is configured to execute the method according to any embodiment of the first feature where the processor executes the program instructions stored in the memory.

According to a fifth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium on which instructions are stored, the instructions, when executed by a computer, causing the computer to execute a method according to any embodiment of the first aspect.

According to a sixth aspect of an embodiment of the present invention, there is provided a computer program that, when executed on a computer, causes the computer to execute a method according to any embodiment of the first aspect.

According to the display window overlay processing method, apparatus and electronics according to embodiments of the present invention, in the process of displaying the next window overlay on the main window, the display content image of the next window is captured according to the image capture time interval; and the captured image is displayed at a display position corresponding to the next window in the main window, so that when the display level of the main window is higher than the display level of the next window, that is, when the main window is superimposed on top of the next window, the main window still displays the captured image corresponding the displayed content of the next window. In this way, the user will think that he/she is still viewing the displayed content of the next window and will not experience visible flickering, which improves the display quality of the next window and improves the user's experience when viewing the next window.

Brief description of the figures

In fig. 1 is a simplified diagram of a scenario of the present invention according to an embodiment of the present invention;

In fig. 2 is a first block diagram of a local service displaying a video window in a browser according to the following method according to the prior art;

In fig. 3 is a second block diagram of a local service displaying a video window in a browser according to the following method according to the prior art;

In fig. 4 shows a simplified display diagram when a user clicks on a browser according to the prior art;

In fig. 5 is a flowchart of a display window overlay processing method according to an embodiment of the present invention;

In fig. 6 is a simplified display diagram showing when a user clicks on a browser according to an embodiment of the present invention;

In fig. 7 is a flowchart of a display window overlay processing method according to another embodiment of the present invention;

In fig. 8 is a block diagram of a display window overlay processing apparatus according to an embodiment of the present invention;

In fig. 9 is a block diagram of a display window overlay processing apparatus according to another embodiment of the present invention;

In fig. 10 is a simplified diagram of an electronic device according to an embodiment of the present invention.

Detailed Disclosure of the Present Invention

First, scenarios of the present invention and some terms associated with embodiments of the present invention are described herein.

In fig. 1 is a simplified diagram of a scenario of the present invention according to an embodiment of the present invention; As shown in FIG. 1, application program A and application program B may be installed in the electronic device mentioned in the embodiment of the present invention, where application program A may be called the next window application program, and application program B may be called the main window application program.

As an example, application program A may be a program that runs in the task manager of an electronic device. Application A does not by default create an independent display window, but it needs to establish a connection with Application B on the electronic device and create the next window on demand while interacting with Application B using instructions. Then, the next window can be superimposed and displayed on the main window corresponding to application program B, so that the next window is superimposed and displayed on the main window according to the following method.

You can see that after the next window is created, the display level of the next window is higher than the display level of the main window before the main window has focus.

According to this embodiment of the present invention, the next window can be superimposed and displayed on the main window according to the following method, which means that the next window moves along with the display position of the main window.

As an example, application A or the application for the next window may be a local service of the electronic device, and accordingly, the next window may be a video display window of the local service. Application B or the main window application may be a browser in the electronic device, and accordingly, the main window may be a browser display window.

The connection established between application A and application B according to an embodiment of the present invention may be, inter alia, a full duplex communication protocol (Web Socket, WS) connection based on Transmission Control Protocol (TCP)/full duplex protocol secure data transfer (secure web socket, WSS) based on the TCP protocol, or a long-polling connection with the Hypertext Transfer Protocol (HTTP)/Hypertext Transfer Protocol Secure (HTTPS).

The electronic device mentioned in the embodiments of the present invention may be, but is not limited to, a desktop computer, an all-in-one computer, a laptop, a PDA, or a tablet computer.

The browser mentioned in the embodiments of the present invention may be, but is not limited to, Internet Explorer (IE), Google Chrome, Apple Safari, Firefox browser (Mozilla Firefox, FX), and the like.

As for the image capture time interval mentioned in the embodiments of the present invention, the image capture time interval of the displayed content of the next window in the electronic device can be used as it.

Image capture means converting all or part of the content displayed on the screen into a picture.

Referred to in embodiments of the present invention, the following window may contain one sub-window or multiple sub-windows (this is equivalent to dividing the entire window into multiple sub-windows), each sub-window being separately used to display the corresponding content. For example, the following window contains a subwindow subwindow and subwindow and subwindow and the subwindow used to display content 1, subwindow used to display content 2, subwindow is used to display content 3, and the subwindow used to display content 4.

Referred to in embodiments of the present invention, focusing on the next window means that the user has selected the next window using an input device such as a mouse, or the user has selected the next window using a touch. It can be understood that when a user selects a window corresponding to a different application program than the next window using an input device such as a mouse, or when a user selects a window corresponding to a different application program than the next window using a touch, then the following the window goes out of focus.

After detecting that the next window is in focus, the electronic device keeps the display level of the next window higher than the display level of the main window.

When referred to in embodiments of the present invention, focusing on the main window means that the user has selected the main window using an input device such as a mouse, or the user has selected the main window using a touch. It can be understood that when the user selects a window corresponding to a different application program than the main window using an input device such as a mouse, or when the user selects a window corresponding to a different application program than the main window using a touch, then the main the window goes out of focus.

In the case where the display level of the next window is higher than the display level of the main window, if it has been detected that the main window is in focus, the electronic device first adjusts the display level of the main window to be higher than the display level of the next window; then the electronic device adjusts the display level of the next window to be higher than the display level of the main window; if the main window subsequently continues to be in focus after the display level of the next window has been reset to be higher than the display level of the main window, the display level of the next window in the above-mentioned electronic device is held higher than the display level of the main window.

Numerical definitions of “first” and “second” in embodiments of the present invention are used only to distinguish like objects, not necessarily to describe a particular order or sequence, and they should not be considered to impose any limitation on the embodiments of the present invention.

In fig. 2 is a block diagram 1 of a local service displaying a video display window in a browser according to the following method according to the related art; in fig. 3 is a block diagram 2 of a local service displaying a video display window in a browser according to the following method according to the prior art. As shown in FIG. 2 and fig. 3, after the browser in the electronic device establishes a connection with the above-mentioned local service, the above-mentioned local service will return to the browser a universally unique identifier (UUID), which is used as an identifier when subsequent protocol-based communication uses instructions between the local service and the browser . In the following description, an example video display window is used as a preview window. The browser will change the title bar of the browser window to the UUID, and using the request, send instructions to the local service to create a preview window. In this case, the instructions can pass the above-mentioned UUID, parameters for the position of the preview window, parameters for the size of the preview window, etc. Other information may also be transmitted in the instructions mentioned above without any restrictions. After being instructed to create a preview window, the local service traverses all browser windows on the desktop and determines whether there is a window whose name is the same as the UUID; if so, it uses a match to find the browser window for the browser, and at the same time obtains a handle to the browser window. The local service creates a preview window according to the position parameters and size parameters of the preview window passed in the instructions for creating the preview window, and monitors the display position change information of the browser window to adjust the display position of the preview window in real time according to the change information display position of the browser window, so that the display position of the preview window changes with the display position of the browser window.

A browser window, that is, a browser's display window, can be understood as the main window corresponding to the browser. The preview window, that is, the video display window, can be understood as the following window corresponding to the local service. The window title is the information in the title bar of the browser window.

In fig. 4 shows a simplified display diagram when a user clicks on a browser window in accordance with the prior art. As shown in FIG. 4, Since the local service and the browser are two different application programs in the electronic device, when the user clicks on the browser display window, the display level of the browser display window will be adjusted above the display level of the video display window, so that the browser display window will be superimposed on top of the video display window . When the tracking detects that the display level of the browser display window is higher than the display level of the video display window, the local service will reconfigure the display level of the video display window to a higher level than that of the browser display window, so that the user can view the video display window again.

It can be seen that in the prior art, when the display level of the browser display window is higher than the display level of the video display window, the browser display window will be superimposed on top of the video display window. Although the local service resets the display level of the video display window to a higher level than that of the browser display window, to the user it visually appears as a window blinking process, which results in the video display window being poorly displayed in the next mode or so-called next window.

According to the display window overlay processing method, apparatus and electronics according to embodiments of the present invention, in the process of displaying the next window overlay on the main window, the display content image of the next window is captured according to the image capture time interval; and the captured image is displayed at a display position corresponding to the next window in the main window, so that when the display level of the main window is higher than the display level of the next window, that is, when the main window is superimposed on top of the next window, the main window still displays the captured image corresponding the displayed content of the next window. In this way, the user will think that he/she continues to view the displayed content of the next window, and will not feel visible flickering, this solves the technical problem of poor display quality of the video display window in the following method or the so-called next window in the current art.

The technical solution proposed in the embodiment of the present invention and how the technical solution proposed in the embodiment of the present invention solves the technical problem will be explained in detail below in specific implementations. The following specific embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

In fig. 5 is a flowchart of a display window overlay processing method according to an embodiment of the present invention. An embodiment of an embodiment of the present invention may be an electronic device or display window overlay processing apparatus. For example, the display window overlay processing apparatus may be implemented in software and/or hardware. As shown in FIG. 5, the method according to an embodiment of the present invention may include the step of:

S501, in the process of overlaying the image of the next window on the main window, an image of the displayed content of the next window is captured according to the image capture time interval.

At this stage, in the process of superimposing the image of the next window on the main window in the electronic device, the electronic device may regularly perform image capture of the display content of the next window according to the image capturing time interval to obtain a picture containing the display content of the next window. The image capture time interval may be a capture time interval preset in the electronic device, or may be a capture time interval predetermined before the electronic device executes step S501. The image capture time interval may also be an image capture time interval obtained in a different manner, which is not limited in any way in this embodiment of the present invention.

According to an embodiment of the present invention, capturing a next window display content image can be understood as capturing an image of the next window display content to obtain a picture containing the next window display content.

As an example, the following window may be a video display window of a local service in an electronic device; and/or the main window may be a browser display window on the electronic device. It can be understood that the next window and the main window may also be other display windows in the electronic device, which is not limited in any way in this embodiment of the present invention.

According to an embodiment of the present invention, if the next window contains one subwindow, the captured image of the content of the next window contains a captured image corresponding to the displayed content of the subwindow. If the next window contains a plurality of sub-windows, the captured image of the displayed content of the above-mentioned next window contains a captured image corresponding to the displayed content of the plurality of sub-windows.

If necessary, in the process of superimposing the image of the next window onto the main window in the electronic device, the application program for the next window in the electronic device can capture an image of the display content of the next window according to the image capture time interval, and push the captured image to the application program for the main window in the electronic device so that to show the captured image in the display position corresponding to the next window in the main window. Thus, even if the display level of the main window is higher than the display level of the next window, since the main window still shows the captured image corresponding to the displayed content of the next window, the user will think that he/she is still viewing the displayed content of the next window and will not experience the visible blinking.

According to this embodiment of the present invention, each time the application for the next window receives one captured image, it pushes the captured image to the application for the main window. Therefore, the image capture time interval can be understood as the time interval when the application program pushes the captured image to the application program for the main window. For ease of understanding, the time interval when the application for the next window pushes the captured image into the application for the main window is called the push time interval for simplicity.

As an example, if the application program for the next window is a local service in an electronic device, then the next window is a video display window of the local service. And/or, if the application for the main window may be a browser on an electronic device, then the main window will be the browser display window. It can be understood that the next window application and the main window application may also be other application programs in the electronic device. Accordingly, the next window and the main window may also be other display windows in the electronic device, which is not limited in any way in this embodiment of the present invention.

According to a possible embodiment, after the electronic device creates the next window, the next window is overlaid on top and displayed on the main window. In this case, the electronic device can capture an image of the displayed content of the next window according to the image capture time interval regardless of whether the next window has been monitored to be in focus. In this way, the captured image displayed in the main window can be continuously updated so that the most realistic captured image can be displayed in the main window.

According to another possible embodiment, after the electronic device creates the next window, the next window is superimposed and displayed on the main window, and the electronic device captures an image of the display content of the next window according to an image capture time interval. Subsequently, the electronic device can monitor that the next window is in focus, and that the main window is in focus. In this case, in order to save resource consumption, the electronic device may stop image capturing of the displayed content of the next window according to the image capturing time interval when the main window was observed to be in focus. Subsequently, when it is detected that the next window is in focus, the electronic device may trigger image capture of the displayed content of the next window according to the image capture time interval. This way, you can not only eliminate the problem of visible flickering when the main window is subsequently tracked as being back in focus, but also ensure that the most realistic captured image is displayed.

As an example, an application program for the next window in an electronic device may capture an image of the displayed content of the next window according to the image capture time interval when it was observed that the user selected the next window.

Optionally, if it has been observed that the number of sub-windows contained in the next window is greater than a preset value (for example, 9 or 16, etc.), the displayed content of each of the sub-windows is relatively small, and the user's visual sensitivity will be correspondingly low. At this time, the electronic device may increase the image capture time interval of the displayed content of the next window according to the increased image capture time interval. Thus, the display quality of the next window will not be degraded and there will be further savings in consumed resources.

The above preset value can be adjusted according to actual requirements, which is not limited in any way.

It can be understood that to ensure the action of pushing the captured image of the displayed content of the next window, the image capturing time interval cannot exceed the pop time interval threshold, and that the extended image capturing time interval cannot exceed the pop time interval threshold.

The push time interval threshold can be understood as the maximum push time interval that satisfies the visual viewing requirements. When visual viewing requirements are low, the ejection time interval threshold can be adjusted to a higher value; when visual viewing requirements are high, the push time interval threshold can be adjusted to a lower value.

S502, the captured image is displayed at the display position corresponding to the next window in the main window.

According to an embodiment of the present invention, after capturing an image of the display content of the next window according to the image capturing time interval, the electronic device will display the captured image at a display position corresponding to the next window in the main window. That is, the electronic device regularly captures the display content image of the next window according to the image capture time interval, and updates the displayed captured image at a display position corresponding to the next window in the main window with the captured image with the display content of the next window. Thus, even if the display level of the main window is higher than the display level of the next window, that is, when the main window is superimposed on top of the next window, since the main window is still showing the captured image corresponding to the displayed content of the next window, the user will think that he/she continues to view the displayed content of the next window, and will not experience any visible flickering.

As an example, while receiving a captured image pushed by the next window application, the main window application in the electronic device may display the captured image at a display position corresponding to the next window in the main window.

According to an embodiment of the present invention, when the monitoring detects that the display level of the main window is higher than the display level of the next window, that is, the main window is superimposed on top of the next window, the electronic device may also reset the display level of the next window to a higher level than the next window. display of the main window, that is, the next window is superimposed on top of the main window, so that the user can view the next window again.

As an example, the electronic device may call a preset system interface (display level setting interface) to implement display level setting. Moreover, methods of the existing state of the art can be used as specific methods for calling the interface.

For ease of understanding, according to the next embodiment of the present invention, the next window is considered as a local service video display window and the main window is considered as a browser display window as an example to introduce the display window overlay processing method proposed according to the embodiment of the present invention.

In fig. 6 is a simplified display diagram showing when a user clicks on a browser according to an embodiment of the present invention; As shown in FIG. 6, Since the local service and the browser are two different application programs in the electronic device, when the user clicks on the browser display window, the display level of the browser window will be adjusted above the display level of the local service video display window, so that the browser display window will be superimposed on top of the display window local service video. However, since the browser display window is still showing the captured image corresponding to the displayed content of the video display window, the user will think that he/she is still viewing the displayed content of the local service video display window. At the same time, when the tracking detects that the display level of the browser display window is higher than the display level of the video display window, the electronic device will reset the display level of the video display window to a higher level than that of the browser display window, so that the user can again view the local service video display window. It can be seen that even if the display level of the browser display window is higher than the display level of the video display window, that is, when the browser display window is superimposed on top of the local service's video display window, because the browser display window still shows the captured image corresponding to the display content of the next window, the user will think that he/she is still viewing the displayed content of the next window and will not experience any visible flickering.

Briefly, in the process of displaying an overlay of the next window on the main window, an image of the display content of the next window is captured according to the image capture time interval; and the captured image is displayed at a display position corresponding to the next window in the main window, so that when the display level of the main window is higher than the display level of the next window, that is, when the main window is superimposed on top of the next window, the main window still displays the captured image corresponding the displayed content of the next window. In this way, the user will think that he/she is still viewing the displayed content of the next window and will not experience visible flickering, which improves the display quality of the next window and improves the user's experience when viewing the next window.

It can be understood that when a change in the display position of the main window is monitored, the electronic device can also adjust the display position of the next window in the main window in real time according to the information about the change in the position of the main window, so that the next window moves when the display position of the main window changes.

Optionally, after detecting that the main window is in focus, the electronic device adjusts the display level of the next window higher than the display level of the main window, and therefore the next window continues to be superimposed on top of the main window so that the user can continue to view the displayed content of the next window. Therefore, the electronic device can stop image capturing of the displayed content of the next window according to the image capturing time interval, thereby further saving consumed resources.

According to an embodiment of the present invention, when it has been detected that the main window is in focus, the electronic device immediately stops image capturing of the displayed content of the next window according to the image capturing time interval.

When tracking detects that the main window is in focus, the electronic device first adjusts the main window's display level higher than the next window's display level, but then resets the next window's display level to a higher level than the main window's display level. When the electronic device detects that the main window is in focus, in a process where the electronic device adjusts the display level of the main window higher than the display level of the next window, and resets the display level of the next window to a higher level than the display level of the main window, the main window shows the captured image corresponding to the displayed content of the next window until the main window comes into focus. After the electronic device resets the display level of the next window to a higher level than the display level of the main window, the next window will be superimposed on top and will be displayed again on the main window. Thus, when the electronic device monitors that the main window is in focus, even if image capturing of the displayed content of the next window is stopped, the displayed content of the next window will always be shown on the main window and there will be no problem of visible flickering, and further savings in resource consumption will occur.

As an example, when it is detected that the user has selected the main window, the application program for the next window in the electronic device may stop capturing an image of the display content of the next window according to the image capturing time interval, and stop pushing the captured image to the application program for the main window, thereby additionally saving consumed resources.

If necessary, when it has been monitored that the display level of the next window is greater than the display level of the main window after it has been monitored that the main window is in focus, the electronic device stops image capture of the display content of the next window according to the image capture time interval to ensure displaying the most realistic captured image in the main window.

Then, when it is detected that the next window is in focus, the electronic device will trigger image capture of the displayed content of the next window according to the image capture time interval.

According to the above embodiment, before executing step S501, the electronic device can also determine an image capture time interval based on the display content of the next window, so that the electronic device can regularly capture an image of the display content of the next window according to the image capture time interval, and display the captured image at the position display corresponding to the next window in the main window.

As an example, an application program for the next window in an electronic device can determine an image capture time interval based on the display content of the next window, so that the application program for the next window can regularly capture an image of the display content of the next window according to the image capture time interval, and eject the captured image to a main window application program in the electronic device, such that the main window application program displays the captured image at a display position corresponding to the next window in the main window.

In the following embodiments of the present invention, a possible implementation of “determining an image capture time interval based on the displayed content of the next window” is explained in detail.

According to a possible implementation, if the displayed content of the next window contains non-video content, the electronic device may determine an image capture time interval based on a predetermined ejection time interval. As an example, the electronic device may determine that the image capture time interval is equal to a predetermined ejection time interval (eg, 100 ms). According to another possible embodiment, if the displayed content of the next window includes video content, the electronic device can determine the image capture time interval based on the frame rate of the video to reduce visible differences caused by changing images in the video content and improve the user experience.

The predetermined eject time interval can be understood as the default time interval for the electronic device to display the captured image in the display position corresponding to the next window in the main window, or as the default time interval for the application program for the next window to eject the captured image to the application program for main window. The default ejection time interval can be adjusted according to the visual requirements and technical specifications of the electronic device. For example, the default time interval could be 100 ms, 150 ms, 200 ms, etc.

As an example, if the next window contains one sub-window, the electronic device may take the frame rate of the video content played in the sub-window as the video frame rate of the video content played in the next window. Alternatively, if the next window window includes a plurality of sub-windows, each of which is used to play a corresponding sub-video content, the electronic device may take the maximum frame rate of all video frame rates of all sub-video contents corresponding to the plurality of sub-windows as the video frame rate for the video. content played in the next window. It can be understood that if the next window contains a plurality of sub-windows, then the captured image of the displayed content of the above-mentioned next window contains the captured image for the sub-video contents played in the plurality of sub-windows. In this case, video content played in sub-windows is called sub-video content.

For example, the following window contains a subwindow subwindow and subwindow and subwindow and the subwindow used to display video content 1, subwindow used to display video content 2, subwindow is used to display 3 video content, and the subwindow used to display video content 4. The video frame rate of video content 2 is greater than the video frame rate of video content 1, than the video frame rate of video content 3, than the video frame rate of video content 4, then the electronic device can take the video frame rate of video content 2 as the video frame rate of the video content played in the next window, and then calculate the image capture time interval.

If necessary, the electronic device determines the display duration of each video image for the video content played in the next window. As an example, an electronic device may divide 1000 ms by the video frame rate to determine the duration of display of each video image. The electronic device may also determine in another manner the duration of display of each video image based on the frame rate of the video. In this case, video images can be understood as video content.

After determining the display duration of each video image, the electronic device determines whether the display duration of each video image exceeds a predetermined ejection time interval (eg, 100 ms). If the display duration of each video image is longer than the predetermined ejection time interval, the electronic device may determine that the image capture time interval is equal to the predetermined ejection time interval. Alternatively, if the display duration of each video image is not longer than the predetermined ejection time interval, the electronic device may determine the image capture time interval based on the display duration of each video image.

As an example, if the display duration of each video image is no longer than a predetermined ejection time interval, then the electronic device may round the display duration of each video image down to the nearest integer to obtain an image capture time interval. As an example, if the display duration of each video image is 35.8 ms, then the electronic device may round the 35.8 ms display duration of each video image down to the nearest integer and the resulting image capture time interval will be 35 ms.

According to an embodiment of the present invention, rounding the display duration of each video image down to the nearest integer can be understood as finding a maximum integer time interval not exceeding the display duration of each video image. For example, if the display duration of each video image is 35.8 ms, then the maximum integer time interval that does not exceed 35.8 ms is 35 ms. Therefore, the display duration of each video image of 35.8 ms is rounded down and the resulting image capture time interval is 35 ms.

As another example, if the display duration of each video image is no longer than a predetermined ejection time interval, then the electronic device may further round down the display duration of each video image to the nearest multiple of 10 ms to obtain an image capture time interval. For example, if the display duration of each video image is 35.8 ms, then the electronic device may round the 35.8 ms display duration of each video image down to the nearest multiple of 10 ms (i.e., 30 ms) to obtain a capture time interval Images.

According to an embodiment of the present invention, rounding the display duration of each video image down to the nearest multiple of 10 ms to obtain an image capture time interval can be understood as finding the maximum integer time interval not exceeding the display duration of each video image, and a multiple of 10 ms, to use as the image capture time interval. For example, if the display duration of each video image is 35.8 ms, then the maximum integer time interval not exceeding 35.8 ms and a multiple of 10 ms is 30 ms. Therefore, the display duration of each video image of 35.8 ms is rounded down to the nearest multiple of 10 ms, and the resulting image capture time interval is 30 ms.

Briefly, according to an embodiment of the present invention, an electronic device determines an image capture time interval based on the display content of the next window, so that the electronic device can regularly capture an image of the display content of the next window according to the image capture time interval, and display the captured image at a display position. corresponding to the next window in the main window. Thus, even if the display level of the main window is higher than the display level of the next window, that is, when the main window is superimposed on top of the next window, the main window still shows a captured image corresponding to the display content of the next window. Thus, the user will think that he/she is still viewing the displayed content of the next window and will not experience visible flickering, which improves the display quality of the next window.

In fig. 7 is a flowchart of a display window overlay processing method according to another embodiment of the present invention. According to the above implementations, for ease of understanding, the display window overlay processing method proposed in the implementation of the present invention will be described in the following implementation of the present invention by considering the application program for the next window as a local service of an electronic device, and the next window is considered as a video display window of a local service , the application program for the main window is considered as a browser of an electronic device, and the main window is considered as a browser display window, all of these are done as examples. As shown in FIG. 7, the method according to an embodiment of the present invention may include the steps of:

In step S701, a browser in the electronic device establishes a connection to a local service.

In step S702, the local electronic device service creates a video display window.

At this time, the display level of the video display window is higher than the display level of the browser display window.

In step S703, the local electronic device service determines the image capture time interval.

More specifically, according to possible implementations of a local service in an electronic device, corresponding contents in the above-described embodiment of the present invention, which will not be repeated here, can be used to determine the image capture time interval.

At this time, the local service in the electronic device captures an image of the display content of the video display window according to the image capture time interval, and pushes the captured image corresponding to the display content of the video display window to the browser while it is observed that the browser display window is in focus. The browser in the electronic device displays the captured image in a display position corresponding to the video display window in the browser display window.

In step S704, the local service of the electronic device monitors whether the video display window of the local service is in focus.

As an example, if it has been monitored that the local service video display window is in focus, step S707 is executed; otherwise, execution of step S704 continues.

In step S705, the local service of the electronic device monitors whether the display level of the browser display window is higher than the display level of the local service display window.

As an example, if it has been detected that the display level of the browser display window is higher than the display level of the local service display window, step S709 is executed; otherwise, execution of step S705 continues.

At step S706, the local electronic device service monitors whether the browser display window is in focus.

As an example, if it has been detected that the browser display window is in focus, step S710 is executed; otherwise, execution of step S706 continues.

According to another example, if it has been monitored that the browser display window is in focus, and the local service monitors that the display level of the video display window of the local service is higher than the display level of the browser display window, step S710 is executed; otherwise, execution of step S706 continues.

In step S707, the local service in the electronic device captures an image of the display content of the video display window according to the image capture time interval, and pushes the captured image corresponding to the display content of the video display window to the browser.

In step S708, the browser in the electronic device displays the captured image at a display position corresponding to the video display window in the browser display window.

At this stage, the browser in the electronic device displays the captured image in the video display window at a position corresponding to the video display window in the browser display window, so that when the display level of the browser display window is higher than the display level of the video display window, the browser display window still shows the captured image. corresponding to the displayed content of the video display window. In this way, the user will think that he/she is still viewing the displayed content of the video display window and will not experience visible flickering.

In step S709, the local electronic device service reconfigures the display level of the video display window above the display level of the browser display window.

At step S710, the local service in the electronic device pushes the captured image corresponding to the displayed content of the video display window to a browser in the electronic device.

According to an embodiment of the present invention, on the one hand, when it has been detected by the local service that the video display window for the local service is in focus, the electronic device captures an image of the display content of the video display window according to a certain image capture time interval, and ejects the captured image, corresponding to the displayed content of the video display window to the browser, so that the browser in the electronic device can display the captured image of the video display window at a display position corresponding to the display window in the browser display window. Therefore, even when the display level of the browser display window is higher than the display level of the video display window, the browser display window will still show the captured image corresponding to the displayed content of the video display window. On the other hand, when the tracking detects that the display level of the browser display window is higher than the display level of the video display window, the electronic device will reset the display level of the video display window to a higher level than that of the browser display window, so that the user can view the window again display local service video. It can be seen that according to an embodiment of the present invention, when the user clicks on the browser while viewing the video display window corresponding to the local service, even if the display level of the browser display window is higher than the display level of the video display window, the browser display window is superimposed on top of the video display window, since the browser display window still shows the captured image corresponding to the displayed content of the video display window corresponding to the local service. In this way, the user will think that he/she continues to view the displayed content of the video display window corresponding to the local service and will not experience visible flickering, which improves the display quality of the video display window in the following method.

According to an embodiment of the present invention, when it has been detected that the browser display window is in focus, the electronic device stops capturing an image of the displayed content of the video display window according to a determined image capturing time interval, and stops pushing the captured image corresponding to the display content of the video display window into the browser , thus saving resource consumption.

In fig. 8 is a block diagram of a display window overlay processing apparatus according to an embodiment of the present invention. As an example, the display window overlay processing apparatus according to an embodiment of the present invention may be an electronic device or may be hardware in an electronic device. As shown in FIG. 8, the display window overlay processing apparatus 80 according to an embodiment of the present invention may include an image capturing unit 801 and a display unit 802.

The image capturing unit 801 is configured to, in the process of overlaying the image of the next window on the main window, capture an image of the displayed content of the next window according to the image capturing time interval.

The display unit 802 is configured to display the captured image at a display position corresponding to the next window in the main window.

According to a possible implementation, the display window overlay processing apparatus 80 may further include:

defining module configured to determine the interval

image capture time based on the displayed content of the next window. According to a possible implementation, the defining module contains: a first defining block configured to determine the interval

image capture time equal to a predetermined time interval

pop if the next window's displayed content includes non-video content; or,

a second determining unit configured to determine an image capture time interval based on the frame rate of the video content if the displayed content of the next window includes video content.

According to a possible implementation, the second defining block may be specifically configured to:

determining the display duration of each video image based on the frame rate of the video in the video content;

determining whether the display duration of each video image is longer than a predetermined popping time interval;

if the display duration of each video image is longer than the predetermined popping time interval, then the image capture time interval is determined to be equal to the predetermined popping time interval.

if the display duration of each video image is not longer than the predetermined popping time interval, then the display duration of each video image is rounded down to the nearest integer to obtain an image capture time interval; or,

If the display duration of each video image is no longer than the predetermined pop time interval, then the display duration of each video image is rounded down to the nearest multiple of 10 ms to obtain the image capture time interval.

According to a possible implementation, if the next window contains a plurality of sub-windows, each of which is used to play the corresponding sub-video content, the video frame rate of the video content is determined to be equal to the maximum video frame rate among the video frame rates of all sub-video contents corresponding to the plurality of sub-windows.

According to a possible implementation, the image capture module 801 may be specifically configured to:

if it has been detected that the next window is in focus, then an image of the displayed content of the next window is captured according to the image capture time interval;

if the next window has focus, then the display level of the next window is higher than the display level of the main window.

According to a possible implementation, the image capture module 801 may be specifically configured to:

if it has been observed that the number of subwindows contained in the next window is greater than a predetermined value, then an image of the displayed content of the next window is captured according to the increased image capture time interval.

According to a possible implementation, the image capturing module 801 may further be configured to:

If it has been detected that the main window is in focus, then image capture of the displayed content of the next window is stopped according to the image capture time interval.

According to a possible implementation, the image capturing module 801 may further be configured to:

if it is detected that the display level of the next window is greater than the display level of the main window after it has been detected that the main window is in focus, image capturing of the display content of the next window is stopped according to the image capturing time interval.

According to a possible implementation, the main window is a browser display window in an electronic device; and/or,

The following window is the electronic device local service video display window.

The display window overlay processing apparatus according to an embodiment of the present invention can be used to implement a technical solution when implementing the display window overlay processing method according to the present invention, and the principles and technical consequences of these implementations are similar and will not be repeated here.

In fig. 9 is a block diagram of a display window overlay processing apparatus according to another embodiment of the present invention. As an example, the display window overlay processing apparatus according to an embodiment of the present invention may be an electronic device or may be hardware in an electronic device. As shown in FIG. 9, the display window overlay processing apparatus 90 according to an embodiment of the present invention may include an application program 901 for the next window and an application program 902 for the main window.

The next window application 901 is configured, in the process of overlaying the next window image on the main window, to capture an image of the display content of the next window according to the image capturing time interval, and push the captured image to the main window application.

The main window application 902 is configured to display the captured image at a display position corresponding to the next window in the main window.

According to a possible implementation, the next window application 901 may be further configured to, before capturing an image of the display content of the next window according to the image capture time interval, determine the image capture time interval based on the display content of the next window.

According to a possible implementation, the next window application 901 may be specifically configured to:

if the displayed content of the next window includes non-video content, the image capture time interval is determined to be equal to the predetermined pop time interval; or,

If the displayed content of the next window includes video content, the image capture time interval is determined based on the frame rate of the video content.

According to a possible implementation, the next window application 901 may be specifically configured to:

determining the display duration of each video image based on the frame rate of the video in the video content;

determining whether the display duration of each video image is longer than a predetermined popping time interval;

if the display duration of each video image is longer than the predetermined popping time interval, then the image capture time interval is determined to be equal to the predetermined popping time interval.

if the display duration of each video image is not longer than the predetermined popping time interval, then the display duration of each video image is rounded down to the nearest integer to obtain an image capture time interval; or,

If the display duration of each video image is no longer than the predetermined pop time interval, then the display duration of each video image is rounded down to the nearest multiple of 10 ms to obtain the image capture time interval.

According to a possible implementation, if the next window contains a plurality of sub-windows, each of which is used to play the corresponding sub-video content, the video frame rate of the video content is determined to be equal to the maximum video frame rate among the video frame rates of all sub-video contents corresponding to the plurality of sub-windows.

According to a possible implementation, the next window application 901 may be specifically configured to:

if it has been detected that the next window is in focus, then an image of the displayed content of the next window is captured according to the image capture time interval;

if the next window has focus, then the display level of the next window is higher than the display level of the main window.

According to a possible implementation, the next window application 901 may be specifically configured to:

if it has been observed that the number of subwindows contained in the next window is greater than a predetermined value, then an image of the displayed content of the next window is captured according to the increased image capture time interval.

According to a possible implementation, the next window application 901 may be further configured to:

if it has been observed that the main window is in focus, then stopping the image capture of the displayed content of the next window according to the image capturing time interval, and stopping the pushing of the captured image to the application program 902 for the main window.

According to a possible implementation, the application program for the next window may be further configured to:

if it has been detected that the display level of the next window is greater than the display level of the main window after it has been detected that the main window is in focus, then the image capture of the display content of the next window is stopped according to the image capture time interval, and the ejection of the captured image is stopped into the application program for the main window.

According to a possible implementation, the main window application 902 is a browser on an electronic device; and, accordingly, the main window is the browser display window; and/or,

the application program 901 for the next window is a local service of the electronic device, and the next window is a video display window of the local service.

The display window overlay processing apparatus according to an embodiment of the present invention can be used to implement a technical solution when implementing the display window overlay processing method according to the present invention, and the principles and technical consequences of these implementations are similar and will not be repeated here.

In fig. 10 is a simplified diagram of an electronic device according to an embodiment of the present invention. As shown in FIG. 10, the electronic device 100 according to an embodiment of the present invention may include, among other things, a processor 1001 and a memory 1002.

The memory 1002 is configured to store program instructions.

The processor 1001 is configured to recall and execute program instructions stored in the memory 1002, and the electronic device is configured to execute a technical solution according to an embodiment of the display window overlay processing method of the present invention when the processor executes the program instructions stored in the memory 1002, and the principles and technical consequences These executions are similar and will not be repeated here.

According to an embodiment of the present invention, there is provided a computer-readable storage medium on which instructions are stored, the instructions, when executed by a computer, causing the computer to execute a technical solution according to an embodiment of the display window overlay processing method of the present invention, and the principles and technical consequences of these implementations are similar and not will be repeated here.

According to an embodiment of the present invention, a computer program is provided which, when executed on a computer, causes the computer to execute a technical solution according to an embodiment of the display window overlay processing method of the present invention, and the principles and technical consequences of these implementations are similar and will not be repeated here.

Those skilled in the art also understand that according to various embodiments of the present invention, the sequence number values of the above-mentioned processes do not affect the execution sequence, and the execution sequence of each process is determined by its functions and internal logic, and the present invention does not impose any restrictions on the execution process .

Those skilled in the art will clearly understand that for convenience and brevity of description, only the above-mentioned portion of the functional modules is given as an example. In practical application, the above-mentioned function modules can be supplemented with various function modules according to needs, so that the internal structure of the hardware can be divided into various function modules to perform all or some of the above-mentioned functions. For a specific process of operation of the above-mentioned apparatus, references to the corresponding process in the above-mentioned method embodiment can be used, and this will not be repeated here.

Those skilled in the art will appreciate that all or some of the steps for implementing the above method embodiments may be performed by hardware in conjunction with program instructions. The above-mentioned programs may be stored in a computer-readable storage medium. During execution of the program, the steps provided in the above-mentioned method embodiments are performed; the above-mentioned computer-readable storage medium comprises a read-only memory (ROM), random access memory (RAM), a magnetic or optical disk drive, or any other medium on which program codes can be stored.

In conclusion, it should be noted that the above-mentioned embodiments are used only to illustrate the technical solution of the present invention and not to limit it. Although the invention has been described in detail with reference to the above-mentioned embodiments, those skilled in the art will understand that they may still modify the technical solution described in the above-mentioned embodiments, or equivalently replace some or all of the technical features. Such changes or substitutions do not lead to a deviation from the scope of the corresponding technical solution from the scope of the technical solution of the embodiments of the present invention.

Claims (33)

1. A display window overlay processing method applied to an electronic device on which a next window application program and a main window application program are installed; wherein after the next window is created, the display level of the next window is greater than the display level of the main window before the main window is in focus; In the case where the display level of the next window is higher than the display level of the main window, if it is detected that the main window is in focus, the electronic device first sets the display level of the main window higher than the next window, and then the electronic device resets the display level of the next window so that that it becomes above the display level of the main window, and the method involves the stages: in the process of overlaying the image of the next window on the main window, an image of the displayed content of the next window is captured according to the image capture time interval; displaying the captured image at a display position corresponding to the next window in the main window, so that when the display level of the main window is higher than the display level of the next window, the main window still displays the captured image corresponding to the display content of the next window, before capturing the display content image next window according to the image capture time interval, the method further includes the step of: determining the image capture time interval based on the displayed content of the next window and determining the image capture time interval based on the displayed content of the next window includes the steps of: if the displayed content of the next window includes non-video content, the image capture time interval is determined to be equal to the predetermined pop time interval; or, if the displayed content of the next window contains video content, the image capture time interval is determined based on the frame rate of the video content, wherein an indication that the display level of the main window is higher than the display level of the next window means that the main window is superimposed on the next window, and an indication that the display level of the next window is higher than the display level of the main window means that the next window overlays the main window, and indicating that the main window has focus means that the main window is selected by the user. 2. The method according to claim 1, wherein determining the image capture time interval based on the video frame rate in the video content involves the steps of: determining the display duration of each video image based on the frame rate of the video in the video content; determining whether the display duration of each video image is longer than a predetermined popping time interval; if the display duration of each video image is longer than the predetermined popping time interval, then the image capture time interval is determined to be equal to the predetermined popping time interval; if the display duration of each video image is not longer than the predetermined popping time interval, then the display duration of each video image is rounded down to the nearest integer to obtain an image capture time interval; or, If the display duration of each video image is no longer than the predetermined pop time interval, then the display duration of each video image is rounded down to the nearest multiple of 10 ms to obtain the image capture time interval. 3. The method according to claim 1 or 2, and if the next window contains a plurality of sub-windows, each of which is used to play the corresponding sub-video content, the video frame rate in the video content is determined equal to the maximum video frame rate among the video frame rates of all contents subvideos corresponding to many subwindows. 4. The method according to claim 1 or 2, wherein before capturing an image of the displayed content of the next window according to the image capture time interval, the method comprises the steps of: if it has been detected that the next window is in focus, then an image of the displayed content of the next window is captured according to the image capture time interval; if the next window has focus, then the display level of the next window is higher than the display level of the main window; or, wherein capturing an image of the displayed content of the next window according to the image capturing time interval comprises the step of: if it has been observed that the number of sub-windows contained in the next window is greater than a predetermined value, then an image of the displayed content of the next window is captured according to the increased image capture time interval; or, wherein the method further comprises the step of: if it has been detected that the main window is in focus, then image capture of the displayed content of the next window is stopped according to the image capture time interval; or, wherein the method further comprises the step of: if it has been detected that the display level of the next window is greater than the display level of the main window after it has been detected that the main window is in focus, image capturing of the display content of the next window is stopped according to the image capturing time interval; or, wherein the main window is a browser display window on the electronic device; and/or The following window is the electronic device local service video display window. 5. A display window overlay processing apparatus, configured to apply the method of claim 1 to an electronic device on which an application program for the next window and an application program for the main window are installed; wherein after the next window is created, the display level of the next window is greater than the display level of the main window before the main window is in focus; In the case where the display level of the next window is higher than the display level of the main window, if it is detected that the main window is in focus, the electronic device first sets the display level of the main window higher than the next window, and then the electronic device resets the display level of the next window so that that it becomes above the display level of the main window, and the equipment contains: an image capturing module configured to, in the process of overlaying the image of the next window on the main window, capture an image of the displayed content of the next window according to the image capturing time interval; a display module configured to display a captured image at a display position corresponding to the next window in the main window, such that when the display level of the main window is higher than the display level of the next window, the main window is still configured to display the captured image corresponding to the displayed content next window. 6. An electronic device for processing display window overlay, containing a processor and memory; wherein the memory is configured to store program instructions; the processor is configured to call and execute program instructions stored in memory, and the electronic device is configured to execute the method according to any one of claims. 1-4 when the processor executes program instructions stored in memory. 7. A computer-readable storage medium on which instructions are stored, wherein the instructions, when executed by a computer, cause the computer to carry out the method of any one of claims. 1-4.

Images