WO2023083052A1 - Interaction method and apparatus - Google Patents

Abstract

The present application relates to an interaction method and apparatus. The method comprises: a first device acquiring a first image and position information of a window area in the first image, wherein the first image represents a picture that the first device currently displays, and the window area represents an area in the first image where the first device performs shared display with a second device; the first device encoding the first image on the basis of the position information of the window area, so as to obtain a second image, such that the transparency of areas, other than the window area, in the second image is greater than or equal to a preset value; the first device sending the second image to the second device; and the second device receiving and displaying the second image. By means of the interaction method and apparatus provided in the embodiments of the present application, the smoothness of running of an electronic device during screen content sharing can be improved, thereby improving the user experience.

Description

An interactive method and device

This application claims the priority of a Chinese patent application with application number 202111349438.7 and application title "An Interaction Method and Device" filed with the China Patent Office on November 15, 2021, the entire contents of which are incorporated herein by reference.

technical field

The present application relates to the field of multimedia technologies, in particular to an interaction method and device.

Background technique

When a user browses wonderful content on the display screen of an electronic device, he may wish to share the content to other electronic devices for display, which is screen content sharing. The systems run by the electronic devices for screen content sharing may be the same or different. For example, screen content can be shared between a tablet running Android and a PC running Windows. When the screen content of the tablet computer is shared and displayed on the personal computer, the operation (for example, creating a window, moving the window, or closing the window, etc.) performed by the user on the tablet computer through an input device such as a touch screen or a stylus can be performed on the computer. At the same time, the user can operate the application on the tablet computer like other applications in the personal computer through input devices such as a keyboard and a mouse.

FIG. 1 shows an architecture diagram of an interactive system in the related art. As shown in FIG. 1 , the interactive system includes a personal computer and a tablet computer, and the interactive system can be used to share screen content between the personal computer and the tablet computer. In the scenario where the tablet is used as a virtual extended screen of the personal computer, and the personal computer shares and displays the local window on the tablet, a data transmission channel is established between the personal computer and the tablet. The personal computer detects the local window by calling the application window detection interface. When a change in the local window is detected, the personal computer captures the screen, encodes the captured image and sends it to the tablet computer, and at the same time transmits each window that needs to be shared through a separate channel. After the tablet computer receives the image, it decodes and displays it, and at the same time creates a separate floating window for each window. When the user operates the floating window on the tablet computer, the tablet computer can transmit the corresponding events (such as moving, closing, zooming, etc.) and coordinate position information of the user operation to the personal computer, thereby reversely controlling the windows displayed on the personal computer .

In the related art, each window requires a separate coding transmission channel, so that the cumulative number of coding channels required by multiple windows increases continuously. In addition, the tablet computer needs to create a separate floating window for each shared window, and the number of floating windows will continue to accumulate. In this way, the functions and delay performance of personal computers and tablet computers are affected, so that the usage scenarios are limited to the situation where there are few shared windows. How to improve the smoothness of device operation when sharing screen content has become an urgent problem to be solved.

Contents of the invention

In view of this, an interaction method and device are proposed, which can improve the running fluency of electronic equipment when screen content is shared.

In a first aspect, an embodiment of the present application provides an interaction method, the method comprising: acquiring a first image and position information of a window area in the first image, wherein the first image represents a first device In the currently displayed picture, the window area represents an area shared by the first device and the second device in the first image; encoding the first image based on the position information of the window area to obtain a second image Two images, so that the transparency of the area other than the window area in the second image is greater than or equal to a preset value; sending the second image.

In this embodiment of the present application, the first image representing the screen currently displayed by the first device is encoded to obtain the second image, so that the area in the second image other than the area that needs to be shared by the first device and the second device The transparency is greater than or equal to the preset value, so that the window area sharing can be realized by transmitting the second image between the first device and the second device and displaying the second image on the second device, and it is not necessary to create an image for each window in the window area Establishing a separate encoding channel eliminates the need to create a floating window for each window in the window area, which reduces the delay and resource occupation, thereby improving the smoothness of electronic device operation during screen content sharing and improving user experience. For a scene with a large number of windows in the window area, it can also effectively improve the smoothness of the operation of the electronic device, and expand the application scene of screen content sharing.

According to the first aspect, in a first possible implementation manner of the method, the method further includes: sending position information of the window area.

In the embodiment of this application, the first device sends the location information of the window area to the second device, which can facilitate the second device to determine the reverse control area and the self-control area, so as to determine whether the user operation is for the reverse control of the first device or Local control for a second device improves user experience.

According to the first aspect or the first possible implementation of the first aspect, in the second possible implementation of the method, the acquiring the first image and the position information of the window area in the first image The method includes: acquiring the first image and position information of the window area in response to a window control instruction, where the window control instruction is used to control the window area displayed on the first device.

In the embodiment of the present application, when the window area is controlled and changes, the first image is reacquired, and then a new second image is obtained, which can realize updating of the second image on the second device.

According to the second possible implementation of the first aspect, in the third possible implementation of the method, the method further includes: acquiring the first coordinate information and the first event type in response to the first user operation , the first user operation represents a user operation performed on the window area displayed on the first device; based on the first coordinate information and the first event type, the window control instruction is generated; according to the window The control instruction controls the window area displayed on the first device.

In the embodiment of the present application, the user can operate the shared window area on the first device, and in response to the user's operation, the window area changes and the second image is updated accordingly.

According to the second possible implementation of the first aspect, in the fourth possible implementation of the method, the method further includes: in response to the received counter-control message, obtaining from the counter-control message second coordinate information and a second event type, the counter-control message is generated according to a second user operation, and the second user operation represents a user operation performed on the window area displayed on the second device; performing coordinate transformation on the second coordinate information; generating the window control instruction based on the transformed second coordinate information and the second event type; and performing the window display on the first device according to the window control instruction area is controlled.

In the embodiment of the present application, the user may operate the shared window area on the second device, and in response to the user's operation, the window area changes and the second image is updated accordingly.

In a possible implementation manner, the acquiring the first image and the position information of the window area in the first image includes: acquiring the first image and the window area in response to a screen content sharing operation location information.

In a possible implementation manner, the window area includes one or more windows.

In the embodiment of the present application, by transmitting the second image, one or more windows can be shared, which expands the application scenarios.

In the second aspect, the embodiment of the present application provides an interaction method, the method includes: receiving a second image, the transparency of the area other than the window area in the second image is greater than or equal to a preset value, the window The area represents an area shared by the second device in the screen currently displayed by the first device; the second image is displayed on the second device.

According to the second aspect, in the first possible implementation manner of the method, the method further includes: receiving the location information of the window area; an anti-control area; in response to a second user operation on the anti-control area, acquiring second coordinate information and a second event type; based on the second coordinate information and the second event type, generating an anti-control message; returning the anti-control message to control the first device.

According to the first possible implementation of the second aspect, in the second possible implementation of the method, the method further includes: acquiring third coordinate information and The third event type, the self-control area represents the area other than the anti-control area in the screen currently displayed by the second device; based on the third coordinate information and the third event type, for the second The device is controlled.

According to the second aspect, or any possible implementation manner of the above second aspect, in a third possible implementation manner of the method, the window area includes one or more windows.

In a third aspect, an embodiment of the present application provides an interaction device, the device comprising:

The first acquisition module is configured to acquire a first image and position information of a window area in the first image, wherein the first image represents the screen currently displayed by the first device, and the window area represents the first In the image, the area where the first device and the second device perform shared display;

An encoding module, configured to encode the first image acquired by the first acquisition module based on the position information of the window area acquired by the first acquisition module, to obtain a second image, so that the second image excludes the The transparency of the area outside the window area is greater than or equal to the preset value;

The first sending module is configured to send the second image obtained by the coding module.

In a possible implementation manner, the device further includes:

The second sending module is configured to send the location information of the window area.

In a possible implementation manner, the first obtaining module is also used for:

Acquire the first image and position information of the window area in response to a window control instruction, where the window control instruction is used to control the window area displayed on the first device.

In a possible implementation manner, the device further includes:

A second acquiring module, configured to acquire first coordinate information and a first event type in response to a first user operation, where the first user operation represents a user operation performed on the window area displayed on the first device;

A first generating module, configured to generate the window control instruction based on the first coordinate information and the first event type;

The first control module is configured to control the window area displayed on the first device according to the window control instruction.

In a possible implementation manner, the device further includes:

A third acquiring module, configured to acquire second coordinate information and a second event type from the counter-control message in response to the received counter-control message, the counter-control message is generated according to a second user operation, the The second user operation represents a user operation performed on the window area displayed on the second device;

a transformation module, configured to perform coordinate transformation on the second coordinate information;

A second generating module, configured to generate the window control instruction based on the transformed second coordinate information and the second event type;

The second control module is configured to control the window area displayed on the first device according to the window control instruction.

In a possible implementation manner, the first obtaining module is also used for:

In response to a screen content sharing operation, acquire the first image and position information of the window area.

In a possible implementation manner, the window area includes one or more windows.

In a fourth aspect, an embodiment of the present application provides an interaction device, the device comprising:

The first receiving module is used to receive the second image, the transparency of the area other than the window area in the second image is greater than or equal to a preset value, and the window area indicates that the screen currently displayed by the first device is compatible with the second device shared area;

A display module, configured to display the second image received by the first receiving module on the second device.

In a possible implementation manner, the device further includes:

A second receiving module, configured to receive position information of the window area;

A determination module, configured to determine an anti-control area in the second device according to the location information of the window area;

A first acquiring module, configured to acquire second coordinate information and a second event type in response to a second user operation on the anti-control area;

A generating module, configured to generate a counter-control message based on the second coordinate information and the second event type;

A return module, configured to return the counter-control message to control the first device.

In a possible implementation manner, the device further includes:

The second acquisition module is configured to acquire third coordinate information and a third event type in response to a third user operation on the self-control area, where the self-control area indicates that the anti-control area is excluded from the screen currently displayed by the second device areas other than

A control module, configured to control the second device based on the third coordinate information and the third event type.

In a possible implementation manner, the window area includes one or more windows.

In the fifth aspect, the embodiments of the present application provide an electronic device, which can execute one or more of the interaction methods of the above-mentioned first aspect or multiple possible implementations of the first aspect, or, execute The above-mentioned second aspect or one or more interaction methods among the multiple possible implementation manners of the second aspect.

In a sixth aspect, an embodiment of the present application provides an electronic device, the electronic device includes a display screen for displaying images; a processor; a memory for storing instructions executable by the processor; wherein the processor is configured To implement the above-mentioned first aspect or one or more of the interaction methods in the multiple possible implementations of the first aspect when executing the instructions, or to implement the above-mentioned second aspect or multiple possible implementations of the second aspect One of several interaction methods.

In the seventh aspect, the embodiments of the present application provide a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above-mentioned first aspect or multiple possible implementations of the first aspect can be realized one or more of the interaction methods in the above-mentioned second aspect or one or more of the multiple possible implementation manners of the second aspect.

In an eighth aspect, the embodiments of the present application provide a computer program product, including computer readable code, or a non-volatile computer readable storage medium bearing computer readable code, when the computer readable code is stored in an electronic When the device is running, the processor in the electronic device executes one or more of the interaction methods of the above-mentioned first aspect or multiple possible implementations of the first aspect, or executes the above-mentioned second aspect or the second One or more of the various possible implementations of an aspect.

These and other aspects of the present application will be made more apparent in the following description of the embodiment(s).

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate exemplary embodiments, features, and aspects of the application and, together with the specification, serve to explain the principles of the application.

FIG. 1 shows an architecture diagram of an interactive system in the related art;

FIG. 2 shows a schematic diagram of an application scenario of an interaction method provided by an embodiment of the present application;

FIG. 3 shows a schematic diagram of the architecture of the interactive system provided by the embodiment of the present application;

FIG. 4 shows an interaction flowchart of an interaction method provided by an embodiment of the present application;

Fig. 5a shows an exemplary schematic diagram of the first image of the embodiment of the present application;

Figure 5b, Figure 5c and Figure 5d respectively show an exemplary schematic view of the window area in the embodiment of the present application;

FIG. 6 shows an interaction flowchart of an interaction method provided by an embodiment of the present application;

FIG. 7 shows a schematic structural diagram of an interaction device provided by an embodiment of the present application;

FIG. 8 shows a schematic structural diagram of an interaction device provided by an embodiment of the present application;

FIG. 9 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

In addition, in order to better illustrate the present application, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present application may be practiced without certain of the specific details. In some instances, methods, means, components and circuits well known to those skilled in the art have not been described in detail in order to highlight the gist of the present application.

FIG. 2 shows a schematic diagram of an application scenario of the interaction method provided by the embodiment of the present application. As shown in Figure 2, personal computer 11, panel computer 12, mobile phone 13 and digital television 14, and outdoor large screen (not shown), projector (not shown) and other electronic devices with display screens can be screened. content sharing. The electronic device of the present application may refer to a device with a wireless connection function, and the wireless connection function means that it can be connected with other electronic devices through wireless connection methods such as Wireless Fidelity (Wireless Fidelity, WiFi), Bluetooth, etc. The electronic device of the present application It may also have the function of communicating by wired connection. The operating system running on the electronic device of the present application includes but not limited to Android system and Windows system. The electronic device of the present application can be a touch screen or a non-touch screen; the touch screen can control the electronic device by clicking and sliding on the display screen with a finger, a stylus, etc.; the non-touch screen device can be Connect the mouse, keyboard, touch panel and other input devices, and control the electronic devices through the input devices. The embodiment of the present application does not limit the electronic devices for screen content sharing, nor does it limit the operating systems running on the electronic devices, nor does it limit the communication connections established between electronic devices and the types of display screens of the electronic devices.

In the embodiment of the present application, electronic devices may share screen content through interaction. The screen displayed on the display screen of one electronic device can be shared and displayed on one or more other electronic devices. For example, as shown in FIG. 2 , the screen displayed on the tablet computer 12 can be shared and displayed on the mobile phone 13 ; the screen displayed on the tablet computer 12 can be shared and displayed on the personal computer 11 and the mobile phone 13 at the same time. In this embodiment of the present application, when an electronic device shares screen content with other electronic devices, it can share all the screens displayed on the current display screen of the electronic device, or share part of the screens displayed on the current screen of the electronic device (for example, one or more windows). Wherein, the window may be an instant chat application window, a web page window, a video playback window, an audio playback window, an image display window, or a text display window. The positions of the windows can overlap. In the embodiment of the present application, there is no limit to the number of electronic devices for sharing screen content, and no limit to the content to be shared.

FIG. 3 shows a schematic diagram of an architecture of an interactive system provided by an embodiment of the present application. As shown in FIG. 3 , the interactive system includes a first device 31 and a second device 32 . The first device 31 and the second device 32 include, but are not limited to, the electronic devices shown in FIG. 2 . In this embodiment of the present application, the first device 301 may include a first negotiation module 311 , a driver module 312 , a first display module 313 , an image processing module 314 and a first transmission module 315 . The second device 32 may include a second negotiation module 321 , a second display module 322 and a second transmission module 323 .

As shown in FIG. 3 , the first device 31 and the second device 32 perform service negotiation through a first negotiation module 311 and a second negotiation module 321 . After the negotiation is completed, the initialization of screen content sharing is completed between the first device 31 and the second device 32, and screen content sharing can be performed.

The user performs a screen content sharing operation on the first device 31 (for example, may click a screen content sharing button, or issue a voice instruction for screen content sharing, etc.). In response to the screen content sharing operation, the driver module 312 of the first device 31 may enable the virtual display driver, so that the first display module 313 enters the extended screen mode. At this time, the image processing module 314 of the first device 31 may acquire the screen currently displayed by the first display module 313 as the first image, and acquire the position information of the window area used for sharing in the first image. The image processing module 314 may encode the first image based on the position information of the window area to obtain the second image, so that the transparency of the area other than the window area in the second image is greater than or equal to a preset value. Afterwards, the first device 31 may send the second image to the second transmission module 323 of the second device 32 through the first transmission module 315 . After the second transmission module 323 receives the second image, the second display module 322 can display the second image.

In the embodiment of the present application, since the transparency of the area other than the window area in the second image is greater than or equal to the preset value, when the second device 32 displays the second image, it can achieve The effect of displaying the window area shared by the first device 31 on the previously displayed screen) realizes the sharing of screen content between the first device 31 and the second device 32 . In the above-mentioned screen content sharing process, only the second image needs to be transmitted between the first device 31 and the second device 32, and there is no need to establish a separate encoding channel for each window in the window area; at the same time, the second device 32 only needs to display Second image, no need to create a floating window for every window in the window area. Therefore, in the embodiment of the present application, the smoothness of the operation of the electronic device during screen content sharing is improved, and the user experience is improved.

In a possible implementation manner, the first device 31 shown in FIG. 3 may further include a window management module 316 . The user can perform operations on the window area on the first device 31 or the second device 32 (for example, can close the window in the window area, move the window in the window area, change the size of the window in the window area, etc.). In response to a user operation on the window area, the window management module 316 of the first device 31 may detect that the window area has changed, and at this time the image processing module 314 of the first device 31 may reacquire the picture currently displayed by the first display module 313 As the first image, and obtaining the position information of the window area used for sharing in the first image, and then obtaining a new second image, and displaying the new second image in the second display module 322 of the second device 32, The updating of the shared content displayed in the second device is realized.

FIG. 4 shows an interaction flowchart of an interaction method provided by an embodiment of the present application. This method can be applied to the interactive system shown in FIG. 3 . As shown in Figure 4, the method may include:

In step S401, the first device acquires a first image and position information of a window area in the first image.

Wherein, the first image represents the screen currently displayed by the first device, and the window area represents the shared display area of the first device and the second device in the first image. In a possible implementation manner, the window area may include one or more windows. Wherein, the window may be a window of an instant chat communication application, a browser window, a document window, a picture window, or a player window. A window can be displayed full screen or not. When the window area includes multiple windows, the windows can overlap, that is, any window can be covered by other windows partially or completely, and can also cover some or all of the other windows; Overlapping means that any window is neither covered by nor covered by other windows. The user may select a window to be shared as required, and the first device may determine the position information of the window area according to the position information of the window selected by the user and the stacking relationship of the windows selected by the user.

Fig. 5a shows an exemplary schematic diagram of the first image of the embodiment of the present application. As shown in FIG. 5 a , three windows in total of window 1 , window 2 and window 3 are displayed in the first image. Fig. 5b, Fig. 5c and Fig. 5d respectively show an exemplary schematic view of the window area in the embodiment of the present application. In one example, the user selects window 1 for shared display, and at this time, the window area in step S401 is the area where window 1 is located (as shown in FIG. 5b ). In yet another example, the user selects window 1 and window 2 for shared display. At this time, the window area in step S401 is the union of the area where window 1 is located and the area where window 2 is located (as shown in FIG. 5c ). In another example, the user selects window 1 and window 3 for shared display. At this time, the window area in step S401 is the union of the area where window 1 is located and the area where window 3 is located (as shown in FIG. 5d ). It can be understood that, the above is only an exemplary description of the window area, and is not used to limit the window area.

In an example, the position information of the window area may be the coordinates of each vertex of the window area. Taking Fig. 5b as an example, the position information of the window area may be the coordinates of four vertices. Taking FIG. 5c as an example, the position information of the window area may be the coordinates of eight points. Taking FIG. 5d as an example, the position information of the window area may include two sets of coordinates, and each set of coordinates includes coordinates of four vertices. In yet another example, the position information of the window area may be coordinates of vertices and lengths of sides of the window area. Taking FIG. 5b as an example, the position information of the window area may be the coordinates, width and height of the lower left corner vertex. The above is only an example of the location information of the window area, and is not intended to limit the location information of the window area.

In a possible implementation manner, step S401 may include: the first device acquires the first image and position information of the window area in response to a window control instruction.

Wherein, the window control instruction may be used to control the window area displayed on the first device. In one example, the window control instruction can be used to implement one or more of the following control functions: move the window in the window area, close the window in the window area, remove the window in the window area, move the window to the window area Add an existing window, and create a window and add the created window to the window area. For example, the window area is shown in FIG. 5d, and the window control instruction is used to remove the left window from the window area (that is, the left window is no longer shared and displayed). In response to the window control instruction, the first device can obtain the first image, and the location information of the window area shown in Figure 5b.

In an example, the window control instruction may be generated based on the first user operation. Wherein, the first user operation refers to a user operation performed on the window area displayed on the first device. That is to say, the first user operation is an operation performed by the user on the first device through an input device such as a mouse, a keyboard, a touch screen, or a stylus, and the first user operation is an operation performed on the window area. The coordinate information corresponding to the first user operation may be referred to as first coordinate information. For example, the first coordinate information may be coordinate information of a position clicked by a mouse on the first device, coordinate information of a touched position on a touch screen of the first device, and the like. The event type corresponding to the first user operation may be referred to as a first event type. For example, the first event type may include, but not limited to: one or more of creating a window, moving a window, closing a window, removing a shared window, adding a shared window, and the like.

In this embodiment of the present application, the first device acquires the first coordinate information and the first event type in response to the first user operation; generates the window control instruction based on the first coordinate information and the first event type; according to The window control instruction controls the window area displayed on the first device. In a specific implementation, the first device may control the window area displayed on the first device according to the first coordinate information and the first event type corresponding to the window control instruction. For example, the position indicated by the first coordinate information is on the close button of window 1 displayed by the first device, and the first event type is to close the window, and the first device may, according to the first coordinate information and the first event type, Window 1 in the window area displayed on a device is closed.

In yet another example, the window control instruction may be generated based on the second user operation. Wherein, the second user operation refers to a user operation performed on the window area displayed on the second device. That is to say, the second user operation is an operation performed by the user on the second device through an input device such as a mouse, a keyboard, a touch screen, or a stylus, and the second user operation is an operation performed on the window area. The coordinate information corresponding to the second user operation may be referred to as second coordinate information. For example, the second coordinate information may be the coordinate information of the position clicked by the mouse on the second device, the coordinate information of the touched position on the touch screen of the second device, and the like. The event type corresponding to the second user operation may be referred to as a second event type. For example, the second event type may include, but is not limited to, one or more of moving a window, closing a window, moving out of a shared window, and the like. The second device may generate a counter-control message based on the second coordinate information and the second event type, and send the counter-control message to the first device.

In this embodiment of the present application, the first device obtains the second coordinate information and the second event type from the counter-control message in response to the received counter-control message, and the counter-control message is generated according to the second user operation ; performing coordinate transformation on the second coordinate information; generating the window control instruction based on the transformed second coordinate information and the second event type; according to the window control instruction, displaying on the first device window area to control. In a specific implementation, the first device may control the window area displayed on the first device according to the converted second coordinate information and the second event type corresponding to the window control instruction. For example, the position indicated by the second coordinate information is on the close button of window 1 displayed by the second device, and the second event type is closing the window, and the first device according to the converted second coordinate information and the second event type, Window 1 in the window area displayed on the first device may be closed. It can be understood that the size and/or resolution of the display screen of the first device and the display screen of the second device may be different, therefore, the first device needs to transform the second coordinate information before performing control. In the embodiment of the present application, no limitation is imposed on the transformation method of the second coordinate information.

Afterwards, the first device may acquire the first image and the position information of the window area in response to the window control instruction generated based on the first user operation or the window control instruction generated based on the second user operation, and then may perform the steps From S402 to step S405, update the window area shared and displayed on the second device.

In a possible implementation manner, step S401 may include: the first device acquires the first image and position information of the window area in response to a screen content sharing operation.

Wherein, the screen content sharing operation may be used to share and display one or more windows displayed on the first device. After the user selects the window to be shared, the user can trigger (for example, single-click, double-click, etc.) The selected window is dragged out of the display screen, etc. to perform screen content sharing operations. The first device detects the screen content sharing operation, which indicates that the user wants to perform window sharing display. Therefore, the first device can obtain the first image and the position information of the window area in response to the screen content sharing operation, and then execute steps S402 to Step S405, sharing the display window area on the second device.

Step S402, the first device encodes the first image based on the position information of the window area to obtain the second image, so that the transparency of the area in the second image except the window area is greater than or equal to a preset value.

Wherein, the preset value can be set as required. Taking 0 as completely opaque and 100% as completely transparent as an example, the preset value can be set to 90%, 95% or 100%. It can be understood that the larger the preset value, the more blurred the non-window area (ie, the area other than the window area) in the second image (ie, the less likely to be seen by the user), and the area covered by the non-window area in the second image The clearer the picture (that is, the easier it is for the user to see). When the default value is set to 100%, the area in the second image other than the window area is not displayed at all, and the user will not see the non-window area in the second image, which is covered by the non-window area in the second image The screen can be displayed normally (that is, for a screen, when the screen is covered by the non-window area in the second image and when the screen is not covered by the non-window area in the second image, the user sees the same effect). When the default value is set to 0, the area other than the window area in the second window is normally displayed, and the user can clearly see the area other than the window area in the second image, and the user will not see the area that is included in the second image. A screen covered by an area other than the window area.

When the transparency of the area other than the window area in the second image is greater than or equal to the preset value, it means that the user cannot see or clearly see the area other than the window area in the second image. Correspondingly, the user can see more clearly to see the frame covered by the area in the second image other than the window area. In this way, when the first device sends the second image to the second device for display, the shared display of the window area on the first device and the second device can be realized, and the non-window area in the second image can be avoided from being displayed on the second device. The leakage of information caused by clear display improves the privacy security of the first device and the flexibility of sharing content. At the same time, when the second device displays the second image, the user can clearly see the screen covered by the non-window area in the second image on the second device, which reduces the impact of screen content sharing on the user's use of the second device and improves improve the user experience of using the second device. In addition, the shared display of the window area can be realized by transmitting the second image between the first device and the second device, without creating a separate encoding channel for each window, and without the need for the second device to create a separate floating window for each window, reducing the It reduces the sharing delay, saves resources, improves the smooth operation of electronic devices when sharing screen content, and improves the user experience. Especially for the case where the window area includes many windows, the improvement effect is more obvious, expanding the application scene, making The application scenario is no longer limited to the situation where there are few shared windows.

In an example, the first device may encode the first image in an ARGB encoding manner to obtain the second image. In the ARGB encoding method, A represents the Alpha channel, which is used to indicate transparency, R represents the Red channel, which is used to indicate red, G represents the Green channel, which is used to indicate green, and B represents the Blue channel, which is used to indicate blue. When adopting the ARGB encoding method to encode the first image, the value of the Alpha channel of the non-window area in the first image can be set to a specified value, and the RGB channel (including the Red channel, the Green channel and the Blue channel) is normally encoded; The alpha channel and RGB channel of the window area in the first image are encoded normally. In practical applications, the value of the Alpha channel represents non-transparency. When the value of the Alpha channel is 0, it is completely transparent. When the value of the Alpha channel is 100%, it is completely opaque. The value of the Alpha channel is between 0 and 100%. Between means semi-transparent. In the embodiment of the present application, the value of the Alpha channel can be set to a value smaller than (100%-preset value), so that the transparency of the non-window area in the second image can be greater than or equal to the preset value.

Step S403, the first device sends the second image to the second device.

Step S404, the second device receives the second image.

Step S405, the second device displays the second image.

In this embodiment of the present application, the first image representing the screen currently displayed by the first device is encoded to obtain the second image, so that the area in the second image other than the area that needs to be shared by the first device and the second device The transparency is greater than or equal to the preset value, so that the window area sharing can be realized by transmitting the second image between the first device and the second device and displaying the second image on the second device, and it is not necessary to create an image for each window in the window area Establishing a separate encoding channel eliminates the need to create a floating window for each window in the window area, which reduces the delay and resource occupation, thereby improving the smoothness of electronic device operation during screen content sharing and improving user experience. For a scene with a large number of windows in the window area, it can also effectively improve the smoothness of the operation of the electronic device, and expand the application scene of screen content sharing.

FIG. 6 shows an interaction flowchart of an interaction method provided by an embodiment of the present application. This method can be applied to the interactive system shown in FIG. 3 . As shown in Figure 6, the method may include:

In step S501, the first device acquires a first image and position information of a window area in the first image.

For this step, reference may be made to step S401, which will not be repeated here.

In step S502, the first device encodes the first image based on the position information of the window area to obtain a second image, so that the transparency of areas other than the window area in the second image is greater than or equal to a preset value.

For this step, reference may be made to step S402, which will not be repeated here.

Step S503, the first device sends the second image and the location information of the window area to the second device.

In this step, the first device may first send the second image to the second device, and then send the location information of the window area to the second device; or send the location information of the window area to the second device first, and then send the location information of the window area to the second device Sending the second image; it is also possible to send the second image and the location information of the window area to the second device at the same time.

Step S504, the second device receives the second image and the location information of the window area.

Step S505, the second device displays the second image.

Step S506, the second device determines the anti-control area according to the location information of the window area.

In this step, the second device may first transform the position information of the window area, and then determine the anti-control area according to the transformed position information of the window area. Specifically, the area indicated by the position information of the converted window area in the screen displayed by the second device is the anti-control area, and the areas other than the anti-control area may be called self-control areas. The anti-control area refers to an area where reverse control is performed on the first device. The self-controlled area means an area that directly controls the second device itself. It can be understood that when the user operation received by the second device is in the anti-control area, it indicates that the user wants to operate the window of the first device and needs to control the first device; the user operation received by the second device is processed In the self-control area, it indicates that the user wants to operate the local window and needs to control the second device. Among them, the user operation in the anti-control area can be recorded as the second user operation, and the user operation in the self-control area can be recorded as the third user operation .

The second device can determine whether the user operation is in the anti-control area or the self-control area according to the coordinate information corresponding to the user operation, the position information of the anti-control area, and the position information of the self-control area, that is, determine whether the user operation is for the second part of the anti-control area. The user operation is also a third user operation directed at the self-control area. If the user operation is a second user operation, step S507 to step S514 may be performed; if the user operation is a third user operation, step S515 may be performed.

In step S507, the second device acquires second coordinate information and a second event type in response to a second user operation on the anti-control area.

The second coordinate information indicates the coordinate information corresponding to the second user operation, and the second event type indicates the event type corresponding to the second user operation. Refer to step S401 for the second coordinate information and the second event type, which will not be repeated here.

In step S508, the second device generates a counter-control message based on the second coordinate information and the second event type.

For the anti-control message, refer to step S401, which will not be repeated here.

In step S509, the second device returns a counter-control message to the first device.

In step S510, the first device acquires second coordinate information and a second event type from the counter-control message in response to the received counter-control message.

In step S511, the first device performs coordinate transformation on the second coordinate information.

In step S512, the first device generates a window control instruction based on the converted second coordinate information and the second event type.

Wherein, the window control instruction may be used to control the windows in the first device.

In step S513, the first device controls the displayed window area according to the window control instruction.

In this step, the first device may control the window area displayed on the first device according to the converted second coordinate information and the second event type corresponding to the window control instruction. For example, the position indicated by the second coordinate information is on the close button of window 1 displayed by the second device, and the second event type is closing the window, and the first device according to the converted second coordinate information and the second event type, Window 1 in the window area displayed on the first device may be closed.

In step S514, the first device executes step S501 in response to the window control instruction.

In this step, since the first device controls the displayed window area according to the window control instruction, for example, the windows in the window area are moved, the windows in the window area are closed, and one or more windows are moved out of the window area. It can be seen that the window area in the first device has changed, and the displayed picture has also changed. Therefore, the first device can reacquire the first image and the position of the window area in the first image in response to the window control command. Information, encoding the first image based on the position information of the window area to obtain a new second image, and then sending the new second image and the position information of the new window area to the second device. The second device displays the new second image, realizing the update of the shared display window area. The second device can determine a new anti-control area and self-control area according to the position information of the new window area, and realize the update of the anti-control area and the self-control area, so that the user operation is more accurate.

In step S515, the second device acquires third coordinate information and a third event type in response to a third user operation on the self-control area, and controls the second device based on the third coordinate information and the third event type.

In this step, the second device can control the local window, so that the use and operation of the second device itself will not be affected when the shared display is performed, thereby improving user experience.

In an example, the second device may directly inject the third coordinate information and the third event type into the second device by using a virtual driver technology, so as not to affect the operation and experience of the application on the second device itself.

In this embodiment of the application, based on the position information of the window area, the anti-control area and the self-control area in the second device are determined, and the first device is reversely controlled or the second device is controlled according to whether the user operation is in the anti-control area or the self-control area. The device performs local control. In this way, the effect of the second device counter-controlling the window area of the first device is realized without affecting the operation of the application of the second device itself, thereby improving user experience.

FIG. 7 shows a schematic structural diagram of an interaction device provided by an embodiment of the present application. This device can be applied to the first device 31 shown in FIG. 3 . As shown in FIG. 7 , the apparatus 70 may include: a first acquiring module 71, configured to acquire a first image and position information of a window area in the first image, wherein the first image indicates that the first device currently displays , the window area represents the shared display area of the first device and the second device in the first image; the encoding module 72 is configured to obtain the position information of the window area based on the first obtaining module 71 Encoding the first image acquired by the first acquisition module 71 to obtain a second image, so that the transparency of the area other than the window area in the second image is greater than or equal to a preset value; the first sending module 73, configured to send the second image obtained by the encoding module.

In this embodiment of the present application, the first image representing the screen currently displayed by the first device is encoded to obtain the second image, so that the area in the second image other than the area that needs to be shared by the first device and the second device The transparency is greater than or equal to the preset value, so that the window area sharing can be realized by transmitting the second image between the first device and the second device and displaying the second image on the second device, and it is not necessary to create an image for each window in the window area Establishing a separate encoding channel eliminates the need to create a floating window for each window in the window area, which reduces the delay and resource occupation, thereby improving the smoothness of electronic device operation during screen content sharing and improving user experience. For a scene with a large number of windows in the window area, it can also effectively improve the smoothness of the operation of the electronic device, and expand the application scene of screen content sharing.

In a possible implementation manner, the device further includes: a second sending module, configured to send the location information of the window area.

In a possible implementation manner, the first acquiring module is further configured to: acquire the first image and the position information of the window area in response to a window control instruction, and the window control instruction is used to control the window area displayed on the first device.

In a possible implementation manner, the device further includes: a second acquiring module, configured to acquire first coordinate information and a first event type in response to a first user operation, where the first user operation indicates that the A user operation performed on the window area displayed on the first device; a first generation module, configured to generate the window control instruction based on the first coordinate information and the first event type; a first control module, configured to follow The window control instruction controls the window area displayed on the first device.

In a possible implementation manner, the device further includes: a third acquiring module, configured to acquire the second coordinate information and the second event type from the counter-control message in response to the received counter-control message, the The anti-control message is generated according to a second user operation, and the second user operation represents a user operation performed on the window area displayed on the second device; a transformation module is configured to convert the second coordinate information Coordinate transformation; a second generating module, configured to generate the window control instruction based on the transformed second coordinate information and the second event type; a second control module, configured to generate the window control instruction according to the window control instruction control the window area displayed on the first device.

In a possible implementation manner, the first acquiring module is further configured to: acquire the first image and position information of the window area in response to a screen content sharing operation.

In a possible implementation manner, the window area includes one or more windows.

FIG. 8 shows a schematic structural diagram of an interaction device provided by an embodiment of the present application. This arrangement can be applied to the second device 32 shown in FIG. 3 . As shown in Figure 8, the device 80 may include:

The first receiving module 81 is configured to receive the second image, the transparency of the area other than the window area in the second image is greater than or equal to a preset value, and the window area indicates that the screen currently displayed by the first device is different from the second image. Areas where devices are shared;

The display module 82 is configured to display the second image received by the first receiving module 81 on the second device.

In this embodiment of the present application, the second device displays the second image, wherein the transparency of the area in the second image other than the shared display area between the first device and the second device is greater than or equal to a preset value, so that the first device Sharing the window area can be realized by transmitting the second image with the second device and displaying the second image on the second device. There is no need to establish a separate encoding channel for each window in the window area, and it is not necessary to create a separate encoding channel for each window in the window area. The window creates a floating window, which reduces the delay and resource occupation, thereby improving the smoothness of the operation of the electronic device when the screen content is shared, and improving the user experience. For a scene with a large number of windows in the window area, it can also effectively improve the smoothness of the operation of the electronic device, and expand the application scene of screen content sharing.

In a possible implementation manner, the device further includes: a second receiving module, configured to receive the location information of the window area; a determining module, configured to determine the second location information according to the location information of the window area. An anti-control area in the device; a first acquisition module, configured to acquire second coordinate information and a second event type in response to a second user operation on the anti-control area; a generation module, configured to obtain information based on the second coordinates information and the second event type, to generate a counter-control message; a return module, configured to return the counter-control message to control the first device.

In a possible implementation manner, the device further includes: a second acquiring module, configured to acquire third coordinate information and a third event type in response to a third user operation on a self-control area, where the self-control area represents the An area other than the anti-control area in the screen currently displayed by the second device; a control module configured to control the second device based on the third coordinate information and the third event type.

In a possible implementation manner, the window area includes one or more windows.

FIG. 9 shows a schematic structural diagram of an electronic device provided by an embodiment of the present application. Taking the electronic device as a mobile phone as an example, FIG. 9 shows a schematic structural diagram of a mobile phone 200 .

The mobile phone 200 may include a processor 210, an external memory interface 220, an internal memory 221, a USB interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 251, a wireless communication module 252, Audio module 270, speaker 270A, receiver 270B, microphone 270C, headphone jack 270D, sensor module 280, button 290, motor 291, indicator 292, camera 293, display screen 294, and SIM card interface 295, etc. The sensor module 280 may include a gyroscope sensor, an acceleration sensor, a proximity light sensor, a fingerprint sensor, a touch sensor, a temperature sensor, a pressure sensor, a distance sensor, a magnetic sensor, an ambient light sensor, an air pressure sensor, a bone conduction sensor, etc. (not shown in the figure) Shows).

It can be understood that the structure shown in the embodiment of the present application does not constitute a specific limitation on the mobile phone 200 . In other embodiments of the present application, the mobile phone 200 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 210 may include one or more processing units, for example: the processor 210 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (Neural-network Processing Unit, NPU) wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. Wherein, the controller may be the nerve center and command center of the mobile phone 200 . The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 210 for storing instructions and data. In some embodiments, the memory in processor 210 is a cache memory. The memory may hold instructions or data that the processor 210 has just used or recycled. If the processor 210 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 210 is reduced, thereby improving the efficiency of the system.

The processor 210 may run the method provided by the embodiment of the present application, so as to improve the fluency of the mobile phone 200 when sharing screen content. The processor 210 may include different devices. For example, when a CPU and a GPU are integrated, the CPU and the GPU may cooperate to execute the message processing method provided in the embodiment of the present application. For example, in the message processing method, part of the algorithm is executed by the CPU, and another part of the algorithm is executed by the GPU. for faster processing efficiency.

The display screen 294 is used to display images, videos and the like. Display 294 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the mobile phone 200 may include 1 or N display screens 294, where N is a positive integer greater than 1. The display screen 294 can be used to display information input by the user or information provided to the user and various graphical user interfaces (graphical user interface, GUI). For example, the display 294 can display photos, videos, web pages, or files, and the like. As another example, display 294 may display a graphical user interface. Wherein, the GUI includes a status bar, a hideable navigation bar, time and weather widgets (widgets), and application icons, such as browser icons. The status bar includes the operator name (such as China Mobile), mobile network (such as 4G), time and remaining power. The navigation bar includes a back key icon, a main screen (home) key icon and a forward key icon. In addition, it can be understood that, in some embodiments, the status bar may also include a Bluetooth icon, a Wi-Fi icon, an external device icon, and the like. It can also be understood that, in some other embodiments, the graphical user interface may further include a Dock bar, and the Dock bar may include commonly used application icons and the like. When the processor 210 detects a user's finger (or stylus, etc.) touch event on a certain application icon, in response to the touch event, the user interface of the application corresponding to the application icon is opened and displayed on the display 294. The app's user interface.

In the embodiment of the present application, the display screen 294 may be an integral flexible display screen, or a spliced display screen composed of two rigid screens and a flexible screen between the two rigid screens.

After the processor 210 runs the method provided by the embodiment of the present application, the mobile phone 200 can establish a connection with other electronic devices through the antenna 1, the antenna 2 or the USB interface, and transmit data and control the display screen 294 according to the method provided by the embodiment of the present application. The corresponding GUI is displayed. In the embodiment of the present application, data such as the second image, position information of the window area, anti-control message, etc. may be transmitted, and the display screen 294 may be controlled to display the first image or the second image.

A camera 293 (either a front camera or a rear camera, or a camera that can function as both a front camera and a rear camera) is used to capture still images or video. Generally, the camera 293 may include a photosensitive element such as a lens group and an image sensor, wherein the lens group includes a plurality of lenses (convex lens or concave lens), which are used to collect light signals reflected by objects to be photographed, and transmit the collected light signals to the image sensor . The image sensor generates an original image of the object to be photographed according to the light signal.

The internal memory 221 may be used to store computer-executable program codes including instructions. The processor 210 executes various functional applications and data processing of the mobile phone 200 by executing instructions stored in the internal memory 221 . The internal memory 221 may include an area for storing programs and an area for storing data. Wherein, the stored program area can store codes of operating systems and application programs (such as camera applications, WeChat applications, etc.). The data storage area can store data created during the use of the mobile phone 200 (such as images and videos collected by a camera application, etc.).

The internal memory 221 may also store one or more computer programs 1310 corresponding to the message processing method provided in the embodiment of the present application. The one or more computer programs 1304 are stored in the aforementioned memory 221 and are configured to be executed by the one or more processors 210 , the one or more computer programs 1310 comprising instructions.

In addition, the internal memory 221 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (universal flash storage, UFS) and the like.

Certainly, the code of the interaction method provided in the embodiment of the present application may also be stored in an external memory. In this case, the processor 210 may run the code of the interaction method stored in the external memory through the external memory interface 220 .

The following introduces the touch sensor 280K, also known as "touch panel". The touch sensor 280K can be arranged on the display screen 294, and the touch sensor 280K and the display screen 294 form a touch screen, also called “touch screen”. The touch sensor 280K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations can be provided through the display screen 294 . In some other embodiments, the touch sensor 280K may also be disposed on the surface of the mobile phone 200 , which is different from the position of the display screen 294 .

Exemplarily, the display screen 294 of the mobile phone 200 displays a main interface, which includes icons of multiple applications (such as camera applications, WeChat applications, etc.). The user clicks the icon of the camera application in the main interface through the touch sensor 280K, triggers the processor 210 to start the camera application, and turns on the camera 293 . The display screen 294 displays an interface of the camera application, such as a viewfinder interface.

The wireless communication function of the mobile phone 200 can be realized by the antenna 1, the antenna 2, the mobile communication module 251, the wireless communication module 252, the modem processor and the baseband processor.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in handset 200 can be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 251 can provide wireless communication solutions including 2G/3G/4G/5G applied on the mobile phone 200 . The mobile communication module 251 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 251 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 251 can also amplify the signal modulated by the modem processor, convert it into electromagnetic waves and radiate it through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 251 may be set in the processor 210 . In some embodiments, at least part of the functional modules of the mobile communication module 251 and at least part of the modules of the processor 210 may be set in the same device. In the embodiment of the present application, the mobile communication module 251 may also be used for information interaction with other electronic devices. For example, send the second image and the position information of the window area to the second device, receive the counter-control message sent by the second device, or receive the second image and the position information of the window area sent by the first device, and send the Counter-control news.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 270A, receiver 270B, etc.), or displays images or videos through display screen 294 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent of the processor 210, and be set in the same device as the mobile communication module 251 or other functional modules.

The wireless communication module 252 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite system, etc. applied on the mobile phone 200. (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 252 may be one or more devices integrating at least one communication processing module. The wireless communication module 252 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 210 . The wireless communication module 252 can also receive the signal to be transmitted from the processor 210 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation. In the embodiment of the present application, the wireless communication module 252 is used to transmit data between other electronic devices under the control of the processor 210. For example, when the processor 210 runs the message processing method provided in the embodiment of the present application, the processor can control The wireless communication module 252 sends data such as the second image to the second device.

In addition, the mobile phone 200 can realize the audio function through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the earphone interface 270D, and the application processor. Such as music playback, recording, etc. The mobile phone 200 can receive the input of the key 290 and generate key signal input related to the user setting and function control of the mobile phone 200 . The mobile phone 200 can use the motor 291 to generate a vibration prompt (such as a vibration prompt for an incoming call). The indicator 292 in the mobile phone 200 can be an indicator light, which can be used to indicate charging status, power change, and can also be used to indicate messages, missed calls, notifications and the like. The SIM card interface 295 in the mobile phone 200 is used for connecting a SIM card. The SIM card can be inserted into the SIM card interface 295 or pulled out from the SIM card interface 295 to realize contact and separation with the mobile phone 200 .

It should be understood that, in practical applications, the mobile phone 200 may include more or fewer components than those shown in FIG. 4 , which is not limited in this embodiment of the present application. The illustrated handset 200 is only one example, and the handset 200 may have more or fewer components than shown, may combine two or more components, or may have a different configuration of components. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing and/or application specific integrated circuits.

An embodiment of the present application provides an electronic device, including: a display screen for displaying a picture, a processor, and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions Implement the above method.

An embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, and the above method is implemented when the computer program instructions are executed by a processor.

An embodiment of the present application provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium bearing computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media include: portable computer disk, hard disk, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), erasable Electrically Programmable Read-Only-Memory (EPROM or flash memory), Static Random-Access Memory (Static Random-Access Memory, SRAM), Portable Compression Disk Read-Only Memory (Compact Disc Read-Only Memory, CD -ROM), Digital Video Disc (DVD), memory sticks, floppy disks, mechanically encoded devices such as punched cards or raised structures in grooves with instructions stored thereon, and any suitable combination of the foregoing .

Computer readable program instructions or codes described herein may be downloaded from a computer readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, local area network, wide area network, and/or wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present application may be assembly instructions, instruction set architecture (Instruction Set Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or in one or more source or object code written in any combination of programming languages, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer such as use an Internet service provider to connect via the Internet). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (Field-Programmable Gate Array, FPGA) or programmable logic arrays (Programmable Logic Array, PLA), the electronic circuit can execute computer-readable program instructions, thereby realizing various aspects of the present application.

Aspects of the present application are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart and/or block diagrams, and combinations of blocks in the flowchart and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowchart and block diagrams in the figures show the architecture, functions and operations of possible implementations of apparatuses, systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.

It should also be noted that each block in the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented with hardware (such as circuits or ASIC (Application Specific Integrated Circuit, application-specific integrated circuit)), or it can be realized by a combination of hardware and software, such as firmware.

Although the present invention has been described in conjunction with various embodiments herein, in the process of implementing the claimed invention, those skilled in the art can understand and Other variations of the disclosed embodiments are implemented. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that these measures cannot be combined to advantage.

Having described various embodiments of the present application above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Claims (25)

An interactive method, characterized in that the method comprises: Acquiring a first image and position information of a window area in the first image, wherein the first image represents the screen currently displayed by the first device, and the window area represents the first device in the first image an area for shared display with the second device; Encoding the first image based on the position information of the window area to obtain a second image, so that the transparency of areas other than the window area in the second image is greater than or equal to a preset value; The second image is sent. The method according to claim 1, further comprising: Send the location information of the window area. The method according to claim 1 or 2, wherein the acquiring the first image and the position information of the window area in the first image include: Acquire the first image and position information of the window area in response to a window control instruction, where the window control instruction is used to control the window area displayed on the first device. The method according to claim 3, characterized in that the method further comprises: Acquire first coordinate information and a first event type in response to a first user operation, where the first user operation represents a user operation performed on the window area displayed on the first device; generating the window control instruction based on the first coordinate information and the first event type; According to the window control instruction, the window area displayed on the first device is controlled. The method according to claim 3, characterized in that the method further comprises: In response to the received anti-control message, acquire second coordinate information and a second event type from the anti-control message, the anti-control message is generated according to a second user operation, and the second user operation indicates that the user operations performed on the window area displayed on the second device; performing coordinate transformation on the second coordinate information; generating the window control instruction based on the transformed second coordinate information and the second event type; According to the window control instruction, the window area displayed on the first device is controlled. The method according to claim 1 or 2, wherein said obtaining the first image, and the position information of the window area in the first image comprises: In response to a screen content sharing operation, acquire the first image and position information of the window area. The method according to any one of claims 1 to 6, wherein the window area includes one or more windows. An interactive method, characterized in that the method comprises: Receiving a second image, the transparency of the area in the second image other than the window area is greater than or equal to a preset value, and the window area represents an area shared with the second device in the screen currently displayed by the first device; The second image is displayed on the second device. The method according to claim 8, characterized in that the method further comprises: receiving position information of the window area; determining an anti-control area in the second device according to the location information of the window area; Acquire second coordinate information and a second event type in response to a second user operation on the anti-control area; generating a counter-control message based on the second coordinate information and the second event type; returning the anti-control message to control the first device. The method according to claim 9, characterized in that the method further comprises: Acquiring third coordinate information and a third event type in response to a third user operation on the self-control area, where the self-control area represents an area other than the anti-control area in the screen currently displayed by the second device; The second device is controlled based on the third coordinate information and the third event type. The method according to any one of claims 8 to 10, wherein the window area includes one or more windows. An interactive device, characterized in that the device comprises: The first acquisition module is configured to acquire a first image and position information of a window area in the first image, wherein the first image represents the screen currently displayed by the first device, and the window area represents the first In the image, the area where the first device and the second device perform shared display; An encoding module, configured to encode the first image acquired by the first acquisition module based on the position information of the window area acquired by the first acquisition module, to obtain a second image, so that the second image excludes the The transparency of the area outside the window area is greater than or equal to the preset value; The first sending module is configured to send the second image obtained by the coding module. The device according to claim 12, wherein the device further comprises: The second sending module is configured to send the location information of the window area. The device according to claim 12 or 13, wherein the first acquisition module is also used for: Acquire the first image and position information of the window area in response to a window control instruction, where the window control instruction is used to control the window area displayed on the first device. The device according to claim 14, further comprising: A second acquiring module, configured to acquire first coordinate information and a first event type in response to a first user operation, where the first user operation represents a user operation performed on the window area displayed on the first device; A first generating module, configured to generate the window control instruction based on the first coordinate information and the first event type; The first control module is configured to control the window area displayed on the first device according to the window control instruction. The device according to claim 14, further comprising: A third acquiring module, configured to acquire second coordinate information and a second event type from the counter-control message in response to the received counter-control message, the counter-control message is generated according to a second user operation, the The second user operation represents a user operation performed on the window area displayed on the second device; a transformation module, configured to perform coordinate transformation on the second coordinate information; A second generating module, configured to generate the window control instruction based on the transformed second coordinate information and the second event type; The second control module is configured to control the window area displayed on the first device according to the window control instruction. The device according to claim 12 or 13, wherein the first acquisition module is also used for: In response to a screen content sharing operation, acquire the first image and position information of the window area. The apparatus according to any one of claims 12 to 17, wherein the window area comprises one or more windows. An interactive device, characterized in that the device comprises: The first receiving module is used to receive the second image, the transparency of the area other than the window area in the second image is greater than or equal to a preset value, and the window area indicates that the screen currently displayed by the first device is compatible with the second device shared area; A display module, configured to display the second image received by the first receiving module on the second device. The device according to claim 19, further comprising: A second receiving module, configured to receive position information of the window area; A determination module, configured to determine an anti-control area in the second device according to the location information of the window area; A first acquiring module, configured to acquire second coordinate information and a second event type in response to a second user operation on the anti-control area; A generating module, configured to generate a counter-control message based on the second coordinate information and the second event type; A return module, configured to return the counter-control message to control the first device. The device according to claim 20, further comprising: The second acquisition module is configured to acquire third coordinate information and a third event type in response to a third user operation on the self-control area, where the self-control area indicates that the anti-control area is excluded from the screen currently displayed by the second device areas other than A control module, configured to control the second device based on the third coordinate information and the third event type. The apparatus according to any one of claims 19 to 21, wherein the window area comprises one or more windows. An electronic device, characterized in that it comprises: a display screen for displaying images; processor; memory for storing processor-executable instructions; Wherein, the processor is configured to implement the method of any one of claims 1 to 7, or to implement the method of any one of claims 8 to 11 when executing the instructions. A computer-readable storage medium, on which computer program instructions are stored, characterized in that, when the computer program instructions are executed by a processor, the method described in any one of claims 1 to 7 is implemented, or the method of any one of claims 1 to 7 is implemented, or the claims The method described in any one of 8 to 11. A computer program product, comprising computer-readable codes, or a non-volatile computer-readable storage medium bearing computer-readable codes, when the computer-readable codes run in an electronic device, the The processor executes the method of any one of claims 1-7, or the method of any one of claims 8-11.

Images