WO2025097978A1 - Image rendering method and electronic device - Google Patents

Abstract

The present application discloses an image rendering method and an electronic device. The method comprises: an electronic device displaying a first interface comprising a first component and a second component; in response to receiving an operation on the first interface, the electronic device determining whether the first component and the second component are focus nodes within a first refresh period; when the first component is a focus node and the second component is a non-focus node, within the first refresh period, rendering the first component by means of a first process, and rendering the second component by means of a second process, wherein the rendering priority of the first process is higher than that of the second process; and when the second process does not finish the rendering of the second component, reusing an existing texture image of the second component. In this way, the electronic device only needs to ensure real-time rendering of the focus node to which a user pays attention, and preferentially ensures the high frame rate of the focus node (such as a focus window/a focus control), so that smooth image display of the focus node is ensured, and the phenomena of stuttering and white screen are avoided.

Description

Image rendering method and electronic device

This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on November 6, 2023, with application number 202311471303.7 and application name “Image rendering method and electronic device”, the entire contents of which are incorporated by reference in this application.

Technical Field

The embodiments of the present application relate to the field of image drawing technology, and in particular to an image rendering method and an electronic device.

Background Art

The process of image drawing is generally: the electronic device receives a user's click operation on the application icon displayed on the display screen of the electronic device. In response to the operation, the electronic device receives the first refresh information at the first moment. The electronic device determines the information of the first frame image, such as the information of each control at the first refresh moment. The electronic device renders each control according to the information of the first frame image to complete the rendering of the entire window, and then draws to obtain the first frame image, and sends it to the display screen for display. Since the refresh cycle of the screen of the electronic device is fixed, at the second refresh moment, the electronic device receives the second refresh information and determines the information of the second frame image, such as the information of each control at the second refresh moment. The electronic device renders each control according to the information of the second frame image to complete the rendering of the entire window, and then draws to obtain the second frame image, and sends it to the display screen for display. In this way, by analogy, the electronic device obtains multiple frame images. These frame images are displayed sequentially on the screen of the electronic device, so that the screen can display an interface with animation effects at a constant frame rate. It can be seen that each control and each window need to be rendered during the drawing process, and in a high-load scenario, more controls and windows need to be rendered. Therefore, when faced with high-load scenarios, the window that the user is paying attention to, the entire machine, or the entire application process may become stuck, resulting in a poor user experience.

Summary of the invention

The image rendering method and electronic device provided in the embodiments of the present application give priority to ensuring the high frame rate of the focus component (such as the focus window/focus control), thereby improving the user's performance smoothness experience in high-load scenarios.

In order to achieve the above-mentioned purpose, the embodiment of the present application adopts the following technical solution.

In a first aspect, the present application provides an image rendering method, the execution subject of the method may be an electronic device, or a component located in the electronic device (for example, a chip, a chip system or a processor, etc.), and the following description is made by taking the execution subject being an electronic device as an example. The method may include: the electronic device displays a first interface, the first interface includes a first component and a second component, and the first interface is an interface of a first application. In response to receiving an operation on the first interface, the electronic device determines whether the first component and the second component are focus nodes within a first refresh cycle based on first information of the first component and first information of the second component. When the first component is a focus node and the second component is a non-focus node, the electronic device renders the first component through a first process within the first refresh cycle, and renders the second component through a second process, and the rendering priority of the first process is higher than the rendering priority of the second process.

The first component may be understood as a first window or a first control, and the second component may be understood as a second window or a second control.

The focus node can be understood as a node that needs to be processed first, such as the node currently operated by the user, that is, the node that the user pays attention to; or the node affected by the input event.

In this way, after receiving the user's operation on the interface, the electronic device determines whether each component on the interface is a focus node. If the first component is a focus node, the first process of the electronic device renders the first component in real time. If the second component is a non-focus node, the second process of the electronic device renders the second component. In this way, the electronic device only needs to ensure the real-time rendering of the focus node that the user is concerned about, and give priority to ensuring the high frame rate of the focus node (such as the focus window/focus control), thereby ensuring that the image of the focus node is displayed smoothly, without stuttering or white screen phenomenon, and improving the user's performance experience in high-load scenarios.

In some design solutions, the method further includes: when the second process has not completed rendering the second component within the first refresh cycle, the electronic device reuses the existing texture image of the second component within the second refresh cycle, and the second refresh cycle is the cycle before the first refresh cycle. In this way, when the second process has not completed rendering the second component, the existing texture image of the second component can be reused, and there is no need to render the second component in real time. It is sufficient to ensure the real-time rendering of the focus node that the user is concerned about, and give priority to ensuring the high frame rate of the focus node (such as the focus window/focus control).

In some embodiments, the method further includes: the electronic device displays a second interface, the second interface includes a third component, The second interface is an interface of a second application, and the second application is different from the first application. In response to an operation on the second interface, the electronic device determines whether the first component, the second component, and the third component are focus nodes in a first refresh cycle according to the first information of the first component, the first information of the second component, and the first information of the third component. When the third component is a focus node and the first component and the second component are both non-focus nodes, the electronic device renders the third component through the first process in the first refresh cycle, and renders the first component and the second component through the second process.

When there are multiple non-focus nodes, the electronic device determines the rendering priority of each non-focus node. After that, the electronic device renders the components corresponding to each non-focus node according to the rendering priority of each non-focus node, thereby achieving orderly rendering of each component and alleviating the tension of CPU and GPU resources in high-load scenarios.

In one design solution, the rendering of the first component and the second component through the second process is specifically: the electronic device determines the rendering priority of the first component and the second component according to the second information of the first component and the second information of the second component, and the second information and its weight. When the rendering priority of the first component is higher than the rendering priority of the second component, the electronic device renders the first component and the second component in sequence through the second process.

In one design scheme, the second information includes at least one of the following: a visible window without generated cache, an invisible window without generated cache, a window with generated cache and visible dirty area, a window with generated cache and only invisible dirty area, and a static window with generated cache.

In one design, the first information includes one or more of transparency, a focus window of a window management service, an overlapping order, whether it is blocked, and whether animation is performed.

In one design solution, the first information includes transparency, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component is specifically:

When the transparency of the first component is greater than the transparency of the second component, it is determined that the first component is a non-focus node and the second component is a focus node.

In one design solution, the first information includes a focus window of a window management service, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component is specifically:

When it is detected that the user operates the first component but does not operate the second component, it is determined that the first component is a focus node and the second component is a non-focus node.

In one design solution, the first information includes a focus window of a window management service, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component is specifically:

When it is detected that the user's sight falls on the first component and does not fall on the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node.

In one design solution, the first information includes a stacking order, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component is specifically:

When the stacking order of the first component is higher than the stacking order of the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node.

In one design solution, the first information includes whether to perform animation, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component is specifically:

When the image of the first component is a multi-frame image and the image of the second component is a single-frame image, the first component is determined to be a focus node and the second component is determined to be a non-focus node.

In one design, the first information includes transparency, a focus window of a window management service, an overlapping order, whether it is blocked, and whether animation is performed; and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component is specifically:

According to the following information of the first component and the second component: transparency and its weight, the focus window of the window management service and its weight, the stacking order and its weight, whether it is blocked and its weight, whether to perform animation and its weight, determine whether the first component and the second component are focus nodes.

In one design solution, the component includes a window or a control.

In one design, the focus node is a node that needs to be processed first, such as a node currently operated by a user, that is, a node that the user is paying attention to; or a node affected by an input event.

In a second aspect, the present application provides an electronic device, comprising: a display module, a determination module and a rendering module, wherein the display module is used to display a first interface, the first interface includes a first component and a second component, and the first interface is an interface of a first application. The determination module is used to determine whether the first component and the second component are focus nodes in a first refresh cycle in response to receiving an operation on the first interface, according to first information of the first component and first information of the second component, and the focus node is a node that the user is concerned about. The rendering module is used to render the first component through a first process in the first refresh cycle, and render the second component through a second process when the first component is a focus node and the second component is a non-focus node, and the rendering priority of the first process is higher than the rendering priority of the second process.

The first component may be understood as a first window or a first control, and the second component may be understood as a second window or a second control.

The focus node is a node that needs to be processed first, such as a node currently operated by a user, that is, a node that the user is paying attention to; or a node affected by an input event.

In this way, after receiving the user's operation on the interface, the electronic device determines whether each component on the interface is a focus node. If the first component is a focus node, the first process of the electronic device renders the first component in real time. If the second component is a non-focus node, the second process of the electronic device renders the second component. In this way, the electronic device only needs to ensure the real-time rendering of the focus node that the user is concerned about, and give priority to ensuring the high frame rate of the focus node (such as the focus window/focus control), thereby ensuring that the image of the focus node is displayed smoothly, without stuttering or white screen phenomenon, and improving the user's performance experience in high-load scenarios.

In one design, the rendering module is used to reuse the existing texture image of the second component in the second refresh cycle when the second process has not completed rendering the second component in the first refresh cycle, and the second refresh cycle is the cycle before the first refresh cycle. In this way, when the second process has not completed rendering the second component, the existing texture image of the second component can be reused, and there is no need to render the second component in real time. It is sufficient to ensure the real-time rendering of the focus node that the user is concerned about, and give priority to ensuring the high frame rate of the focus node (such as the focus window/focus control).

In one design scheme, the display module is used to display a second interface, the second interface includes a third component, the second interface is an interface of a second application, and the second application is different from the first application. The determination module is used to determine whether the first component, the second component, and the third component are focus nodes in a first refresh cycle in response to an operation on the second interface, according to the first information of the first component, the first information of the second component, and the first information of the third component. The rendering module is used to render the third component through the first process in the first refresh cycle, and render the first component and the second component through the second process when the third component is the focus node and the first component and the second component are both non-focus nodes.

When there are multiple non-focus nodes, the electronic device determines the rendering priority of each non-focus node. After that, the electronic device renders the components corresponding to each non-focus node according to the rendering priority of each non-focus node, thereby achieving orderly rendering of each component and alleviating the tension of CPU and GPU resources in high-load scenarios.

In one design scheme, the rendering module is used to: determine the rendering priority of the first component and the second component based on the second information of the first component and the second information of the second component, and the second information and its weight; when the rendering priority of the first component is higher than the rendering priority of the second component, render the first component and the second component in sequence through the second process.

In one design scheme, the second information includes at least one of the following: a visible window without generated cache, an invisible window without generated cache, a window with generated cache and visible dirty area, a window with generated cache and only invisible dirty area, and a static window with generated cache.

In one design, the first information includes one or more of transparency, a focus window of a window management service, an overlapping order, whether it is blocked, and whether animation is performed.

In one design solution, the first information includes transparency, and the determining module is configured to:

When the transparency of the first component is greater than the transparency of the second component, it is determined that the first component is a non-focus node and the second component is a focus node.

In one design solution, the first information includes a focus window of a window management service, and the determining module is configured to:

When it is detected that the user operates the first component but does not operate the second component, it is determined that the first component is a focus node and the second component is a non-focus node.

In one design solution, the first information includes a focus window of a window management service, and the determining module is configured to:

When it is detected that the user's sight falls on the first component and does not fall on the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node.

In one design solution, the first information includes an overlay order, and the determining module is configured to:

When the stacking order of the first component is higher than the stacking order of the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node.

In one design solution, the first information includes whether to perform animation, and the determination module is used to:

When the image of the first component is a multi-frame image and the image of the second component is a single-frame image, the first component is determined to be a focus node and the second component is determined to be a non-focus node.

In one design, the first information includes transparency, a focus window of a window management service, an overlapping order, whether it is blocked, and whether animation is performed; and the determining module is used to:

Determine whether the first component and the second component are focus nodes based on the following information of the first component and the second component: transparency and its weight, focus window of the window management service and its weight, stacking order and its weight, whether it is blocked and its weight, whether it is animated and its weight.

In one design solution, the component includes a window or a control.

In one design, the focus node is a node that needs to be processed first, such as a node currently operated by a user, that is, a node that the user is paying attention to; or a node affected by an input event.

In a third aspect, the present application provides an electronic device, comprising: one or more processors; and a memory, wherein the memory stores a code; when the code is executed by the processor, the electronic device executes the method described in the first aspect.

In a fourth aspect, the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the method described in the first aspect.

Among them, the specific implementation methods and corresponding technical effects of each of the second to fourth aspects mentioned above can refer to the specific implementation methods and technical effects of the first aspect mentioned above.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

FIG2 is a schematic diagram of the architecture of the combination of software and hardware of the display system of the electronic device provided in an embodiment of the present application;

FIG3-1 is a schematic diagram of a flow chart of an image rendering method provided in an embodiment of the present application;

FIG3-2 is a schematic diagram of a flow chart of an image rendering method provided in an embodiment of the present application;

FIG4 is a logic diagram of a separate rendering provided by an embodiment of the present application;

FIG5 is a logic diagram of unified rendering provided by an embodiment of the present application;

6 is a schematic diagram of a main and secondary thread node processing timing of an image rendering method provided by an embodiment of the present application;

FIG7 is a schematic diagram of a flow chart of an image rendering method provided in an embodiment of the present application;

FIG8 is a schematic diagram of an interface of another electronic device provided in an embodiment of the present application;

FIG9 is a schematic diagram of an interface of another electronic device provided in an embodiment of the present application;

FIG. 10 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

DETAILED DESCRIPTION

A window is a basic unit in a graphical user interface that an application sets up to use data. It is a visual interface of an application. The application and data are integrated in the window. In the window, users can operate the application in the window to manage, generate and edit data.

A window is a container for controls, and each window has its own controls (or views, etc.). A control is a graphical user interface element, such as a window or text box, whose displayed arrangement of information can be changed by the user. And, as a basic visual building block, a control is included in an application and controls all data processed by the application and the interactions with that data. Interoperability.

The process of drawing the above interface is generally: the electronic device receives an operation on the interface displayed on the display screen of the electronic device. Exemplarily, the electronic device receives a user's click operation on an icon (such as an icon for setting an application) displayed on the display screen of the electronic device. In response to the operation, the electronic device receives the first refresh information at the first moment. The electronic device determines the information of the first frame image, such as the information of each control at the first refresh moment. The electronic device renders each control according to the information of the first frame image to complete the rendering of the entire window, and then draws to obtain the first frame image, and sends it to the display screen for display. Since the refresh cycle of the screen of the electronic device is fixed, at the second refresh moment, the electronic device receives the second refresh information and determines the information of the second frame image, such as the information of each control at the second refresh moment. The electronic device renders each control according to the information of the second frame image to complete the rendering of the entire window, and then draws to obtain the second frame image, and sends it to the display screen for display. In this way, by analogy, the electronic device obtains multiple frame images. These frame images are displayed sequentially on the screen of the electronic device, so that the screen can display an interface with animation effects at a constant frame rate.

It can be seen that each control and each window needs to be rendered during the drawing process. In high-load scenarios, more controls and windows need to be rendered, and frame loss, white screen, freeze, etc. may occur, affecting the user experience.

In order to solve the above technical problems, the present application provides an image rendering method, which is applied to an electronic device, and the method may include: the electronic device displays a first interface, the interface includes a first component and a second component, and the first component and the second component can be windows or controls. In response to receiving an operation on the first interface, the electronic device determines whether the first component and the second component are focus nodes in the first refresh cycle, and the focus node is a node that the user is concerned about. When the first component is a focus node and the second component is a non-focus node, the first component is rendered by the first process in the first refresh cycle, and the second component is rendered by the second process, and the rendering priority of the first process is higher than the rendering priority of the second process. That is, after the electronic device receives the user's operation on the interface, the electronic device determines whether each component on the interface is a focus node. If the first component is a focus node, the first process of the electronic device renders the first component in real time. If the second component is a non-focus node, the second process of the electronic device renders the second component, and the texture image of the existing second component can be reused when the second process has not finished rendering the second component. In this way, the electronic device only needs to ensure the real-time rendering of the focus node that the user is concerned about, and give priority to ensuring the high frame rate of the focus node (such as the focus window/focus control), so as to ensure that the image of the focus node is displayed smoothly without any stuttering or white screen phenomenon, thereby improving the user's performance experience in high-load scenarios.

FIG1 is a structural block diagram of an electronic device provided in an embodiment of the present application.

As shown in FIG1 , the electronic device 100 may include a processor 110, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, a memory 120, an antenna 1, a wireless communication module 160, a display screen 170, and a sensor module 150, etc. The sensor module 150 may include a pressure sensor 150A, a touch sensor 150B, etc.

It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and/or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated in one or more processors.

The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

The charging management module 140 is used to receive charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 can receive wireless charging input through a wireless charging coil of the electronic device. While the charging management module 140 is charging the battery 142, it can also charge the battery 142 through the power management module 141. Provides power to electronic devices.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, and supplies power to the processor 110, the display screen 170, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

The wireless communication function of the electronic device can be implemented through the antenna 1, the wireless communication module 160, the modem processor and the baseband processor.

Antenna 1 is used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) and the like applied to electronic devices. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 1, modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency, amplify it, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, the wireless communication module 160 receives application information sent by the server.

The electronic device implements the display function through a GPU, a display screen 170, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 170 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 170 is used to display images, videos, etc. The display screen 170 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diodes (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 170, where N is a positive integer greater than 1.

The memory 120 may be used to store computer executable program codes, which include instructions. The memory 120 may include a program storage area and a data storage area. The program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device (such as audio data, a phone book, etc.), etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device by running instructions stored in the memory 120 and/or instructions stored in a memory provided in the processor.

The pressure sensor 150A is used to sense the pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 150A can be set on the display screen 170. There are many types of pressure sensors 150A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can be a parallel plate including at least two conductive materials. When a force acts on the pressure sensor 150A, the capacitance between the electrodes changes. The electronic device determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 170, the electronic device detects the touch operation intensity according to the pressure sensor 150A. The electronic device can also calculate the touch position according to the detection signal of the pressure sensor 150A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.

The touch sensor 150B is also called a "touch control device". The touch sensor 150B can be set on the display screen 170. The touch sensor 150B and the display screen 170 form a touch screen, also called a "touch control screen". The touch sensor 150B is used to detect a touch operation acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation may be provided through the display screen 170. In other embodiments, the touch sensor 150B may also be disposed on the surface of the electronic device, at a position different from that of the display screen 170.

Of course, the electronic device may also include other functional units, which are not limited in the embodiments of the present application.

In addition, the actions, terms, etc. involved in the various embodiments of the present application can refer to each other without limitation. The names of the messages or parameter names in the messages in the embodiments of the present application are only examples, and other names can also be used in the specific implementation without limitation.

Among them, the above-mentioned electronic devices can be mobile phones, tablet computers, laptops, notebook computers, ultra-mobile personal computers (UMPC), handheld computers, netbooks, personal digital assistants (PDA), wearable electronic devices and other devices. The specific form of the electronic devices is not particularly limited in the embodiments of the present application.

In some embodiments, the software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. Exemplarily, taking the layered architecture adopted by the software system of the electronic device 100 as an example, in combination with the image rendering scheme described in FIG. 2 , FIG. 2 shows a schematic diagram of the structure of the combination of software and hardware of the electronic device 100 in the image rendering scheme.

The layered architecture divides the software into several layers, each with a clear role and division of labor. The layers communicate with each other through software interfaces.

As shown in FIG. 2 , the electronic device 100 includes at least: an application layer (or application program layer), an application architecture layer, a system service layer and a hardware layer.

The application layer includes a series of application packages, such as settings, desktop, system applications, third-party applications, game applications, video applications, etc. Third-party applications may include but are not limited to third-party applications downloaded from application markets, such as social applications, shopping applications (not shown in the figure), etc. The windows/controls of each application may be rendered separately, or they may be rendered uniformly by the rendering service, and the rendered windows may be submitted to the synthesizer of the system service layer for synthesis. In some embodiments, the application in the application layer may submit rendering instructions to the rendering service in the system service layer to complete the rendering of the application interface, etc.

The application architecture layer may include Rosen rendering backend, user program framework, Ability framework, and Window.

The system service layer includes rendering service, window manager service (WMS), data management service (DMS), production management system (PMS), application management service (AMS), intranet management system (IMS) and compositor. Among them, WMS is used to manage the windows of each application, such as desktop window, SystemUI window, negative one screen window, etc. For example, WMS can manage window type, window level, window position, calculate window layout, and perform window form (such as full screen, floating window, split screen, maximize, minimize, etc.) switching management.

The rendering service includes an animation module and a unified rendering module. The animation module can be used to modify the properties (such as position, size, etc.) of the control in each frame (or image) included in the animation to generate an animation effect. Optionally, the properties of the control are different in different frames.

Optionally, the above-mentioned animation module may also be located in the application layer and executed through the application user interface (UI) thread of the application.

The unified rendering module can perform rendering services based on the unified rendering mechanism, which can also be called the unified rendering service. The unified rendering mechanism means that the unified rendering module can receive rendering nodes of all applications and then render all rendering nodes of all applications together. Specifically, it can traverse all rendering nodes based on the control properties modified by the animation module to complete the rendering operation, and then send the rendering results to the display screen for display.

In the embodiment of the present application, a rendering node may also be referred to as a layer, a layer or other names.

The hardware layer includes a synthesizer, which can be used to synthesize the layers corresponding to each window, such as the desktop layer, the SystemUI layer, the negative one screen layer, etc. Among them, the desktop window corresponds to the desktop layer, the SystemUI window corresponds to the SystemUI layer, the negative one screen window corresponds to the negative one screen layer, etc. Exemplarily, the synthesizer may include a hardware synthesizer and a graphics processing unit (GPU), etc.

It is understood that the software and hardware combined architecture of the electronic device 100 shown in FIG2 is only an exemplary description. In actual applications, it may also include more or fewer modules, and the levels to which each module belongs may also be different. For example, there may also be a framework module between the unified application and the rendering service. Each level may also have other division methods, and this application does not limit this.

The technical solutions involved in the following embodiments can all be implemented in a device having the structure shown in FIG. 1 and FIG. 2 .

FIG3-1 and FIG3-2 are schematic flow charts of an image rendering method provided in an embodiment of the present application. As shown in FIG3-2 , the method is applied to an electronic device. The method may include:

S301. An electronic device displays a first interface, where the first interface includes at least one component, such as a first component and a second component, and the first interface is an interface of a first application.

It can be understood that the first component and the second component may belong to the same application.

In one example, the above components (such as the first component and the second component) can be understood as windows or controls. Accordingly, the first component can be understood as the first window or the first control, and the second component can be understood as the second window or the second control. Similarly, the subsequent third component can also be understood as the third window or the third control.

In one example, a window is a container for controls, and each window has its own control (or view, or widget, etc.). A control is a graphical user interface element, such as a window or text box, whose displayed arrangement of information can be changed by the user. Moreover, a control is a basic visual building block included in an application, controlling all data processed by the application and the interactive operations on the data.

In one example, a window is the most important part of a user interface. It is a rectangular area on the screen of an electronic device corresponding to an application and is a visual interface of an application. When a user starts an application and the application starts running, the application creates and displays a window. In some embodiments, the user can also perform various operations on the window, such as: when the user operates an object in the window, the application will respond accordingly. For another example: the user can terminate the operation of the application by closing the window. For another example: the user can also select the corresponding application by selecting the window.

S302: In response to receiving an operation on the first interface, the electronic device determines whether the first component and the second component are focus nodes in a first refresh cycle, where the focus node is a node that the user is paying attention to.

In a specific implementation, as shown in FIG3-1, in step ①, in response to receiving an operation on the interface of the application, the focus node determination module of the electronic device determines whether the first component and the second component are focus nodes within the first refresh cycle.

In one example, the operation can be understood as any operation on the first interface, such as a click operation, a browse operation, a slide operation, an edit operation, a cursor touch operation, etc.

In a specific implementation manner, S302 may be specifically implemented as follows: the electronic device determines, according to the first information of the first component and the second component, whether the first component and the second component are focus nodes in a first refresh cycle.

The first information is used to distinguish between focus nodes and non-focus nodes. Since the determination of focus nodes and non-focus nodes is based on the user's attention, that is, the component that the user pays attention to is the focus node, and the component that the user does not pay attention to is the non-focus node. Therefore, the first information tends to be more visible information, so the first information can include one or more of transparency, the focus window of the window management service, the stacking order (z-order), whether it is blocked, and whether animation is performed.

In a specific implementation, the first information includes transparency, and S302 can be specifically implemented as follows: when the transparency of the first component is greater than the transparency of the second component, the electronic device determines that the first component is a non-focus node and the second component is a focus node. Exemplarily, assuming that the first information of the first component includes a transparency of 50%, and the first information of the second component includes a transparency of 10%, and 50%>10%, the electronic device determines that the first component is a non-focus node and the second component is a focus node.

In a specific implementation, the first information includes the focus window of the window management service, that is, the window management service detects the user's window operation, for example, the window management service detects the component that the mouse is hovering over, or the window management service detects that the display position of the component is located in the middle area of the display screen, or the window management service detects the component affected by the user's input event. S302 can be specifically implemented as follows: when it is detected that the user operates the first component and does not operate the second component, the electronic device determines that the first component is the focus node and the second component is the non-focus node. Exemplarily, the first component is the first window and the second component is the second window. When the user edits on the first window, it means that the user pays more attention to the first window. At this time, the electronic device detects the user's operation and determines that the first window is the focus window and the second window is the non-focus window.

In a specific implementation, the first information includes a focus window of a window management service, for example, the window management service detects the user's sight. S302 can be specifically implemented as follows: when it is detected that the user's sight falls on the first component and does not fall on the second component, the electronic device determines that the first component is a focus node and the second component is a non-focus node.

In a specific implementation method, the first information includes a stacking order, and S302 can be specifically implemented as follows: when the stacking order of the first component is the top layer, and the stacking order of the second component is the bottom layer, that is, the setting level of the first component is closer to the user interface than the setting level of the second component. At this time, the electronic device determines that the first component is a focus node and the second component is a non-focus node.

In a specific implementation, the first information includes whether to perform animation, and S302 can be specifically implemented as follows: when the image of the first component When the image is a multi-frame image (ie, animation or video display) and the image of the second component is a single-frame image (ie, picture display), the electronic device determines that the first component is a focus node and the second component is a non-focus node.

In a specific implementation manner, the first information includes whether it is blocked, and S302 can be specifically implemented as follows: when it is detected that the first component is not blocked and the second component is blocked, the electronic device determines that the first component is a focus node and the second component is a non-focus node.

Of course, the above implementations can be combined with each other. Specifically, the first information may include transparency and whether it is blocked, or the first information may include the stacking order and the focus window of the window management service, or the first information may include transparency, whether to animate, and whether it is blocked. The embodiments of the present application are not specifically limited. When the first information includes multiple dimensional information, the electronic device can determine whether each component is a focus node based on each dimensional information and its weight.

For example, it is assumed that the first information includes transparency, the focus window of the window management service, the stacking order, whether it is blocked, and whether animation is performed. The weight of a large transparency is 0, and the weight of a small transparency is 1; the focus window of the window management service is 1, and the non-focus window of the window management service is 0; the weight of a stacking order closer to the user is 1, and vice versa; the weight of a blocked window is 0, and the weight of a non-blocked window is 1; the weight of an animated window is 1, and the weight of a non-animated window is 0.

For example, assuming that the first information of the first component may include: the user operates the first component, the transparency is 50%, the stacking order is the top layer, it is not blocked, and animation is performed; the first information of the second component may include: the transparency is 10%, the stacking order is the bottom layer, it is blocked, and no animation is performed. Then, the score of the first component is 0+1+1+1=3, and the score of the second component is 1+0+0+0=1, 3>1, and the electronic device can determine that the first component is a focus node and the second component is a non-focus node.

For example, assuming that the first information of the first component may include: the first component is located in the middle of the display area, is not blocked, and is animated; the first information of the second component may include: the first component is located in the lower part of the display area, is not blocked, and is animated. Then, the score of the first component is 1+1+1=3, and the score of the second component is 0+1+1=2, 3>2, and the electronic device may determine that the first component is a focus node and the second component is a non-focus node.

Before introducing S303, two rendering architectures to which the technical solution provided by the embodiment of the present application can be applied are first introduced here. These two rendering architectures can include a separate rendering architecture and a unified rendering architecture. That is, the graphics architecture of the operating system of the electronic device provided by the embodiment of the present application can adopt a separate rendering architecture or a unified rendering architecture. The specific contents are as follows:

First, electronic devices adopt a separate rendering architecture.

FIG4 shows a logic diagram of separate rendering. As shown in FIG4 , each application window (referred to as window) can be rendered through a control tree and a rendering tree. Among them, the rendering tree is a data structure for generating an application program interface. That is, the rendering tree records all the information for generating a frame of the application program. Exemplarily, the control tree of the window may include multiple control modules (or controls), such as a root control, control 1, control 2, control 3, etc. Each control includes control information of the corresponding control, such as control 1 includes control information of control 1 in the window (such as the display position, etc.), and the control information can be used to control the display of control 1. Each control in the control tree corresponds to each rendering node in the rendering tree. The rendering tree may include multiple rendering nodes, such as rendering node 0, rendering node 1, rendering node 2, rendering node 3, etc. Correspondingly, the root control may correspond to rendering node 0, control 1 may correspond to rendering node 1, control 2 may correspond to rendering node 2, and control 3 may correspond to rendering node 3. According to the above, it is determined whether rendering node 0, rendering node 1, rendering node 2, and rendering node 3 are focus nodes. Assume that rendering node 2 is determined to be the focus node, and rendering node 0, rendering node 1, and rendering node 3 are non-focus nodes. As described below, rendering node 2 is assigned to the first process for rendering, that is, the first process renders control 2 according to the control information of control 2. Rendering node 0, rendering node 1, and rendering node 3 are assigned to the second process for rendering. That is, the second process renders the root control according to the control information of the root control, the second process renders control 1 according to the control information of control 1, and the second process renders control 3 according to the control information of control 3. Then, the rendering results of each process are submitted to the synthesizer, wherein each application window corresponds to a layer in the synthesizer, and the synthesizer can synthesize each application window, that is, synthesize the layer corresponding to each application window. In an embodiment of the present application, the rendering tree may also be referred to as a layer tree, a layer tree, or other names.

Among them, under the above-mentioned separated rendering architecture, the hierarchical relationship of each window, the size of the window, etc. are managed by the window manager service (WMS). That is, the hierarchical management and size of the layers corresponding to each window are managed by WMS.

Second, electronic devices adopt a unified rendering architecture.

FIG5 shows a logic diagram of a unified rendering mechanism provided by an embodiment of the present application. As shown in FIG5, compared with the above-mentioned separate rendering architecture, the unified rendering architecture has a unified application, which can be used to abstract each window into a control for management, and each window is a control in the unified application. For example, as shown in FIG5, the unified application can manage controls 1, The layout and hierarchy of various controls such as control 2, container control 3, window control 4, and status bar control 5. Among them, control 1 and control 2 can be existing defined controls, such as icons. Both window control 4 and status bar control 5 can be child nodes of container control 3. The unified application can generate a corresponding rendering tree based on the control tree. For example, the rendering nodes in the generated rendering tree can include a root rendering node and rendering nodes 1 to 5, and the unified application submits the rendering tree to the rendering service. It can be understood that control 1 corresponds to rendering node 1, control 2 corresponds to rendering node 2, container control 3 corresponds to rendering node 3, window control 4 corresponds to rendering node 4, and status bar control 5 corresponds to rendering node 5. The rendering service traverses each rendering node and determines whether each rendering node is a focus node according to the above description, and assigns the focus node to the main thread (i.e., the first process mentioned above) for rendering, and assigns the non-focus node to the secondary thread (i.e., the second process mentioned above) for rendering, so as to complete the rendering of the entire interface. Subsequently, the rendering result can be directly sent to the display screen for display.

In some embodiments, the unified application can be understood as a virtualized application (or service) with the functions of various applications such as WMS, desktop, SystemUI, etc. For example, the unified application can have functions similar to WMS, such as managing the position, size, and hierarchy of various controls (such as but not limited to window controls, etc.), and having functions such as split screen and full screen gesture management. It can also have functions similar to the desktop, such as managing icons, etc., and having functions such as multi-tasking gesture management, etc.

The two rendering architectures shown in Figures 4 and 5 are introduced here to illustrate that the technical solutions provided in the embodiments of the present application can at least be applied to these two rendering architectures. It can be understood that the technical solutions adopted in the embodiments of the present application can also be applied to other rendering architectures, which are not listed one by one in the embodiments of the present application.

S303: When the first component is a focus node and the second component is a non-focus node, the first component is rendered by a first process in a first refresh cycle, and the second component is rendered by a second process, and the rendering priority of the first process is higher than the rendering priority of the second process.

That is to say, there are parallel rendering processes in the electronic device. The process with a high rendering priority renders the focus node, and the process with a low rendering priority renders the non-focus node. In this way, for the node that the user is concerned about (i.e., the focus node), the focus node is rendered in real time to ensure a high frame rate; for the node that the user is not concerned about (i.e., the non-focus node), the non-focus node is rendered while ensuring the real-time rendering of the focus node, so as to improve the user's experience of the smooth performance of the graphics of the focused node.

Exemplarily, as shown in Figure 6, assuming that window A is the focus node and window C is the non-focus node, when the electronic device receives the first refresh signal, the first process of the electronic device renders window A, and the second process of the electronic device renders window C, and displays the rendered image on the screen.

In a specific implementation, as shown in FIG3-1, in step ②, the focus node determination module of the electronic device determines that the first component is a focus node and the second component is a non-focus node. Afterwards, the focus node determination module of the electronic device sends the determination result to the rendering service of the electronic device, and the rendering service assigns the rendering task corresponding to the first component to the first process for rendering, and assigns the rendering task corresponding to the second component to the second process for rendering according to the determination result.

In an embodiment of the present application, after receiving an operation of the user on the interface, the electronic device determines whether each component on the interface is a focus node. If the first component is a focus node, the first process of the electronic device renders the first component in real time. If the second component is a non-focus node, the second process of the electronic device renders the second component. In this way, the electronic device only needs to ensure the real-time rendering of the focus node that the user is concerned about, and gives priority to ensuring the high frame rate of the focus node (such as the focus window/focus control), thereby ensuring that the image of the focus node is displayed smoothly, without stuttering or white screen phenomenon, and improving the user's performance smoothness experience in high-load scenarios.

S304: when the second process has not completed rendering of the second component within the first refresh cycle, reuse the existing texture image of the second component within the second refresh cycle, where the second refresh cycle is a cycle before the first refresh cycle.

That is to say, if the second process has not completed the rendering of the second component within the first refresh cycle, the cached texture image of the second component will be reused. In this way, for the node that the user is concerned about (i.e., the focus node), the focus node is rendered in real time to ensure a high frame rate; for the nodes that the user is not concerned about (i.e., non-focus nodes), while ensuring the real-time rendering of the focus node, when the rendering of the non-focus node is not completed, the existing texture image of the non-focus node is reused, thereby improving the user's experience of the smooth performance of the graphics of the focused node.

Continuing with the above example, as shown in FIG6 , when the electronic device receives the first refresh signal, the first process of the electronic device renders window A, and the second process of the electronic device renders window C. However, when the second process has not completed the rendering of window C within the first refresh cycle, the electronic device displays the image rendered by the first process and the existing texture image of window C (such as the image rendered within the second refresh cycle) on the screen. After the second process completes the rendering of window C, the texture image of window C is cached.

After the rendering is finished, as shown in FIG3-1, in step ③, the rendering service of the electronic device sends the rendered image to the synthesizer, which synthesizes the rendering results. Then, in step ④, the synthesizer sends the synthesized image to the display screen for display.

In some embodiments, the embodiment of the present application is not limited to the first component and the second component, and may also include a third component, which may belong to the same application as the first component and the second component, or may not belong to the first component and the second component. Exemplarily, taking the third component as an example of not belonging to the first component and the second component, FIG7 is a flow chart of an image rendering method provided by the embodiment of the present application. As shown in FIG7, the method provided by the embodiment of the present application includes the above-mentioned S301, and the method also includes:

S701. The electronic device displays a second interface, where the second interface includes a third component and is an interface of a second application, which is different from the first application.

Specifically, the detailed description of S701 can be found in the above S301, which will not be repeated here.

S702 . In response to an operation on the second interface, determine whether the first component, the second component, and the third component are focus nodes in a first refresh cycle according to first information of the first component, first information of the second component, and first information of the third component.

Specifically, the detailed description of S702 may refer to the above S302, which will not be repeated here.

S703: When the third component is a focus node and the first component and the second component are both non-focus nodes, render the third component through the first process in a first refresh cycle, and render the first component and the second component through the second process.

Specifically, the specific description of S703 can be found in the above S303, which will not be repeated here.

In a specific implementation manner, the electronic device renders the first component and the second component through the second process, which may be:

S7031. The electronic device determines the rendering priority of the first component and the second component according to the second information of the first component and the second information of the second component, and the second information and the weight thereof.

The second information is different from the first information, and the second information is used to distinguish the rendering priority of non-focus nodes. That is, the rendering order of non-focus nodes can be determined by the second information. Therefore, the second information is more inclined to the information of rendering urgency, so the second information can include at least one of the following: a visible window without generated cache, an invisible window without generated cache, a window with generated cache and visible dirty area, a window with generated cache and only invisible dirty area, and a static window with generated cache.

The expression used by the electronic device to determine the priority of the first component and the second component may be:

Among them, ω_i is the weight corresponding to the i-th feature, and f_i is the i-th feature.

Exemplarily, it is assumed that the second information includes a visible window without a generated cache, an invisible window without a generated cache, a window with a generated cache and a visible dirty area, a window with a generated cache and only an invisible dirty area, and a static window with a generated cache. The weights of these information are decreasing in sequence, for example, the weight of a visible window without a generated cache is 5, the weight of an invisible window without a generated cache is 4, the weight of a window with a generated cache and a visible dirty area is 3, the weight of a window with a generated cache and only an invisible dirty area is 2, and the weight of a static window with a generated cache is 1.

Example 1: If the second information of the first component is a visible window without generating a cache, and the second information of the second component is an invisible window without generating a cache, then the priority of the first component is 5, and the priority of the second component is 4.

Example 2: If the second information of the second component is a static window for generating cache, and the second information of the first component is a window for generating cache and having only invisible dirty areas, then the priority of the first component is 2, and the priority of the second component is 1.

S7032: When the rendering priority of the first component is higher than the rendering priority of the second component, render the first component and the second component in sequence through the second process.

Continuing with the above example 1, 5>4, the second process of the electronic device renders the first component first and then the second component.

Using the above example 2, 2>1, the second process of the electronic device renders the first component first and then the second component.

In an embodiment of the present application, when there are multiple non-focus nodes, the electronic device determines the rendering priority of each non-focus node. Thereafter, the electronic device renders the components corresponding to each non-focus node according to the rendering priority of each non-focus node, thereby achieving orderly rendering of the components and alleviating the tension of CPU and GPU resources in high-load scenarios.

In practical applications, the method provided in the embodiment of the present application needs to be described in detail in combination with the following application scenarios. Specifically:

Scenario 1: Rendering of different controls in the same window of the same application.

FIG8 is a schematic diagram of an interface of an electronic device provided by the present application. As shown in FIG8, the electronic device 100 displays an interface 801, and the interface 801 displays common controls 1, video controls 2, and video controls 3. Taking the unified rendering architecture as an example, in combination with the unified rendering architecture shown in FIG4, the interface 801 corresponds to the root control, that is, to the rendering node 0; the common control 1 corresponds to the control 1, that is, to the rendering node 3; Rendering node 1; video control 2 corresponds to control 2, that is, corresponding to rendering node 2; video control 3 corresponds to control 3, that is, corresponding to rendering node 3. In the first refresh cycle, when the user slides the interface 801, the video control 2 of the interface 801 is located in the middle area of the display screen. At this time, the electronic device 100 can detect that: ordinary control 1 is located in the upper area of the display screen, and the image of video control 2 is a single-frame image; video control 2 is located in the middle area of the display screen, and the image of video control 2 is a multi-frame image; video control 3 is located in the lower area of the display screen, and the image of video control 3 is a multi-frame image. At this time, it is assumed that the weight of the control located in the middle area is 1, otherwise it is 0, the weight of the control that is animated is 1, and the weight of the control that is not animated is 0. Then, the electronic device can determine that the score of the common control 1 is 0+0=0, the score of the video control 2 is 1+1=2, and the score of the video control 3 is 0+1=1. 2>1>0 means that the user pays more attention to the video control 2. At this time, the electronic device 100 determines that the video control 2 is the focus node, and the common control 1 and the video control 3 are non-focus nodes. Further, since the common control 1 is a static window that generates a cache, and the video control 3 is a visible window that does not generate a cache, the electronic device 100 can determine that the rendering priority of the video control 3 is higher than the rendering priority of the common node 1. Then, the electronic device 100 assigns the video control 2 to the first process, and the common control 1 and the video control 3 to the second process. The first process of the electronic device 100 renders the video control 2 in real time and displays the rendered image on the screen. The second process of the electronic device 100 renders the video control 3 first, and then renders the common control 1. If the second process does not complete the rendering of the common control 1 and the video control 3 within the first refresh cycle, the electronic device 100 displays the existing texture images of the common control 1 and the video control 3 on the screen. It can be seen that the electronic device can ensure the real-time rendering of the video control 2, so that the animation of the video control 2 is smooth and satisfies the user's experience. Moreover, the rendering of the ordinary control 1 and the video control 3 does not interfere with the video control 2, and if the rendering is not completed, the existing texture image can be used to further ensure the smooth animation of the video control 2 and effectively prevent the video control 2 from freezing.

Scenario 2: Rendering of different windows in different applications.

FIG9 is a schematic diagram of an interface of an electronic device provided by the present application. As shown in FIG9 , the electronic device 100 displays an interface 901, on which window 1, window 2 and window 3 are displayed. Taking the separated rendering architecture as an example, in combination with the separated rendering architecture shown in FIG5 , the interface 901 corresponds to the root control, that is, the root rendering node; window 1 corresponds to control 1, that is, the rendering node 1; window 2 corresponds to container control 3, that is, the rendering node 3; window 3 corresponds to control 2, that is, the rendering node 2. When the mouse is suspended over window 3, the electronic device 100 detects that: the stacking order of window 1 is the bottom layer; the stacking order of window 2 is the middle layer; the stacking order of window 3 is the top layer, and window 3 is the window where the mouse is suspended. At this time, it is assumed that the weight of the stacking order closer to the user is 1, otherwise it is 0. Then, the electronic device can determine that the score of window 1 is 0, the score of window 2 is 0, and the score of window 3 is 1, 1>0, which means that the user is more concerned about window 3. At this time, the electronic device 100 determines that window 3 is the focus node, and window 1 and window 2 are non-focus nodes. Further, since window 1 is a static window for generating cache, and window 2 is a visible window for generating cache, the electronic device 100 can determine that the rendering priority of window 2 is higher than the rendering priority of window 1. Then, the electronic device 100 assigns window 3 to the first process for rendering, and assigns window 1 and window 2 to the second process for rendering. The first process of the electronic device 100 renders window 3 in real time, and displays the rendered image on the screen. The second process of the electronic device 100 renders window 2 first, and then renders window 1. If the second process does not complete the rendering of window 1 and window 2 within the first refresh cycle, the electronic device 100 displays the existing texture images of window 1 and window 2 on the screen. It can be seen that the electronic device can ensure the real-time rendering of window 3, so that window 3 is rendered and displayed in real time as a focus node in a high-load scenario to meet the user's experience. Moreover, window 1 and window 2 can use existing texture images to further ensure the smooth performance of window 3 and effectively prevent window 3 from freezing.

It can be seen that the technical solution provided in the embodiment of the present application can be applicable to the differentiated rendering processing focused on different controls in the same window as shown in Figure 8, and can also be applicable to the differentiated rendering processing focused on different windows as shown in Figure 9. Of course, it can also be applicable to other scenarios, and the embodiment of the present application does not make specific limitations.

Unless otherwise specified or there is no logical conflict, the terms and/or descriptions between the various embodiments of the present application are consistent and can be referenced to each other. The technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and/or software module corresponding to the execution of each function. In combination with the units and algorithm steps of each example described in the embodiment disclosed in this application, the embodiment of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer-driven hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present application.

The present application is an embodiment that can divide the electronic device into functional modules according to the above method example. For example, each functional module can be divided according to each function, or two or more functions can be integrated into one processing unit. The unit can be implemented in the form of hardware or software functional modules. It should be noted that the division of the units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

As shown in FIG10 , it is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, and the electronic device 1000 can be used to implement the methods described in the above method embodiments. Exemplarily, the electronic device 1000 may specifically include: a display unit 1001 and a processing unit 1002 .

The display unit 1001 is used to support the electronic device 1000 to perform step S301 in FIG. 3-2 or steps S301 and S701 in FIG. 7. And/or, the display unit 1001 is also used to support the electronic device 1000 to perform other steps performed by the electronic device in the embodiment of the present application.

The processing unit 1002 is used to support the electronic device 1000 to perform steps S302 to S304 in Figure 3-2, or steps S702 to S703 in Figure 7. And/or, the processing unit 1002 is also used to support the electronic device 1000 to perform other steps performed by the electronic device in the embodiment of the present application.

Optionally, the electronic device 1000 shown in FIG. 10 may further include a communication unit 1003, where the communication unit 1003 is used to support the electronic device 1000 in executing the steps of communication between the electronic device and other devices in the embodiments of the present application.

Optionally, the electronic device 1000 shown in FIG10 may further include a storage unit (not shown in FIG10 ), which stores a program or instruction. When the processing unit 1002 executes the program or instruction, the electronic device 1000 shown in FIG10 may execute the method shown in FIG3-2 , etc.

The technical effects of the electronic device 1000 shown in FIG. 10 can refer to the technical effects of the methods shown in FIG. 3-2, etc., and will not be repeated here. The processing unit 1002 involved in the electronic device 1000 shown in FIG. 10 can be implemented by a processor or a processor-related circuit component, which can be a processor or a processing module. The communication unit can be implemented by a transceiver or a transceiver-related circuit component, which can be a transceiver or a transceiver module. The display unit 1001 can be implemented by display screen-related components.

It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

An embodiment of the present application also provides a computer-readable storage medium, including instructions, which, when executed on a computer, enable the computer to execute any of the above methods.

The embodiment of the present application also provides a computer program product including instructions, which, when executed on a computer, enables the computer to execute any of the above methods.

An embodiment of the present application also provides a chip, which includes a processor and an interface circuit, the interface circuit is coupled to the processor, the processor is used to run a computer program or instruction to implement the above method, and the interface circuit is used to communicate with other modules outside the chip.

All or part of any features or steps in the embodiments of the present application can be freely combined, and the combined technical solutions are also within the scope of the present application.

In the description of this application, unless otherwise specified, "/" means "or", for example, A/B can mean A or B. "And/or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, "at least one" means one or more, and "plurality" means two or more. The words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not limit them to be different.

In the description of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (30)

An image rendering method, applied to an electronic device, is characterized in that the method comprises: Displaying a first interface, wherein the first interface includes a first component and a second component, and the first interface is an interface of a first application; In response to receiving an operation on the first interface, determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component; When the first component is a focus node and the second component is a non-focus node, the first component is rendered by a first process and the second component is rendered by a second process in the first refresh cycle, and the rendering priority of the first process is higher than the rendering priority of the second process. The method according to claim 1, further comprising: When the second process has not completed rendering of the second component within the first refresh cycle, the existing texture image of the second component within the second refresh cycle is reused, and the second refresh cycle is a cycle before the first refresh cycle. The method according to claim 1 or 2, characterized in that it also includes: Displaying a second interface, where the second interface includes a third component, and the second interface is an interface of a second application, which is different from the first application; In response to an operation on the second interface, determining whether the first component, the second component, and the third component are focus nodes in a first refresh cycle according to first information of the first component, first information of the second component, and first information of the third component; When the third component is a focus node and the first component and the second component are both non-focus nodes, the third component is rendered through the first process in the first refresh cycle, and the first component and the second component are rendered through the second process. The method according to claim 3, characterized in that rendering the first component and the second component by the second process comprises: Determining rendering priorities of the first component and the second component according to the second information of the first component and the second information of the second component, and the second information and a weight thereof; When the rendering priority of the first component is higher than the rendering priority of the second component, the first component and the second component are rendered in sequence through the second process. The method according to claim 4 is characterized in that the second information includes at least one of the following: a visible window without generated cache, an invisible window without generated cache, a window with generated cache and visible dirty area, a window with generated cache and only invisible dirty area, and a static window with generated cache. The method according to any one of claims 1-5 is characterized in that the first information includes one or more of transparency, focus window of the window management service, stacking order, whether it is blocked, and whether animation is performed. The method according to claim 6, wherein the first information includes transparency, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component comprises: When the transparency of the first component is greater than the transparency of the second component, it is determined that the first component is a non-focus node and the second component is a focus node. The method according to claim 6, wherein the first information includes a focus window of a window management service, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component comprises: When it is detected that the user operates the first component but does not operate the second component, it is determined that the first component is a focus node and the second component is a non-focus node. The method according to claim 6, wherein the first information includes a focus window of a window management service, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component comprises: When it is detected that the user's sight falls on the first component and does not fall on the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node. The method according to claim 6, characterized in that the first information includes an overlay order, The first information of the first component and the first information of the second component are used to determine whether the first component and the second component are focus nodes in a first refresh cycle, including: When the stacking order of the first component is higher than the stacking order of the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node. The method according to claim 6, wherein the first information includes whether to perform animation, and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component comprises: When the image of the first component is a multi-frame image and the image of the second component is a single-frame image, the first component is determined to be a focus node and the second component is determined to be a non-focus node. The method according to claim 6, wherein the first information includes transparency, a focus window of a window management service, an overlapping order, whether it is blocked, and whether animation is performed; and determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component comprises: Determine whether the first component and the second component are focus nodes based on the following information of the first component and the second component: transparency and its weight, focus window of the window management service and its weight, stacking order and its weight, whether it is blocked and its weight, whether it is animated and its weight. The method according to any one of claims 1-12, characterized in that the component comprises a window or a control. The method according to any one of claims 1-13 is characterized in that the focus node is a node affected by an input event or a node operated by a user. An electronic device, characterized in that it comprises: a display module, a determination module and a rendering module, wherein: The display module is used to display a first interface, the first interface includes a first component and a second component, and the first interface is an interface of a first application; The determination module is used for, in response to receiving an operation on the first interface, determining whether the first component and the second component are focus nodes in a first refresh cycle according to the first information of the first component and the first information of the second component, wherein the focus node is a node that the user focuses on; The rendering module is used to render the first component through a first process and render the second component through a second process within the first refresh cycle when the first component is a focus node and the second component is a non-focus node, and the rendering priority of the first process is higher than the rendering priority of the second process. The electronic device according to claim 15, characterized in that the rendering module is used to: When the second process has not completed rendering of the second component within the first refresh cycle, the existing texture image of the second component within the second refresh cycle is reused, and the second refresh cycle is a cycle before the first refresh cycle. The electronic device according to claim 15 or 16, characterized in that: The display module is used to display a second interface, the second interface includes a third component, the second interface is an interface of a second application, and the second application is different from the first application; The determination module is used to determine whether the first component, the second component and the third component are focus nodes in a first refresh cycle in response to an operation on the second interface according to the first information of the first component, the first information of the second component and the first information of the third component; The rendering module is used to render the third component through the first process within the first refresh cycle and render the first component and the second component through the second process when the third component is a focus node and the first component and the second component are both non-focus nodes. The electronic device according to claim 17, wherein the rendering module is used to: Determining rendering priorities of the first component and the second component according to the second information of the first component and the second information of the second component, and the second information and a weight thereof; When the rendering priority of the first component is higher than the rendering priority of the second component, the first component and the second component are rendered in sequence through the second process. The electronic device according to claim 18 is characterized in that the second information includes at least one of the following: a visible window without generated cache, an invisible window without generated cache, a window with generated cache and visible dirty area, a window with generated cache and only invisible dirty area, and a static window with generated cache. The electronic device according to any one of claims 15-19 is characterized in that the first information includes one or more of transparency, focus window of the window management service, stacking order, whether it is blocked, and whether animation is performed. The electronic device according to claim 20, wherein the first information includes transparency, and the determining module is used to: When the transparency of the first component is greater than the transparency of the second component, it is determined that the first component is a non-focus node and the second component is a focus node. The electronic device according to claim 20, wherein the first information includes a focus window of a window management service, and the determining module is used to: When it is detected that the user operates the first component but does not operate the second component, it is determined that the first component is a focus node and the second component is a non-focus node. The electronic device according to claim 20, wherein the first information includes a focus window of a window management service, and the determining module is used to: When it is detected that the user's sight falls on the first component and does not fall on the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node. The electronic device according to claim 20, wherein the first information includes an overlay order, and the determining module is used to: When the stacking order of the first component is higher than the stacking order of the second component, the first component is determined to be a focus node and the second component is determined to be a non-focus node. The electronic device according to claim 20, wherein the first information includes whether to perform animation, and the determining module is used to: When the image of the first component is a multi-frame image and the image of the second component is a single-frame image, the first component is determined to be a focus node and the second component is determined to be a non-focus node. The electronic device according to claim 20, wherein the first information includes transparency, a focus window of a window management service, an overlapping order, whether it is blocked, and whether animation is performed; and the determining module is used to: Determine whether the first component and the second component are focus nodes based on the following information of the first component and the second component: transparency and its weight, focus window of the window management service and its weight, stacking order and its weight, whether it is blocked and its weight, whether it is animated and its weight. The electronic device according to any one of claims 15-26 is characterized in that the component includes a window or a control. The electronic device according to any one of claims 15-27 is characterized in that the focus node is a node affected by an input event or a node operated by a user. An electronic device, characterized in that the electronic device comprises: one or more processors; and a memory, wherein the memory stores codes; when the codes are executed by the processor, the electronic device executes the method as described in any one of claims 1-14. A computer-readable storage medium, characterized in that it includes computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method as described in any one of claims 1 to 14.