WO2025144352A2

WO2025144352A2 - A method for measuring quality of experience (qoe) in a communication network

Info

Publication number: WO2025144352A2
Application number: PCT/TR2024/051790
Authority: WO
Inventors: Abdurrahman Burak DAYI; Evren TUNA; Huseyin Arslan
Original assignee: Demircioglu Erdem
Current assignee: Demircioglu Erdem
Priority date: 2023-12-27
Filing date: 2024-12-26
Publication date: 2025-07-03
Anticipated expiration: 2026-06-27
Also published as: WO2025144352A3; TR2023018878A2

Abstract

The present invention provides a method which is suitable for being used within a communication network comprising at least one first communication device (1) which is capable of sending and receiving signals, and at least one second communication device (2) which is capable of sending and receiving signals and which is able to communicate with the first communication device (1) by sending signals to it and receiving signals from it. The method is configured for measuring Quality of Experience (QoE) of a communication service between the first communication device (1) and the second communication device (2). Accordingly, a configuration data (3) necessary for generating a reference content which ensures comparison of the requested content and reference content is shared. By the comparison, the QoE measurements can be performed.

Description

DESCRIPTION

A METHOD FOR MEASURING QUALITY OF EXPERIENCE (QoE) IN A COMMUNICATION NETWORK

FIELD OF THE INVENTION

The present invention relates to a method for measuring Quality of Experience (QoE) in a communication network.

BACKGROUND OF THE INVENTION

In telecommunication systems, user applications require services to perform the related operations. Accordingly, the service quality needs to be measured for monitoring the performance of service and applying optimizations with different objectives.

In the prior art, to measure the service quality, Quality of Service (QoS) parameters (these parameters can be exemplified as latency, packet loss rate, block error rate (BLER), bit error rate (BER), jitter, throughput, etc.) are used as the indicators of the network performance. However, the QoS concept deals with the performance measurements with a strictly network-centric approach without regard for the services and content of the packets the network carries, ignoring the user perception of the network performance which may depend on the use case.

Alternatively, to eliminate the disadvantages of the QoS concept, quality of experience (QoE) concept is developed. By using QoE, service quality measurement operations are performed rather a user-centric point of view. QoE regards for the type of the service whose performance to be evaluated, and therefore, may be a reliable indicator for the user satisfaction to the provided service.

QoE measurements can be performed by using subjective or objective methods. Subjective method can be applied by sending out a questionnaire to users and asking them to rate their experience within a scale (1 -5). By the receipt of the answers, the arithmetic mean of the answers is calculated and QoE can be obtained regarding to a service. The main advantage of this is, it directly reflects the user experience, given that there is sufficient data collected from the users. However, it is quite time-consuming, costly and impractical especially for the real time services. Therefore, alternative several objective methods, which are based on mathematical techniques that yield quantitative measures regarding the multimedia content and service quality, have been defined in the literature.

These objective methods can be grouped into two categories: parameter-based methods and reference-based (signal-based) methods. In the parameter-based methods, a QoE metric is calculated based on network or application parameters, such as viewing time, number of times representation of the streamed video has switched, number of times the video stalled, total time during which the video stalled, QoS parameters (which would be then mapped to QoE parameters), etc. On the other hand, in the reference-based objective methods, the signal is analyzed by measuring the difference between the observed signal and a reference signal. By the reference-based objective methods the full content of the reference signal and the observed signal can be compared or some key features of them can be compared. Accordingly, parameter-based methods do not require original content to calculate a QoE score. However, they may fall short, for example, for use cases where audio-video quality is important. While reference-based methods are generally accurate, they cannot be used for measuring QoE of live transmissions as the original content would be unknown at the receiver side. It therefore makes sense to measure QoE with referencebased methods before starting streaming to facilitate efficient resource allocation, network slicing and other QoE-assisted scheduling and network management tasks. This requires a reference sharing mechanism with which the references may be shared among the network entities. When the reference content is sent through wireless resources, the scarce spectral resources are wasted and therefore the spectral efficiency is reduced. In addition to this, the device receiving the reference content may need a large data storage space to store the reference content, due to the variety of services to be supported, which impedes the feasibility of reference-based QoE estimation procedures.

Therefore, all the problems mentioned above have made it necessary to make an innovation in the relevant technical field as a result.

BRIEF DESCRIPTION OF THE INVENTION

The present invention relates to a method for measuring Quality of Experience (QoE) in a communication network by eliminating the above-mentioned disadvantages and bringing new advantages to the relevant technical field. An object of the invention is to develop a method for measuring Quality of Experience (QoE) in a communication network.

Another object of the invention is to develop a method for measuring Quality of Experience (QoE) in a communication network efficiently.

Yet another object of the invention is to develop a method for measuring Quality of Experience (QoE) in a communication network which is objective and user-centric.

The other object of the invention is to develop a method for measuring Quality of Experience (QoE) in a communication network which is reliable.

Another object of the invention is to develop a method for measuring Quality of Experience (QoE) in a communication network usable for all services including live transmissions.

Yet another object of the invention is to develop a method for measuring Quality of Experience (QoE) in a communication network which requires less storage area.

The other object of the invention is to develop a method for measuring Quality of Experience (QoE) in a communication network for ensuring efficient resource usage.

To achieve all the objects mentioned above and that will emerge from the following detailed description, the present invention relates to a method which is suitable for being used within a wireless communication network comprising

• at least one first communication device which is capable of sending and receiving signals,

• at least one second communication device which is capable of sending and receiving signals and which is able to communicate with the first communication device by sending signals to it and receiving signals from it.

The method is configured for measuring Quality of Experience of a communication service between the first communication device and the second communication device and comprises the steps of;

• generating at least one main content related with the communication service;

• generating and sending a configuration data providing the parameters of a reference content related with the main content by the first communication device to the second communication device or by the second communication device to the first communication device;

• generating the reference content according to the received configuration data by the second communication device or first communication device; and

• performing QoE measurement with respect to the generated reference content.

Thanks to the method of the present invention for measuring Quality of Experience in a communication network, measuring can be performed efficiently with a user-centric way. Objective and reliable measurements can be obtained. In addition to these, the method is able to be applied to all services including live transmissions.

DESCRIPTION OF THE DRAWINGS

Exemplary embodiment of the communication network by which the method developed according to the present invention is employed is illustrated in the attached drawings, wherein:

Figure 1 is a schematic view of an embodiment of the communication network by which the method of to the present invention is employed.

Figure 2 is an exemplary schematic view of an uplink application of the method of the present invention.

Figure 3 is an exemplary schematic view of a downlink application of the method of the present invention.

All the parts illustrated in the drawings are individually assigned a reference numeral and the corresponding terms of these numbers are listed as follows:

1 First communication device

2 Second communication device

3 Configuration data

4 First reference content

4.1 Sent first reference content

5 Generated main content

5.1 Second reference content

100 Uplink direction

100a Sending the generated configuration data 100b Sending the generated first reference content

100c Getting information whether the second communication device is capable of generating a first reference content according to a configuration data or not

200 Downlink direction

200a Getting the second reference content by the second communication device 200b Sending the measured QoE to the first communication device

DESCRIPTION OF THE INVENTION

In the wireless communication network, the service quality needs to be measured for monitoring the performance of service and applying optimizations with different objectives. For measuring the service quality, Quality of Service (QoS) measurement concept can be used. However, since QoS is a network-centric approach, this disregards the user-related parameters and accordingly does not give a reliable measurement. Besides QoS, Quality of Experience (QoE) measurement concept can be used. Although QoE is rather usercentric, as explained above, some applications of this are not efficient and some applications of this are not suitable for all types of services such as live transmission services. In order to eliminate these, with the present invention, a method for measuring Quality of Experience (QoE) in a communication network is developed.

The method developed by the invention is suitable for being used within a communication network an exemplary embodiment of which is given in figure 1 and which comprises

• at least one first communication device (1 ) (such as base stations, user devices [mobile phone, loT devices, etc.], core network (CN), etc.) which is capable of sending and receiving signals,

• at least one second communication device (2) (such as user devices [mobile phone, loT devices, etc.]) which is capable of sending and receiving signals and which is able to communicate with the first communication device (1 ) by sending signals to it and receiving signals from it, wherein the method facilitates measuring Quality of Experience (QoE) of a communication service between the first communication device (1 ) and the second communication device (2). The method comprises the steps of,

• generating and sending a configuration data (3) providing the parameters of a reference content related with the main content by the first communication device (1 ) to the second communication device (2) or by the second communication device (2) to the first communication device (1 ) (the configuration data (3) comprises all parameters necessary to generate the reference content such that for instance the requested/generated/reference content is video, the parameters can be resolution, FPS, number of frames, color, bit generator type, bit generator inputs (depending on bit generator type), codec configuration (which video codec to be used e.g., H.264 [5]), scheduling grant on which resources the content to be transmitted, etc.);

• generating the reference content according to the received configuration data (3) by the second communication device (2) or first communication device (1 ); and

• performing QoE measurement with respect to the generated reference content.

In general, by the present invention, a configuration data (3) having parameters necessary for generating a reference content related with a main content of a communication service is shared between devices of a communication network. The communication service can be provided as an uplink application and a downlink application. For the uplink direction (100) (an exemplary embodiment is given in figure 2) the method comprises the steps of; o by the first communication device (1 ), generating the main content and the configuration data (3) including the all parameters required to generate a first reference content (4) related with the main content wherein preferably the parameters also include encoding and decoding parameters of the first reference content (4) (configuration data (3) comprises parameters necessary to generate the first reference content (4) such that for instance the requested/generated/reference content is video, the parameters can be resolution, FPS, number of frames, color, bit generator type, bit generator inputs (depending on bit generator type), codec configuration (which video codec to be used e.g., H.264 [5]), scheduling grant on which resources the content to be transmitted, etc.); o sending the generated configuration data (3) (100a) from the first communication device (1 ) to the second communication device (2); o generating the first reference content (4) by the second communication device (2) with respect to the configuration data (3); o sending the generated first reference content (4) (100b) from the second communication device (2) to the first communication device (1 ) preferably by the radio bearer which is dedicated to the service of the main content; o comparing, at least partly, preferably fully, the generated main content (5) and the sent first reference content (4.1 ) by the first communication device (1 ); and o measuring the QoE of the communication service with respect to the comparison results by the first communication device (1 );

For the downlink direction (200) (an exemplary embodiment is given in figure 3) the method comprises the steps of; o by the first communication device (1 ), generating a second reference content (5.1 ), generating the main content and generating the configuration data (3) including the all parameters required to generate a first reference content (4) related with the main content wherein preferably the parameters also include encoding and decoding parameters of the first reference content (4) (configuration data (3) comprises parameters necessary to generate the first reference content (4) such that for instance the requested/generated/reference content is video, the parameters can be resolution, FPS, number of frames, color, bit generator type, bit generator inputs (depending on bit generator type), codec configuration (which video codec to be used e.g., H.264 [5]), scheduling grant on which resources the content to be transmitted, etc.); o sending the generated configuration data (3) (100a) from the first communication device (1 ) to the second communication device (2); o generating the first reference content (4) by the second communication device (2) with respect to the configuration data (3); o getting the second reference content (5.1 ) by the second communication device (2) (200a), from the first communication device (1 ); o comparing, at least partly, preferably fully, the first reference content (4) and the second reference content (5.1 ) by the second communication device (2); o measuring the QoE of the communication service with respect to the comparison results by the second communication device (2); and o preferably sending the QoE measurement report to the first communication device (1 ) (200b).

In the method of the present invention, for the measurement of QoE of a communication service requested by the second communication device (2), a configuration data (3) necessary to generate a first reference content (4), is generated and sent to the second communication device (2). The first reference content (4) will be used for determining the quality of the generated main content (5) of the requested communication service. By sending the configuration data (3), first communication device (1 ), which is preferably performs operations such as resource allocation by using QoE information, requests from the second communication device (2) generation of the first reference content (4) according to the sent configuration data (3). When second communication device (2) receives the configuration data (3), it generates the first reference content (4) by using the information received within the configuration data (3). If the service request is an uplink application, as shown in figure 2, the generated first reference content (4) is sent from the second communication device (2) to the first communication device (1 ) preferably by the radio bearer which is dedicated to the service of the requested content. At the first communication device (1 ), the generated main content (5) and the sent first reference content (4.1 ) is compared and the QoE of the communication service is measured with respect to the comparison results by the first communication device (1 ). If the service request is a downlink application, a second reference content (5.1 ) is generated by the first communication device (1 ) and sent to the second communication device (2). Then the second communication device (2) compares the generated first reference content (4) and the second reference content (5.1 ) and measures the QoE of the communication service with respect to the comparison results; and preferably sends the measured QoE to the first communication device (1 ). By the method of the present invention, the spectral efficiency can be improved, the need for storing much reference data in limited user device storage is reduced and QoE estimation measurements prior to transmission of the content is facilitated. The requested main content and the reference contents can be in the form of audio, video, file, etc. The QoE estimation measurements is able to be performed based on a configuration created by the network (at the RAN or the CN), at the network entities (base station or user device). This way an arbitrary content may be generated with flexible parameters to align the reference to the actual content to be transmitted/used in the use case without needing to use too much wireless resources, increasing spectral efficiency. Furthermore, by using the same radio bearer, instant QoE of that radio bearer is also able to be measured. Accordingly, optimization of the resources given to that service is able to be performed and therefore, service quality can be optimized. Since the method performed before the transmission of the generated main content (5) is started, the method of the present invention is also applicable to the live services. By using the method, an objective quality measurement is able to be carried out with an observation compared against a reference. For the use cases of future communication technologies (Online gaming, extended Reality (XR), HD tele-conferencing, tele-presence, UHD video streaming, Hi-Fi audio streaming, cloud-media storing and so on) different types of references may be used. For instance, for online gaming use case, a file may be transmitted from the gaming server to the user device, measuring latency, throughput, packet error rate, and some in-game graphics generated by the game engine can be transmitted to evaluate the graphics quality (e.g., FPS, resolution, stalls etc.). On one hand, these measures may be used separately but they also may be used in a joint QoE measurement, e.g., contributing to a single QoE score with defined coefficients. Those skilled in the art can come up with similar examples, pertaining to how a QoE metric can be calculated, for all the potential use cases.

In an exemplary embodiment, if the content of the communication service is a video, the quality of the video can be measured by Peak SNR (PSNR) metric via the below-given equation:

where MAX is the maximum value of a pixel may take (e.g., for a 8-bit pixel MAX=2⁸-1 =255) and MSE is defined as

where (i,j) is the coordinate of a pixel, R(i,j) is the pixel value of reference at (i,j) and V(i,j) is the observed value at (i,j) for a (m x n)-size video. A higher PSNR would indicate a higher display quality. Another objective video quality evaluation metric is structural similarity (SSIM). SSIM is defined as ,

(A AN £ A < A - (K_s X r where x and y are the vectors of reference and observed data, p_x and p_y are the mean values of x and y, respectively and o_x, o_y are the standard deviations of x and y, respectively and o_xy is the covariance of x and y. L is the dynamic range of pixel values. K1 « 1 and K2 « 1 are two scalar constants. In a preferable embodiment of the present invention, the method further comprises getting information whether the second communication device (2) is capable of decoding a message having a configuration data (3) and generating a first reference content (4) according to the configuration data (3) or not (100c), by the first communication device (1 ) from the second communication device (2). This information can be a message informing the communication network that the second communication device (2) is supporting the reference-based configuration transmission. Furthermore, this information is provided by the second communication device (2) to the communication network and gathered by the first communication device (1 ) to obtain the capability information of the second communication device (2) before generating the configuration data (3). If the second communication device (2) is capable of generating a first reference content (4) according to a configuration data (3), then the above-mentioned steps are performed by generating and sending the configuration data (3) from the first communication device (1 ) to the second communication device (2). If the second communication device (2) is not capable of generating a first reference content (4) according to a configuration data (3), then the other methods known in the prior art can be used to obtain QoE.

In another embodiment of the method of the present invention, signals (such as related with the content request, configuration data (3), reference content, requested main content, etc.) received by the first communication device (1 ) and the second communication device (2) can comprise data packets related with the same service or related with the different services (such as VOIP service, online gaming service, video streaming service, etc.). If data packets related with the different services is received, then the data packets are classified according to the services that they are related with, with respect to the Packet Filter Set as defined in the related standards (such as 3GPP TS 23.501 ; System architecture for the 5G System (5GS) (Release 18)). As an example, the data packets are mapped to QoS flows, QoS flow marking is applied to the packets and the packets having the same QFI (QoS Flow Identifier) are classified as a group related with the same service.

In a preferable embodiment, the configuration data (3) further comprises a prompt to an Al- based content generation module, which is in communication with the first communication device (1 ) and/or second communication device (2) and which is capable of generating a reference content. Therefore, content generation is able to be performed by the Al-based module. In another preferable embodiment, all the messaging is carried out on RRC (Radio Resource Control) protocol. In another preferable embodiment, the configuration data (3) further comprises an expiration time. Accordingly, the method further comprises checking whether there is a configuration data (3), before generating the configuration data (3) and if there is a configuration data (3), checking the expiration time has passed or not. For instance, before generating a configuration data (3) related with a content of a communication service, it is checked whether there is a configuration data (3) in the second communication device (2) related with that communication service, or not. If there is a configuration data (3) in the second communication device (2) related with that communication service, then the expiration time of the configuration data (3) is checked. If the expiration time has not passed, then this present configuration data (3) is used to generate first reference content (4). Alternatively, the method further comprises checking whether there is a reference content (such as first reference content (4)), which is generated according to the configuration data (3), before generating the configuration data (3) and if there is a reference content, checking the expiration time has passed or not. For instance, before generating a configuration data (3) related with a content of a communication service, it is checked whether there is a first reference content (4) in the second communication device (2) related with that communication service, or not. If there is a first reference content (4) in the second communication device (2) related with that communication service, then the expiration time of the configuration data (3) is checked. If the expiration time has not passed, then this present first reference content (4) is used to measure QoE of that service. Therefore, the procedure time, and use of resources are able to be reduced.

In a preferable embodiment, the configuration data (3) and resource allocation from which the second communication device (2) is able to be informed about the resources, by which the first reference content (4) to be transmitted, is able to be sent by the same message packets. In another embodiment, the reference content is able to be sent periodically by means of the configuration data (3). Yet another embodiment, the resource allocation is able to be sent by scheduling grant.

Thanks to the method of the present invention for measuring Quality of Experience (QoE) in a communication network, measurements can be performed efficiently with a user-centric way. Objective and reliable measurements can be obtained.

Claims

1. A method which is suitable for being used within a communication network comprising

• at least one first communication device (1 ) which is capable of sending and receiving signals,

• at least one second communication device (2) which is capable of sending and receiving signals and which is able to communicate with the first communication device (1 ) by sending signals to it and receiving signals from it, and which is configured for measuring Quality of Experience (QoE) of a communication service, between the first communication device (1 ) and the second communication device (2) characterised in that the method comprises the steps of;

• generating and sending a configuration data (3) providing the parameters of a reference content related with the main content by the first communication device (1 ) to the second communication device (2) or by the second communication device (2) to the first communication device (1 );

• performing QoE measurement with respect to the generated reference content.

2. The method according to claim 1 wherein; for uplink direction (100), the method comprises the steps of; o by the first communication device (1 ), generating the main content and the configuration data (3) providing the parameters of a first reference content (4) related with the main content; o sending the generated configuration data (3) (100a) from the first communication device (1 ) to the second communication device (2); o generating the first reference content (4) by the second communication device (2) with respect to the configuration data (3); o sending the generated first reference content (4) (100b) from the second communication device (2) to the first communication device (1 ); o comparing, at least partly, the generated main content (5) and the sent first reference content (4.1 ) by the first communication device (1 ); o measuring the QoE of the service with respect to the comparison results by the first communication device (1 ).

3. The method according to claim 1 or claim 2 wherein; for downlink direction (200); o by the first communication device (1 ), generating the main content, generating a second reference content (5.1 ) related with the main content and generating the configuration data (3) providing the parameters of a first reference content (4) related with the main content; o sending the generated configuration data (3) (100a) from the first communication device (1 ) to the second communication device (2); o generating the first reference content (4) by the second communication device (2) with respect to the configuration data (3); o getting the second reference content (5.1 ) by the second communication device (2) (200a), from the first communication device (1 ); o comparing, at least partly, the first reference content (4) and the second reference content (5.1 ) by the second communication device (2); o measuring the QoE of the service with respect to the comparison results by the second communication device (2).

4. The method according to claim 2 or claim 3 wherein; the first reference content (4) is sent from the second communication device (2) to the first communication device (1 ) by the same radio bearer which is dedicated to the communication service whose QoE is to be measured.

5. The method according to claim 3 or claim 4 wherein; following the step of measuring the QoE of the service with respect to the comparison results by the second communication device (2), sending the measured QoE to the first communication device (1 ) (200b).

6. The method according to any one of the claims 2-5 wherein; the method further comprises getting information whether the second communication device (2) is capable of decoding a message having a configuration data (3) and generating a first reference content (4) according to the configuration data (3) or not (100c), by the first communication device (1 ) from the second communication device (2).

7. The method according to any one of the preceding claims wherein; the configuration data (3) further comprises a prompt to an Al-based content generation module, which is in communication with the first communication device (1 ) and/or second communication device (2) and which is capable of generating a reference content.

8. The method according to any one of the preceding claims wherein; the configuration data (3) further comprises an expiration time.

9. The method according to claim 8 wherein; the method further comprises checking whether there is a configuration data (3) before generating the configuration data (3) and if there is a configuration data (3), checking the expiration time has passed or not.

10. The method according to claim 8 wherein; the method further comprises checking whether there is a reference content, which is generated according to the configuration data (3), before generating the configuration data (3) and if there is a reference content checking the expiration time has passed or not.

11. The method according to any one of the preceding claims wherein; the parameters provided by the configuration data (3) also include encoding and decoding parameters of the first reference content (4).

12. The method according to any one of the preceding claims wherein; the configuration data (3) and resource allocation from which the second communication device (2) is able to be informed about the resources, by which the first reference content (4) to be transmitted, is able to be sent by the same message packets.

13. The method according to any one of the preceding claims wherein; the reference content is able to be sent periodically by means of the configuration data (3).

14. The method according to any one of the preceding claims wherein; the resource allocation from which the second communication device (2) is able to be informed about the resources, by which the first reference content (4) to be transmitted, is able to be sent by scheduling grant.