CN111970719A - Method and device for quality evaluation of anchor point station under 5G NSA network - Google Patents

Abstract

The present invention relates to the field of communications, and in particular, to a method and device for quality assessment of anchor stations under a 5G NSA network. It mainly includes: obtaining the quality index data of the anchor site; selecting the evaluation factors that need to be evaluated from the quality index data according to the evaluation dimension; configuring different weights for each evaluation factor according to the evaluation business needs; according to the value and weight of the evaluation factors, The weighted total score of each evaluation factor of the preset evaluation area is calculated as the quality score of the preset evaluation area. The invention can conveniently and accurately obtain the quality status of each anchor station, provide support for 5G NSA network optimization, and improve user perception of 5G service users.

Description

A method and device for quality assessment of anchor station under 5G NSA network

【Technical field】

The present invention relates to the field of communications, and in particular, to a method and device for quality assessment of anchor stations under a 5G NSA network.

【Background technique】

The 5G standard is divided into two networking modes: Non-stand alone (NSA for short) and standalone (SA for short). In the early stage, the NSA architecture was selected, and the method of evolution to the SA architecture was adopted in the later stage. In the NSA architecture, the anchor station is the channel for the 5G base station to access the network, and the continuous coverage performance of the anchor network affects the user's experience of the mobility of the 5G network. Especially for 5G voice, it is implemented through VoLTE and is carried by anchor points. If the continuous coverage performance of the anchor point is poor, the voice experience of 5G users may be inferior to that of VoLTE on the existing network, affecting the user experience.

At present, the commonly used anchor station quality assessment methods include: drive test, single station verification, network management monitoring, and perceptual quality assessment system (Deep Packet Inspection, abbreviated as: DPI) and the like. The road test and fixed-point test can only cover part of the main roads, and the road test cannot be carried out for special scenes or areas such as residential areas, parks, and buildings that are difficult for test vehicles to enter. It is only applied in the initial stage of network construction. The network management monitoring focuses more on the operation quality of network equipment, and provides network service quality from equipment parameter configuration and equipment performance evaluation. Moreover, the network management is mainly oriented to the device side, and cannot fully monitor the user level; on the other hand, the current network management monitoring is mainly used for operation and maintenance indicators. However, as the operator's business gradually shifts from operation and maintenance to operation, operators begin to pay more attention to business perception, and network management monitoring cannot obtain the corresponding indicators of business perception. DPI only covers the core network data, and the service perception evaluation is calculated through the core network data. The results cannot fully reflect or represent the quality of the wireless network and transmission network.

In view of this, it is a problem to be solved in this technical field how to overcome the defects existing in the prior art and solve the phenomenon that the existing anchor site quality assessment cannot easily obtain comprehensive and accurate anchor site quality indicators.

[Content of the invention]

Aiming at the above defects or improvement needs of the prior art, the present invention solves the problems of incomplete coverage and high cost of drive test methods in the current anchor station quality assessment methods, and inability to obtain service perception indicators in network management monitoring methods.

The embodiment of the present invention adopts the following technical solutions:

In a first aspect, the present invention provides a method for evaluating the quality of an anchor station under a 5G NSA network, specifically: obtaining quality index data of the anchor station; selecting an evaluation factor to be evaluated from the quality index data according to the evaluation dimension; Different weights are configured for each evaluation factor according to the evaluation business requirements; according to the value and weight of the evaluation factor, the weighted total score of each evaluation factor in the preset evaluation area is calculated as the quality score of the preset evaluation area.

Preferably, obtaining the quality index data of the anchor station includes: obtaining the quality index data of the anchor station through one or more of a drive test, a fixed-point test, an integrated network management system, MR and/or a perception index evaluation system.

Preferably, the evaluation dimension includes a network service quality dimension and/or a network service perception dimension.

Preferably, different weights are configured for each evaluation factor, including: selecting an evaluation factor to be evaluated according to business needs, setting a weight for each evaluation factor according to business characteristics; selecting an evaluation index to be evaluated for each evaluation factor, according to Business characteristics set weights for each evaluation metric.

Preferably, calculating the weighted total score of each evaluation factor in the preset evaluation area includes: calculating the weighted total score of all evaluation indexes in each evaluation factor according to the value and weight of each evaluation index, and calculating the weighted total score of all evaluation indexes. The score is taken as the value of the evaluation factor; according to the value and weight of each evaluation factor, the weighted total score of all evaluation factors is calculated.

Preferably, calculating the weighted total score of all evaluation factors in the preset evaluation area includes: presetting a reference value and a challenge value for each evaluation index in each evaluation factor; when the value of the first type of evaluation index is less than the reference value , calculate the evaluation index according to the preset lower limit value of the evaluation index; when the value of the first type of evaluation index is greater than or equal to the challenge value, calculate the evaluation index according to the preset upper limit value of the evaluation index; When the value of the second type of evaluation index is greater than the reference value, the evaluation index is calculated according to the preset lower limit of the evaluation index; when the value of the second type of evaluation index is less than or equal to the challenge value, the evaluation index is calculated according to the value of the evaluation index. The preset upper limit value is calculated.

Preferably, when the evaluation dimension is the network service quality dimension, one or more of terminal capability, coverage level, anchor point capacity, interference avoidance and anchor point performance are used as evaluation factors.

Preferably, when the evaluation dimension is the network service perception dimension, one or more of terminals, networks, pipelines, and services are used as evaluation factors.

Preferably, when the terminal is used as the evaluation factor, the terminal is a 5G NSA terminal with the same frequency as the 5G NSA base station in all anchor stations and 5G base stations in the area where the anchor point priority scheme has been implemented and supporting dual connectivity capability.

In another aspect, the present invention provides an apparatus for evaluating the quality of an anchor station under a 5G NSA network, specifically: including at least one processor and a memory, at least one processor and the memory are connected through a data bus, and the memory storage can be At least one instruction is executed by the processor, and after the instruction is executed by the processor, the instruction is used to complete the method for evaluating the quality of the anchor station under the 5G NSA network in the first aspect.

Compared with the prior art, the beneficial effect of the embodiment of the present invention is that data of multiple dimensions are obtained as evaluation factors, corresponding weights are set for evaluation factors according to different services, and the weighted total score of evaluation factors of multiple dimensions is used as an anchor point. Criteria for site quality assessment. Through this quality evaluation method, the quality status of each anchor station can be easily and accurately obtained, which provides support for 5G NSA network optimization and improves the user perception of 5G service users.

【Description of drawings】

In order to describe the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings that need to be used in the embodiments of the present invention. Obviously, the drawings described below are only some embodiments of the present invention, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a flowchart of a method for evaluating the quality of an anchor station under a 5G NSA network according to an embodiment of the present invention;

FIG. 2 is a flowchart of another method for evaluating the quality of an anchor station under a 5G NSA network provided by an embodiment of the present invention;

3 is a flowchart of another method for evaluating the quality of an anchor station under a 5G NSA network provided by an embodiment of the present invention;

4 is an evaluation factor model used in a method for evaluating the quality of an anchor station under a 5G NSA network provided by an embodiment of the present invention;

FIG. 5 is another evaluation factor model used in a method for evaluating the quality of an anchor station under a 5G NSA network provided by an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of an apparatus for evaluating the quality of an anchor station under a 5G NSA network according to an embodiment of the present invention.

【Detailed ways】

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

The present invention is an architecture of a specific functional system. Therefore, the functional logic relationship of each structural module is mainly described in the specific embodiments, and the specific software and hardware implementations are not limited.

In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other. The present invention will be described in detail below with reference to the accompanying drawings and embodiments.

The Option3 family has great advantages in rapid 5G deployment, and the commercial driving force for upgrading LTE to eLTE to connect to NGC needs to be further clarified. In the Option3 family, the introduction of NR in Option3x has the least impact on the existing LTE network and has the best performance. 5GNR is a global 5G standard based on a new air interface design based on OFDM. It is also a very important cellular mobile technology foundation for the next generation. 5G technology will achieve ultra-low latency and high reliability. Option3x, as the common choice of NSA, and Option2, as the common choice of SA, have become the first choice of most operators.

On the one hand, the 5G NSA meets the construction requirements of the first batch of 5G networks, and on the other hand saves the time for deploying the NGC core network, virtualization, and cloud transformation. Option3x LTE eNB and NR gNB adopt the form of dual connection, which significantly improves the downlink peak rate of users. NSA Option3x can formulate flexible offloading strategies according to 4G network load, which can make full use of network resources while avoiding congestion of existing 4G networks. Under the NSA network, users are carried on the LTE network, and the mobility management reuses the existing mobility management of LTE. When the UE moves out of the NR coverage, the data plane is not interrupted and the LTE connection is maintained. Option3x needs to rely on the X2/Xn interface to interconnect with LTE, and network upgrade and transformation requires corresponding LTE capacity matching and version matching, including product implementation that conforms to the latest standard technical specifications.

The main difference between NSA and SA networking is that: NSA anchors 5G control signaling on 4G base stations, and the anchored base stations are called anchor stations; SA scheme is that 5G base stations directly access 5G core network, control signaling Makes it completely independent of the 4G network. Under the NSA standard, there is no need to build a new core network. The new 5G base station is connected to the 4G core network or the 5G core network through the 4G base station. 5G and 4G do not need to be interoperable, and can initiate services on 4G and 5G networks at the same time. 5G terminals need to access 4G networks at the same time, and need to support 4G/5G dual links.

In the NSA solution, users simultaneously access 5G and 4G base stations through dual connections, NAS signaling is transmitted by the anchor station, user data streams are split between 5G base stations and 4G base stations, and 5G services are enjoyed through collaboration between 5G and 4G base stations. . The terms used therein are explained as follows:

(1) Dual connection: The mobile terminal can communicate with both 4G and 5G at the same time, and can download data at the same time. Typically, there will be one master connection and at least one slave connection.

(2) Control plane anchor: The control plane in the dual connection is called the control plane anchor.

(3) Diversion control point: The user's data needs to be divided into two paths of dual connections for independent transmission, and the location where the diversion is performed is called the diversion control point.

Example 1:

In the NSA networking scenario, limited by the frequency range that the NSA terminal can support, or other special factors, it is difficult to use all the LTE base stations on the existing network as NSA anchor stations. Once the NSA user accesses the non-anchor LTE base station, it will result in the inability to add an NR secondary carrier and thus cannot enjoy 5G services. The core network connected to the Option 3X base station is the 4G core network, the control plane anchor is at the 4G base station, and the data offload control point is placed on the 5G base station. Since the anchor station is used as the signaling channel for the NSA terminal to access the network, the 4G frequency point with good continuous network coverage performance should be selected as the anchor point. In order to select a suitable NSA anchor point, it is necessary to select the 4G frequency point with good continuous coverage performance and high edge rate as the anchor point based on the end-to-end support of the anchor point industry chain and the estimated 4G network coverage. The method for evaluating the quality of an anchor station under a 5G NSA network provided in the embodiment of the present invention obtains the quality status of the anchor station by evaluating the quality of the anchor station in different dimensions, so as to provide data support for the optimization of the anchor station.

As shown in FIG. 1 , the specific steps of the method for evaluating the quality of the anchor station under the 5G NSA network provided by the embodiment of the present invention are as follows:

Step 101: Acquire anchor site quality index data.

Traditional quality assessment methods generally only use a single quality assessment method. The main quality assessment methods used include drive test, single-site verification, network management monitoring, and DPI. The road test can intuitively reflect the communication quality in different areas, but it can only cover part of the main roads. The road test cannot be implemented on ordinary streets or roads that are difficult for test vehicles to enter, such as residential areas, parks, buildings, and special scenes. , it is generally only used to optimize the location of the base station in the initial stage of network construction, and the cost of manpower and vehicles required for road testing is relatively high. The network management focuses more on the operation quality of network equipment, and evaluates the service quality provided by the network from equipment parameter configuration and equipment performance. The evaluation focuses on KPI indicators, which are mainly used in equipment operation and maintenance. However, as operators gradually shift from operation and maintenance to operation, operators begin to pay more attention to service perception. The network management evaluation method lacks KQI index evaluation and cannot reflect user service perception. In addition, the network management is mainly oriented to the device side, and the user's perception cannot be fully monitored. A Measurement Report (abbreviated as: MR) means that information is sent every 480ms on a traffic channel (470ms on a signaling channel), and these data can be used for network evaluation and optimization. DPI evaluation method uses DPI data to collect data for business perception evaluation. It is mainly to set different weights and index weights according to the business type to comprehensively evaluate the key indicators in the user's Internet access to browse web pages, watch videos, send/receive instant messages, games, etc. Service awareness, DPI has advantages in signaling plane and service plane data. The data includes user information, terminal type, area, network element, base station, access service type, etc. Therefore, service perception evaluation can be performed according to different dimensions, but DPI only covers core network data, and service perception evaluation is calculated through core network data. The results cannot fully reflect or represent the quality of wireless networks and transmission networks, and cannot End-to-end business-aware evaluation system.

In the anchor site quality evaluation method provided in this embodiment, according to the evaluation dimensions and business requirements, various anchor site quality index data are obtained to perform quality evaluation. In a specific embodiment of the present invention, according to different quality index data to be used, the method of obtaining the quality index data of the anchor station includes: one or more of drive test, single-site verification, integrated network management system, MR and DPI .

The quality index data obtained by the drive test and single-site verification methods include: (1) Basic coverage indicators: SS-RSRP, SINR, CSI-RSRP, SINR, CQI, coverage, CQI distribution statistics, beam-level statistical measurements, etc. (2) Throughput indicators: uplink and downlink average rate, uplink and downlink peak rate, PHY/MAC/RLC/PDCP layer rate distribution, etc. (3) Scheduling capability indicators: MCS statistics, RANK statistics, Blert statistics, MIMO statistics, PDCCH statistics, etc.

The integrated network management system includes wireless network management, transmission network, core network management and other parts. The obtained quality index data includes: (1) Basic network index: uplink interference (average uplink interference level of cell RB, maximum uplink interference level of cell RB) , transmission quality (transmission delay, transmission packet loss rate), channel quality (the number of RBs used for uplink MCS levels 0-28, and the number of RBs used for downlink MCS levels 0-28), CQI, etc. (2) Network performance indicators: access type (SN addition success rate, SN addition average/maximum duration), hold type (SN abnormal release rate), mobility (with SN leg handover success rate), complete type (RLC layer lost packet rate, MAC layer block error rate) and so on. (3) Network traffic indicators: resource type (the number of PRBs occupied by the uplink traffic channel, the number of PRBs occupied by the downlink traffic channel), traffic type (the number of bytes of uplink PDCP PDUs on the user plane of the cell, and the number of bytes of downlink PDCP PDUs on the user plane of the cell), Number of users (average number of terminal RRC connections, maximum number of terminal RRC connections), etc.

The quality index data obtained in the MR mode includes the terminal and the eNodeB. The terminal performs and reports the downlink level strength, quality and TA of the eNodeB cell, and the eNodeB performs and reports the measurement of the uplink terminal's reception level strength and quality.

The quality index data obtained by the DPI method includes: (1) Terminal: The core network obtains terminal access position, terminal capability, terminal user quantity, terminal traffic, terminal signaling index, terminal service index, etc. (2) Access: NSA terminal access anchor station attachment success rate, attachment delay, etc. (3) Dwelling: 5G service usage time dwell ratio, 5G service traffic dwell ratio, etc. (4) Handover: directional handover, anchor station handover, etc. (5) Load: anchor station load, non-anchor station load, etc. (5) Auxiliary station: success rate of dual connection establishment of NR station, etc. (6) Traffic: anchor station traffic, NR station traffic, etc. (7) Perception: business excellence rate, etc.

Step 102: According to the evaluation dimension, select evaluation factor data to be evaluated from the quality index data.

Due to the advantages and disadvantages of several existing quality assessment methods, in order to better and more comprehensively assess the quality of anchor stations, it is necessary to use the quality index data obtained in various ways to jointly establish a set of wireless network, transmission network and core network. An end-to-end perception evaluation system is formed within the system. In the specific implementation process, multiple sets of evaluation indicators can be selected to be combined into multiple evaluation factors, and different aspects of business perception can be evaluated, and each evaluation factor includes one or more evaluation indicators.

In the specific implementation scenario of this embodiment, the evaluation dimensions generally include a network service quality dimension and a network service perception dimension. In actual use, one of the dimensions can be selected for evaluation according to factors such as business characteristics and the data that can be obtained, or two evaluation dimensions can be used for comprehensive perception evaluation to obtain more comprehensive perception data.

When taking network service quality as the evaluation dimension, use one or more of coverage level, anchor capacity, interference avoidance and anchor performance as evaluation factors. Since the network service quality is generally positively correlated with the service quality of the anchor station perceived by the end user, the quality perceived by the end user can be indirectly evaluated by the quality of the network service. In actual use, the evaluation index data may specifically include: obtained from the integrated network management system: coverage rate, uplink PRB utilization rate, downlink PRB utilization rate, RRC reconstruction success rate, call drop rate, etc.; obtained from DPI: the same Frequency, anchor station attachment success rate, handover success rate, etc.

When taking network service perception as the evaluation dimension, use one or more of terminals, networks, pipelines, and services as evaluation factors. The network service perception dimension can directly obtain the user's perception of the service quality of the anchor station from the three aspects of occupation, residence, and experience. In actual use, the evaluation index data can specifically include: Terminal: the number of 5G NSA terminals under the anchor station, terminal function switch status, etc. attached to the anchor station, etc.; Network quality, etc.; Pipeline: service pipeline TCP quality analysis, etc.; Service: high-bandwidth video service quality analysis, etc.

In the specific implementation scenario of this embodiment, only the 5G NSA terminal that has the same frequency as the 5G NSA base station and supports dual-connectivity can access the 5G NSA network, and only the data generated by this type of terminal can be evaluated. value. Therefore, when evaluating a terminal as an evaluation factor, only data from that type of terminal is used for evaluation.

Further, when using the terminal as an evaluation factor, the terminal is a 5G NSA terminal with the same frequency as the anchor station and supporting dual connectivity. NSA terminals must occupy the anchor cell before using 5G services to improve user perception. Therefore, it is an important issue to be solved that NSA terminals need to be migrated to the anchor cell and ensure stable occupation.

4 problems faced by anchor priority occupation;

1. If the terminal occupies the non-anchor point, how to migrate from the non-anchor point to the anchor point in the connected state;

2. If the terminal occupies the non-anchor point, how to migrate from the non-anchor point to the anchor point in the idle state;

3. If the terminal has occupied the anchor point, how to stably occupy the anchor point in the connected state to prevent premature switching to the non-anchor point;

4. If the terminal has occupied the anchor point, how to stably occupy the anchor point in the idle state to prevent premature re-selection to the non-anchor point. In the implementation scenario of this embodiment, the following anchor station priority schemes are generally used, mainly including: directional handover + directional reselection + NSA independent mobility policy + prohibition of NSA load balancing.

(1) Directional switching:

1. When the NSA terminal accesses in a non-anchor cell, it initiates a directional handover to the anchor cell.

2. When the NSA terminal is handed over in a non-anchor cell, it initiates a directional handover to the anchor cell.

3. When the NSA terminal performs the VoLTE service at the non-anchor point, it is forbidden to initiate a directional handover.

4. After the NSA terminal ends the VoLTE service at the non-anchor point, it initiates a directional handover to the anchor point cell.

(2) Directional reselection:

When the NSA terminal is released in the non-anchor cell, the rrcConnectionRelease message carries the anchor frequency and sets it as the highest priority, which can guide the terminal to the anchor frequency.

When the NSA terminal is released in the anchor cell, the rrcConnectionRelease message carries the anchor frequency and sets it as the highest priority, which can guide the terminal to the anchor frequency.

(3) Mobility strategy:

The NSA terminal connection strategy is that the non-anchor cell is easier to switch to the anchor cell, the anchor cell is more difficult to switch to the non-anchor cell, and the anchor cell ensures smooth handover.

(4) Prohibit NSA terminal load balancing configuration:

When the NSA terminal occupies the anchor cell, it is prohibited to load balance the NSA terminal to other frequencies.

Step 103: According to the evaluation business requirements, configure different weights for each evaluation factor data.

In the anchor site quality evaluation method provided in this embodiment, multiple quality index data are used to evaluate the quality. According to different device scenarios, business scenarios and evaluation dimensions, different quality index data have different influences on the service quality perceived by the terminal. Therefore, during quality evaluation, it is necessary to configure different weights for the evaluation factors in the quality index data. .

Further, since each evaluation factor may include more than one evaluation index, and each evaluation index has a different degree of influence on the evaluation factor. Therefore, in specific usage scenarios, it is not only possible to set weights for different evaluation factors to improve the accuracy of quality evaluation, but also to set different weights for the evaluation indicators in each evaluation factor to further distinguish the impact of different evaluation indicators on quality perception. influence level. Specifically, as shown in Figure 2, in actual use, different weights can be configured for each evaluation factor through the following steps:

Step 201: Select evaluation factors to be evaluated according to the evaluation dimensions, and set weights for each evaluation factor according to business characteristics.

Step 202: Select an evaluation index to be evaluated for each evaluation factor, and set a weight for each evaluation index according to business characteristics.

Through the bipolar weight distribution in step 201 and step 202, the influence degree of different evaluation indicators and evaluation factors on business perception can be more accurately distinguished, and the accuracy of evaluation can be improved. In the specific implementation scenario of this embodiment, the number of levels of weight allocation is not limited to two levels, and can be adjusted according to actual needs. When there are few evaluation indicators, only one level can be used, and when there are many evaluation indicators and evaluation factors, it can be further The evaluation indicators are grouped into evaluation sub-factors and other more levels for weight setting to further improve the evaluation accuracy.

Step 104: Calculate the weighted total score of each evaluation factor data of the preset evaluation area according to the value and weight of the evaluation factor data, as the quality score of the preset evaluation area.

After setting the weight for each evaluation factor, the weighted sum of the obtained values of each evaluation factor can be calculated as the quality score of the preset evaluation area, and the quality of the anchor station is evaluated according to the quality score.

Further, in some specific usage scenarios of this embodiment, weights are also set for the evaluation indicators in each evaluation factor. When calculating the value of the evaluation factor, it is necessary to calculate the weighted total score of all evaluation indicators in each evaluation factor. , as the value of the evaluation factor.

In some specific implementation scenarios of this embodiment, the values of some evaluation factors or evaluation indicators may have abnormal values that are much lower than or higher than normal values due to factors such as measurement limits or errors. In order to avoid excessive deviation between the value of the abnormal value and the normal value and affect the calculation result of the quality score, the lower limit of the normal value can be set as the reference value, and the upper limit of the normal value can be set as the challenge value. According to the nature of different evaluation indicators, the evaluation indicators are divided into two categories: (1) The first type of evaluation indicators: such as success rate, download rate indicators, etc., the values lower than the reference value are calculated according to the preset lower limit, and the Values higher than the challenge value are calculated according to the preset upper limit; (2) The second type of evaluation indicators: such as delay, retransmission rate, fragmentation rate indicators, etc., the values higher than the reference value are calculated according to the preset lower limit Value calculation, the value lower than the challenge value will be calculated according to the preset upper limit value. Through the above processing, the influence of the specific value of the outlier on the quality score is reduced. As shown in Figure 3, the calculation can be performed by the following steps.

Step 301: Preset a reference value and a challenge value for each evaluation index data in each evaluation factor.

Step 302: Determine the type of the evaluation index as the first type of evaluation index or the second type of evaluation index.

Step 303: When the value of the first type of evaluation index is less than the reference value, calculate the evaluation index according to the preset lower limit of the evaluation index.

Step 304: When the value of the first type of evaluation index is greater than or equal to the challenge value, calculate the evaluation index according to the preset upper limit value of the evaluation index.

Step 305: When the value of the second type of evaluation index is greater than the reference value, calculate the evaluation index according to the preset lower limit of the evaluation index.

Step 306: When the value of the second type of evaluation index is less than or equal to the challenge value, calculate the evaluation index according to the preset upper limit value of the evaluation index.

Through steps 301 to 306, according to the characteristics of different evaluation indicators, the influence of abnormal values on the perceptual evaluation results is eliminated through the reference value and the challenge value, and the evaluation result deviation caused by the absolute value of the evaluation index being too large or too small is avoided.

After going through steps 101 to 104 provided in this embodiment, the quality index of the anchor site can be evaluated by using quality index data of multiple evaluation dimensions. It avoids the difficulties of implementing various quality evaluation methods, difficult data acquisition, and one-sided evaluation results. It can comprehensively and accurately evaluate the perceived quality of anchor stations, which can be used as the basis for 5G NSA anchor station selection, or other service quality. Evaluation basis.

Example 2:

In some specific implementation manners, the method for evaluating the quality of the anchor station under the 5G NSA network provided in Embodiment 1 can be implemented through the evaluation system model provided in this embodiment.

The evaluation system model used in this embodiment mainly uses evaluation factors such as terminals, coverage, capacity, interference, and performance to perform perceptual evaluation of the quality of the anchor station on the dimension of network service quality.

According to step 102, the evaluation factors for evaluation are selected, and the evaluation principle of each evaluation factor is as follows:

Terminal Capability: Obtained by DPI. NSA terminals do not necessarily support all LTE frequency points on the existing network. Therefore, when planning the NSA anchor point, it is necessary to select the mainstream NSA terminal capability information.

Coverage level: obtained through wireless network management data and network management system. 1) NSA anchors must have continuous coverage, otherwise NR cannot be added in areas without anchor coverage. 2) The NSA anchors supported by commercial terminals are limited. For the anchors supported by mainstream terminals, even if the current coverage is poor, they must be used as high-priority anchors, and the capacity needs to be expanded when necessary.

Anchor Capacity: Obtained by DPI. Including upstream capacity and downstream capacity. 1) Uplink capacity: It is recommended to give priority to the uplink capacity of the anchor point, because 5G uplinks are easily limited and there is a greater demand for uplink offloading. 2) If the downlink distribution needs to be enabled, the frequency point with large downlink capacity needs to be preferentially selected as the high-priority anchor point. If DL CA is activated on the existing LTE network, the combined capability of the terminal NSA DC and LTE CA needs to be considered. However, there is no specific terminal capability information at this stage, so this factor is not considered for the time being. If DL CA is not activated on the existing LTE network, the LTE frequency point with higher downlink capacity of a single frequency point needs to be preferentially selected as the NSA high-priority anchor point.

Interference avoidance: obtained through wireless network management data. Harmonic/intermodulation interference needs to be avoided. There is a risk of harmonic/intermodulation interference in certain LTE+NR band combinations, such as (LTE1.8G+NR3.5G) (LTE2.6G+NR4.9G). It is recommended to de-prioritize LTE frequencies that are at risk of interference when planning NSA anchors. Whether there is an interference risk is subject to the calculation result of the actual formula.

Coupling of mobility strategy: obtained through network management data. 1) The LTE non-essential mobility policy of SRAN 15.0 is not decoupled from the NSA. Common mobility policies include CA PCC anchor point policy/MLB. If the tendency frequency of the non-essential mobility policy is not configured as NSA anchor point, there is NR Risk cannot be added. Therefore, it is recommended to use the tendency frequency points of the non-essential mobility strategy of the LTE network as the NSA anchor point, such as the target frequency point of MLB. 2) For the necessary handover strategy of LTE, NSA users are required to comply, otherwise there is a risk of call drop.

Anchor basic performance: obtained through the network management system. If the basic performance (including access, handover success rate, call drop rate, RRC reconstruction ratio, ping-pong handover times) of each frequency point in the current LTE network is quite different, and the frequency points with poor basic performance are difficult to optimize and improve. Then, it is necessary to select the frequency point with good basic performance as the NSA high-priority anchor point.

The evaluation indicators used in each evaluation factor include:

1. Same frequency: the proportion of terminals that support the same frequency as that of the base station cell for all terminal types in the cell within a certain period of time.

2. Coverage: Coverage is defined as the percentage of test points with F value of 1 in all test points in the test area. Among them: RSRP represents the received power of the downlink pilot signal; SINR represents the signal quality of the received pilot signal; RSRP≥R and RSRQ≥S indicate whether the conditions are met, and R and S are the thresholds of RSRP and RSRQ in the calculation. If both RSRP≥R and RSRQ≥S are satisfied, F takes the value of 1, and if one of them is not satisfied or neither is satisfied, then F takes the value of 0.

3. Uplink PRB utilization and downlink PRB utilization:

TTI PUSCH PRB utilization = number of PUSCH PRBs used per TTI/total number of PUSCH PRBs per TTI.

TTI PDSCH PRB utilization = number of PDSCH PRBs used per TTI/total number of PDSCH PRBs per TTI.

4. Interference rate: the sampling ratio of MR RIP>=-105dBm.

5. The anchor station attachment success rate: the number of successful anchor station attachments/the number of anchor station attachment requests × 100%.

6. Handover success rate (X2 cut in, X2 cut out, S1 cut in, S1 cut out):

X2 hand-in success rate=X2 hand-in success times/X2 hand-in attempts times×100%.

X2 cut-out success rate=X2 cut-out success times/X2 cut-out attempts×100%.

S1 hand-in success rate=S1 hand-in success times/S1 hand-in attempts times×100%.

S1 cut-out success rate=S1 cut-out success times/S1 cut-out attempts×100%.

7. Drop rate: E-RAB drop rate = (the number of E-RABs requested by eNodeB to release due to abnormal reasons + the number of E-RABs included in the UE context requested by eNodeB to release due to abnormal reasons) / the number of successful E-RAB establishments × 100%.

8. RRC reconstruction success rate: RRC reconstruction success rate=the number of successful RRC reconstruction times/the number of times of RRC reconstruction attempts (service related)×100%.

According to step 103, as shown in FIG. 4, weights are set for the evaluation factors and evaluation indicators. The weights of each evaluation factor and the evaluation indicators are shown in Table 1. The weights, benchmark values and challenge values can be adjusted according to actual experience.

Table 1

According to step 104, the weighted total score of the evaluation factors is calculated as the quality score of the evaluation area.

In order to obtain the quality score more accurately, instead of directly using the value of the evaluation index, the following formula can be used to calculate the score of each evaluation index.

Use Equation 1 to calculate the same frequency, coverage, and success rate indicator scores:

Use Equation 2 to calculate the utilization, interference rate, and drop rate indicator scores:

Through the above formula, the value of the evaluation index is calculated, and the weight of the evaluation index and the evaluation factor is calculated to obtain the quality score of the anchor station in the evaluation area.

Example 3:

In some specific implementation manners, the method for evaluating the quality of the anchor station under the 5G NSA network provided in Embodiment 1 can be implemented by using another evaluation system model provided in this embodiment.

The evaluation system model used in this embodiment mainly starts from the three aspects of occupancy, residence, and experience, and uses four evaluation factors of terminal, network, pipeline, and service to perform perception evaluation, and anchors the network service quality dimension and network service perception dimension. Comprehensive perceptual assessment of station quality.

According to step 102, the evaluation factors for evaluation are selected, and the evaluation principle of each evaluation factor is as follows:

Terminal: The number of 5G NSA terminals under the anchor station, the status of terminal function switch, etc. attached to the anchor station.

Network: Analyze network quality based on attachment, dual connectivity, cell user volume, and performance.

Pipeline: Based on the analysis of service pipeline TCP quality and so on.

Service: Based on the analysis of high-bandwidth video service quality, etc.

The evaluation indicators and indicators used in each evaluation factor are calculated as follows:

Number of 5G terminals: obtained by DPI. 4G and 5G signaling counts the number of all 5G terminal users on the network.

The number of 5G networks attached to 5G terminals: obtained through DPI. The number of 5G end users attached to the 5G network.

The proportion of 5G terminals attached to the 5G network: obtained through DPI. [Number of 5G terminals attached to 5G networks]/[Number of 5G terminals].

The total number of 5G terminals reporting the 5G switch status: obtained through DPI. Number of reported terminals with dual connectivity capabilities.

Number of 5G terminals turned on by the 5G switch: Obtained by DPI. The number of terminals that are switched on among the reported dual-connectivity terminals.

5G terminal 5G switch on ratio: [Number of 5G terminals with 5G switch on]/[Total number of 5G terminals reporting 5G switch status].

Anchor station 5G terminal attachment success rate: obtained through DPI. [ATTACH success times]/[ATTACH request times], the statistical range is: 1. 5G terminal, based on 5G terminal library identification; 2. 4G anchor station, based on the anchor station configuration table.

ERAB Modification Indicates Success Rate: Obtained by DPI. [E-RAB modification success times]/[E-RAB modification request times], E-RAB modification indication success times: count the number of E-RAB MODIFICATION CONFIRM messages; E-RAB modification indication request times: count E-RAB MODIFICATION INDICATION messages cumulative number of times.

SgNB addition success rate: obtained through the integrated network management system. [Number of successful SgNB additions]/[Number of SgNB addition requests].

Anchor RRC connection establishment success rate: obtained through the integrated network management system. [RRC connection establishment successful times]/[RRC connection establishment request times].

Regional 5G access times and dwell ratio: obtained by DPI. Calculated in the 5G coverage area, the resident ratio of the number of 5G visits = [the number of 5G terminal accesses in the 5G network service / [the number of 5G terminal access to the 5G network service + the number of 5G terminal access to the 4G network service]; the statistical scope of the 5G coverage area is All anchor stations and 5G stations in the area where the anchor priority scheme has been implemented.

Regional 5G traffic residency ratio: obtained by DPI. Calculated in the 5G coverage area, the 5G traffic residency ratio = [5G terminal traffic on 5G network]/[5G terminal traffic on 5G network + 5G terminal traffic on 4G network]; the statistical scope of 5G coverage area is the priority of implementing anchor points All anchor stations and 5G stations within the area of the scheme.

Regional 5G Dwell Ratio: Obtained by DPI. Calculated in the 5G coverage area, the 5G traffic dwell ratio = [5G terminal in 5G network service time]/[5G terminal in 5G network service time + 5G terminal in 4G network service time]; 5G coverage area statistical scope is the implementation of over-anchor All anchor stations and 5G stations within the area of the point priority scheme.

SN abnormal release rate (drop rate): obtained through the integrated network management system. 1-[SN normal release request times]/[SN release request times].

SN handover success rate: obtained through the integrated network management system. [SN switching success times]/[SN switching request times].

Anchor point wireless drop rate: obtained through the integrated network management system. [Number of abnormal eNodeB requests to release]/[Number of eNodeB requests to release].

Video download rate: obtained by DPI. Streaming media package download volume/streaming media download duration; streaming media package download volume: the data download volume between the SP sending the "HTTP Reply 200OK" message to the terminal and the SP sending the "finish TCP data ack" message to the terminal; the streaming media download duration : the time from when the SP sends the "HTTP Reply 200OK" message to the terminal until the SP sends the "finish TCP data ack" message to the terminal.

Rate bit rate ratio: obtained by DPI. Effective video download rate/code rate; effective download rate=streaming media application layer download amount of the Streaming valid document/(total duration of streaming media download - duration of data transmission without data packets).

Initial buffering success rate: obtained by DPI. Initial buffering success times/initial buffering request times*100.

Initial buffer delay: obtained by DPI. Total initial buffering delay/number of successful initial buffering.

Video freezing rate: obtained by DPI. Number of freezes/number of successful initial buffering*100.

Video card time ratio: obtained by DPI. Sum of buffering duration during playback/Number of successful initial buffering*100.

Return upstream effective rate: obtained through DPI. [Push Streaming Load]/[Streaming Media Download Duration].

Streaming downlink rate (kbps): Obtained by DPI. [Download Downstream Traffic]/[Downstream Transmission Duration].

TCP12 handshake success rate: obtained through DPI. TCP SYN ACK times/TCP SYN times*100%.

TCP 23 handshake success rate: obtained through DPI. TCP ACK times/TCP SYN ACK times*100%.

TCP 12 handshake delay: obtained through DPI. TCP SYN ACK time-sum of TCP SYN time/TCP SYNACK times,

TCP 23 handshake delay: obtained through DPI. TCP ACK time-sum of TCP SYN ACK time/TCP ACK times.

TCP uplink retransmission rate: obtained through DPI. The number of TCP upstream retransmission packets / the number of upstream packets * 100%.

TCP downstream retransmission rate: obtained through DPI. The number of TCP downstream retransmission packets / the number of downstream packets * 100%.

TCP uplink out-of-order rate: obtained by DPI. The number of TCP upstream out-of-order packets/the number of upstream packets*100%.

TCP downlink out-of-order rate: obtained through DPI. TC;P Downstream out-of-order packets/downstream packets*100%.

Upstream PRB average utilization: obtained through the integrated network management system. Average utilization rate of the uplink PRB of the cell in the statistical period.

Average utilization rate of downlink PRB: obtained through the integrated network management system. Average utilization rate of downlink PRBs of the cell in the statistical period.

Upstream interference noise (average): obtained through the integrated network management system. Upstream average interference level.

Average number of users (RRC connection): obtained through the integrated network management system. Average number of RRC connection users in the statistical period.

Anchor paging congestion rate: obtained through the integrated network management system. Number of paging discards/number of paging requests.

Anchor point coverage: obtained through an integrated MR system. Coverage is defined as the percentage of test points with F value of 1 in all test points in the test area.

According to step 103, as shown in FIG. 5, weights are set for the evaluation factors and evaluation indicators. The weights of each evaluation factor and the evaluation indicators are shown in Table 2. The weights, benchmark values and challenge values can be adjusted according to actual experience. Since there are many evaluation index values in the model provided in this example, when setting the weight, not only two levels of evaluation factors and evaluation indexes provided in Example 1 and Example 2 are set, but also some evaluation factors are added. As for the level of the evaluation sub-factors, in the specific use of the anchor station quality evaluation method of the present invention, an appropriate evaluation level can be selected according to actual needs, and the evaluation indicators can be organized.

Table 2

According to step 104, the weighted total score of the evaluation factors is calculated as the quality score of the evaluation area.

In order to obtain the quality score more accurately, instead of directly using the value of the evaluation index, the following formula can be used to calculate the score of each evaluation index.

Use Equation 3 to calculate the scores for quantity, proportion, coverage, success rate, and rate indicators:

Use Equation 4 to calculate the score of the utilization rate, retransmission rate, disconnection rate, congestion rate, stall rate, stall duration ratio, delay, and interference noise indicators:

Through the above formula, the value of the evaluation index is calculated, and the weight of the evaluation index and the evaluation factor is calculated to obtain the quality score of the anchor station in the evaluation area.

A comprehensive comparison between the evaluation system model provided in this embodiment and the evaluation system model in Embodiment 2 shows that the evaluation system model in this embodiment is superior to the evaluation system model in Embodiment 2, and the dimension of the evaluation system model in this implementation is more It is more detailed and has wider coverage, and has more value in end-to-end quality assessment.

Example 4:

On the basis of the methods for quality assessment of anchor stations under 5G NSA network provided by the foregoing Embodiments 1 to 3, the present invention also provides a device for quality assessment of anchor stations under 5G NSA network that can be used to implement the above method , as shown in FIG. 6 , which is a schematic diagram of an apparatus architecture according to an embodiment of the present invention. The apparatus for evaluating the quality of the anchor station under the 5G NSA network in this embodiment includes one or more processors 21 and a memory 22 . Among them, one processor 21 is taken as an example in FIG. 6 .

The processor 21 and the memory 22 may be connected through a bus or in other ways, and the connection through a bus is taken as an example in FIG. 6 .

The memory 22 is used as a non-volatile computer-readable storage medium for the quality assessment method of the anchor station under the 5G NSA network, and can be used to store non-volatile software programs, non-volatile computer-executable programs and modules, as in Embodiment 1 To the method for quality assessment of anchor station under 5G NSA network in Embodiment 3. The processor 21 executes various functional applications and data processing of the apparatus for quality assessment of the anchor station under the 5G NSA network by running the non-volatile software programs, instructions and modules stored in the memory 22, that is, to implement Embodiment 1 to The method for quality assessment of the anchor station under the 5G NSA network of the third embodiment.

Memory 22 may include high speed random access memory, and may also include nonvolatile memory, such as at least one magnetic disk storage device, flash memory device, or other nonvolatile solid state storage device. In some embodiments, the memory 22 may optionally include memory located remotely from the processor 21, and these remote memories may be connected to the processor 21 through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The program instructions/modules are stored in the memory 22, and when executed by one or more processors 21, perform the methods for quality assessment of anchor stations under the 5G network in the above-mentioned Embodiments 1 to 3, for example, perform the above-described methods The various steps shown in Figure 1, Figure 2 and Figure 3.

Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only Memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

The above descriptions are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention shall be included in the protection of the present invention. within the range.

Claims (10)

1. a method for quality assessment of anchor station under a 5G NSA network, characterized in that: Obtain anchor station quality index data; According to the evaluation dimension, the evaluation factors that need to be evaluated are selected from the quality index data; According to the evaluation business needs, configure different weights for each evaluation factor; According to the value and weight of the evaluation factor, the weighted total score of each evaluation factor in the preset evaluation area is calculated as the quality score of the preset evaluation area. 2. The method for evaluating the quality of anchor stations under a 5G NSA network according to claim 1, wherein the obtaining anchor station quality index data comprises: passing drive tests, single-site verification, network management data, and DPI data Obtain anchor site quality index data by one or more of the methods among others. 3 . The method for evaluating the quality of an anchor station under a 5G NSA network according to claim 1 , wherein the evaluation dimension includes a network service quality dimension and/or a network service perception dimension. 4 . 4. The method for evaluating the quality of an anchor station under a 5G NSA network according to claim 1, wherein the different weights are configured for each evaluation factor, comprising: Select evaluation factors to be evaluated according to business needs, and set weights for each evaluation factor according to business characteristics; Select the evaluation indicators that need to be evaluated for each evaluation factor, and set weights for each evaluation indicator according to business characteristics. 5. The method for evaluating the quality of an anchor station under a 5G NSA network according to claim 4, wherein the calculating the weighted total score of each evaluation factor in the preset evaluation area comprises: According to the value and weight of each evaluation indicator, calculate the weighted total score of all evaluation indicators in each evaluation factor, and use the weighted total score of all evaluation indicators as the value of the evaluation factor; According to the value and weight of each evaluation factor, a weighted total score for all evaluation factors is calculated. 6. The method for evaluating the quality of anchor stations under the 5G NSA network according to claim 4, wherein the calculation of the weighted total score of all evaluation factors in the preset evaluation area includes: Preset baseline and challenge values for each evaluation indicator in each evaluation factor; When the value of the first type of evaluation index is less than the reference value, the evaluation index is calculated according to the preset lower limit of the evaluation index; When the value of the first type of evaluation index is greater than or equal to the challenge value, the evaluation index is calculated according to the preset upper limit value of the evaluation index; When the value of the second type of evaluation index is greater than the reference value, the evaluation index is calculated according to the preset lower limit of the evaluation index; When the value of the second type of evaluation index is less than or equal to the challenge value, the evaluation index is calculated according to the preset upper limit value of the evaluation index. 7. The method for evaluating the quality of anchor stations under a 5G NSA network according to claim 3, wherein when the evaluation dimension is a network service quality dimension, coverage level, anchor capacity, interference avoidance and anchor One or more of the performances are used as evaluation factors. 8. The method for evaluating the quality of an anchor station under a 5G NSA network according to claim 3, wherein when the evaluation dimension is the network service perception dimension, one or multiple as an evaluation factor. 9 . The method for evaluating the quality of an anchor station under a 5G NSA network according to claim 8 , wherein when the terminal is used as the evaluation factor, the terminal has the same frequency as the 5G NSA base station and supports dual connection capability. 10 . 5G NSA terminal. 10. A device for quality assessment of anchor stations under a 5G NSA network, characterized in that: It includes at least one processor and a memory, the at least one processor and the memory are connected by a data bus, the memory stores instructions that can be executed by the at least one processor, and after the instructions are executed by the processor , a method for completing the quality assessment of an anchor station under a 5G NSA network according to any one of claims 1-9.

Images