[go: up one dir, main page]

WO2019114947A1 - Technique d'analyse de qualité de service dans un réseau de télécommunications - Google Patents

Technique d'analyse de qualité de service dans un réseau de télécommunications Download PDF

Info

Publication number
WO2019114947A1
WO2019114947A1 PCT/EP2017/082632 EP2017082632W WO2019114947A1 WO 2019114947 A1 WO2019114947 A1 WO 2019114947A1 EP 2017082632 W EP2017082632 W EP 2017082632W WO 2019114947 A1 WO2019114947 A1 WO 2019114947A1
Authority
WO
WIPO (PCT)
Prior art keywords
event
time sequence
types
quality
group
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/EP2017/082632
Other languages
English (en)
Inventor
István GÓDOR
Péter Hága
Zsófia KALLUS
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Telefonaktiebolaget LM Ericsson AB
Original Assignee
Telefonaktiebolaget LM Ericsson AB
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Telefonaktiebolaget LM Ericsson AB filed Critical Telefonaktiebolaget LM Ericsson AB
Priority to PCT/EP2017/082632 priority Critical patent/WO2019114947A1/fr
Publication of WO2019114947A1 publication Critical patent/WO2019114947A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
    • H04L41/5061—Network service management, e.g. ensuring proper service fulfilment according to agreements characterised by the interaction between service providers and their network customers, e.g. customer relationship management
    • H04L41/5067—Customer-centric QoS measurements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/22—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/06—Generation of reports
    • H04L43/067—Generation of reports using time frame reporting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H04L41/064—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving time analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/06—Management of faults, events, alarms or notifications
    • H04L41/0631—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis
    • H04L41/065—Management of faults, events, alarms or notifications using root cause analysis; using analysis of correlation between notifications, alarms or events based on decision criteria, e.g. hierarchy, tree or time analysis involving logical or physical relationship, e.g. grouping and hierarchies

Definitions

  • the present disclosure relates to analyzing quality of service (QoS) in a telecommunications network.
  • QoS quality of service
  • Network operation centers are places where such analysis takes place. The analysis is based on logging events occurring on the network and processing them to identify current network problems with a view to diagnosing the root causes.
  • Mobile communication data can be captured on standardized interfaces (Gi, Gn, Gp, Iu, SI, Sll, etc.) and on certain nodes that provide performance data collection features in Global System for Mobile Communications (GSM), 3 rd Generation (3G), and Long Term Evolution (LTE) networks.
  • GSM Global System for Mobile Communications
  • 3G 3 rd Generation
  • LTE Long Term Evolution
  • NOCs can apply complex analytics to telecommunications usage data collected from user equipments and other network entities.
  • Usage data may include traditional voice calls, traditional text (such as short message service, SMS) messages, data upload and download, social media usage (Twitter, Facebook, WhatsApp etc.).
  • a mobile phone or other user equipment effectively acts as a monitoring device through its modes of use and associated sensor data which it collects, such as global positioning system (GPS) location, photographs or videos.
  • GPS global positioning system
  • expert domain knowledge can be hard-coded into fixed algorithms with fixed parameter settings.
  • fully automatic, data-driven algorithms with machine learning can be used to analyze the usage data.
  • telecommunication networks are capable of generating millions or billions of events every second, such as core node logs, radio access network (RAN) logs, counters, user equipment (UE) reports and signaling events.
  • Real-time processing and root-cause analysis of each newly-occurring complex pattern needs in-depth dedicated analysis.
  • Visualization of system performance can facilitate the process of defining new high-level events and identifying the root causes of drops in quality.
  • new domains of analytics see new, more sophisticated ways of detecting events, new ways of presenting results become a necessity as an analyst should be able to benefit from such analytical results without requiring a difficult learning curve.
  • a com puter-a utomated method for analyzing quality of service delivered to a user of a user equipment in a telecommunications network comprises receiving a data set containing a log of a first time sequence of user session events performed on a user equipment in which a value of a quality of service parameter is recorded for each user session event, and each user event has been classified into one of a plurality of event types.
  • the method further comprises creating a second time sequence from the first time sequence by aggregating the events into event groups, wherein each event group is defined as a plurality of events which are in a specific sequence of event types, each event group being classified into one of a plurality of event group types.
  • the method still further comprises creating a third time sequence from the second time sequence by aggregating the event groups into event super-groups, wherein each event super-group is defined as a plurality of event groups which are in a specific sequence of event group types, each event super-group being classified into one of a plurality of event super-group types.
  • the method also comprises rendering into a visualization at least one of the time sequences such that in the visualization each of said types is visually distinct from other types in the same time sequence.
  • a computer system for analyzing quality of service delivered to a user of a user equipment in a telecommunications network.
  • the system comprises a data input operable to receive a data set containing a log of a first time sequence of user session events performed on a user equipment in which a value of a quality of service parameter is recorded for each user session event, and each user event has been classified into one of a plurality of event types.
  • the system further comprises a memory operable to store the data set.
  • the system still further comprises a processor operable to analyze the data set.
  • the system also comprises an output operable to output the visualization to a display.
  • the processor is operable to analyze the data set through performing the actions of:
  • each event group is defined as a plurality of events which are in a specific sequence of event types, each event group being classified into one of a plurality of event group types;
  • each event super-group is defined as a plurality of event groups which are in a specific sequence of event group types, each event super-group being classified into one of a plurality of event super-group types;
  • the proposed visualization method and system can complement existing monitoring methods and systems by collecting, analyzing and visualizing QoS data from user sessions in a telecommunication network.
  • the quality of service parameter may for example be a video quality parameter, an audio quality parameter, and/or a webpage loading latency quality parameter.
  • Other examples of the quality of service parameter are data download speed, data upload speed, number and/or frequency of call drops for voice, and number, duration and/or frequency of call breaks for voice.
  • any parameter that is relevant for data traffic or any parameter relevant for the user's actual or perceived quality of service may be used.
  • the quality of service parameter may be a compound parameter aggregated from two or more of the above examples. The aggregation may be with equal weightings or different weightings, for example a combination of video and audio quality may be used with a greater weighting on audio than video.
  • the QoS parameter is one of perceived quality, i.e. is a quality as subjectively determined by a user, which is often referred to in the art as quality of experience (QoE).
  • QoS quality of experience
  • the term QoS is used as a generic term to cover both objective and subjective quality measures, with the term QoE being used specifically for subjective quality measures.
  • QoE is thus a subset of QoS.
  • a given QoE value is thus set by a user according to his or her subjective assessment. For example, after a video call or voice call using Voice over IP (VoIP), the user may be asked to rate the call quality, and it is this rating that is used as the parameter value.
  • the QoE rating could follow the mean opinion score (MOS) scheme.
  • MOS is one way for users to give a numerical indication of the perceived quality of a media and has a range of 1 to 5, where 1 is the worst and 5 the best. The values are in defined as:
  • MOS values do not need to be whole numbers. Certain thresholds and limits are often expressed in decimal values from this MOS spectrum. For instance, a MOS value of 4.0 to 4.5 is referred to as toll-quality and causes complete satisfaction. This is the normal value of public switched telephone network (PSTN) and a common goal for VoIP services.
  • PSTN public switched telephone network
  • the quality of service parameter is an objective measure of quality of service and has its value determined by a performance measurement of the telecommunications network, which may take place on user equipments, or at some other location within the telecommunications network.
  • the user equipment can measure the frame rate of a video being played and grade quality according to deviations from the desired frames per second rate, e.g. according to the frequency and magnitude of frame rate interruptions.
  • Another example would be a packet drop rate between two nodes in the telecommunications network.
  • a range of the quality of service parameter values may be represented by a range of values of a visualization parameter, e.g. in color or opacity.
  • a visualization parameter e.g. in color or opacity.
  • QoS degradations can be recognized because the multi-level hierarchical analysis combined with the visualization allows expert's brains to recognize missing information elements and variance of delays between individual events. Specifically a domain expert is able to interpret the visualization of the mid and top-level time sequences, even of incomplete data sets, to separate out patterns relating to different respective problems which overlap, which then allows an exact timing sequence of event types to be recognized as being characteristic of a given kind of network problem. Experts can feed this information to machine learning (ML) functionalities to extend automatic problem detection features of the network operation system.
  • ML machine learning
  • fourth and further higher order time sequences can be created in the same way as the recited second time sequence is created from the first time sequence, and the third time sequence is created from the second. That is an nth order time sequence can be created from an (n-l)th order time sequence by aggregating and classifying, so that the analysis levels can be built up ad infinitum to any desired level.
  • the method or computer system may further comprise creating at least one higher order, nth time sequence from the previous highest order, (n-l)th time sequence by aggregating the groups of the previous highest order, referred to as sub-ordinate groups, into supra-ordinate groups, wherein each supra-ordinate group is defined as a plurality of sub-ordinate groups which are in a specific sequence of sub-ordinate group types, each supra-ordinate group being classified into one of a plurality of event supra-ordinate group types.
  • a com puter-a utomated method for analyzing QoS data, in which the method comprises: receiving a data set containing a log of a first time sequence of network events in which each network event has been classified into one of a plurality of event types; creating a second time sequence from the first time sequence by aggregating the events into event groups, wherein each event group is defined as a plurality of events which are in a specific sequence of event types, each event group being classified into one of a plurality of event group types; and rendering into a visualization the first time sequence and/or the second time sequence such that in the visualization each of said types is visually distinct from other types in the same time sequence.
  • a corresponding computer system for performing this method may also be provided.
  • the proposed method and system can also be integrated into existing monitoring methods and systems extended with streamed data processed by real-time analytics for visual monitoring of events.
  • the events may each be specific to one or more particular network entities, such as a user equipment, a cell, a base station. Moreover, the events may each be specific to a particular application, such as a music or video streaming application, such as Spotify, Facebook live, or Youtube, or a VoIP application, such as Skype.
  • a music or video streaming application such as Spotify, Facebook live, or Youtube
  • a VoIP application such as Skype.
  • a location may be associated with each event, for example: UE location, participating
  • transmitter/receiver network entity location or mobile cell a server or other data repository location in a network from where a cloud service serves UEs.
  • An event is defined or characterized by a specific pattern being present in the measured or deduced timeline of the monitored system. This definition applies at each hierarchical analysis level.
  • An event may be any event defined by the method or system for collection into the data set, in particular events known or thought to have relevance for network performance.
  • An event may be defined by an expert, or be defined automatically as part of the present or a previous analysis.
  • a simple event might be a type of usage, e.g. making a VoIP or video call.
  • a more complex event may be a peak in UE usage of a particular application in a particular geographical area or for a particular class of user, such as a peak in downloading/watching video clips which could be measured in terms of the application(s) being used, e.g. YouTube, Facebook live, Catch up TV application, or file type, e.g. moving pictures expert group (MPEG) 3 or 4 standard.
  • MPEG moving pictures expert group
  • the visualization includes a map representation of events in relation to their network locations and according to at least one of the second time sequence and the third time sequence.
  • the analysis further comprises recognizing a pattern in the data set by matching the current event groups and/or super-groups to a first time period of at least one stored data set in which the same event groups and/or super-groups are present. In this way, it can be predicted how events may develop with reference to a second time period of the matched stored data set or sets, wherein the second time period follows the first time period.
  • future usage can be predicted based on warping, i.e. applying a warp transform to, the first time period of the at least one stored data set onto the current data set and using the warped second time period of the at least one stored data set as the prediction.
  • the network loading can be monitored in real-time to detect that a concert is taking place, and historical data can be analyzed to predict what streaming usage is likely to occur during and after the concert. For example, there may be a peak in live video streaming during the concert as the concertgoers upload live video feeds of the concert, and a peak in downloading music or videos from the artist as the concertgoers are travelling home. Appropriate short term capacity increases can then be put in place by the telecommunication operating company.
  • the visualization may include a map representation of the predicted future network loading. This map representation of the prediction may be integrated in or separate from a map representation of the data set being analyzed.
  • the one or more map representation(s), i.e. the map representation of the analyzed data set or the prediction, may encode time with: color or shading in a single image frame; or such that separate image frames relate to specific times or periods of time which can be displayed in time order.
  • the one or more map representation(s) may be to scale of a real geographical map, or may be a schematic representation of a network diagram showing users locations, user equipment locations, nodes or entities of the network and their interconnections and/or juxtaposition, e.g. network cells.
  • user actuatable controls may be provided for modifying the map
  • User actuatable controls may also be provided for the visualization of at least one of the first time sequences, the second time sequence and the third time sequence to filter in and out based on at least one of event types, event group types, and event super-group types respectively.
  • Changing the visualizations, including the map representation by adjusting the settings of the user actuatable controls may also be used to predict future network loading and associated QoS values. Namely, predicting future network loading can be based on applying said user actuatable controls to filter out at least some of the events contained in the data set. A modified version of the data set can then be saved, with the events which have been filtered out from the visualization being removed, i.e. not saved, so that the saved events are those that have been selected by the user actuatable controls and the filters they represent.
  • the predicted future QoS values can be used to provision additional resource on the network in order to mitigate the possibility of QoS values dropping below acceptable thresholds. Namely, the predicted future network loading can be compared to existing network capacity to predict whether QoS problems are likely to occur. If they are, then action can be taken automatically or by manual intervention to provision additional network capacity to address any such QoS drops before they are predicted to occur.
  • the network location can be a geographical location and/or an association with a network entity in a network diagram.
  • the event group types can be pre-defined, or defined as part of the analysis, or a combination of both. When defined as part of the analysis, the event group types can be defined as part of creating the second time sequence from the first time sequence and according to the first time sequence.
  • the event location could be used (e.g. UE/Internet Protocol (IP) address at residential address, UE/IP address at business address), or data identifying demographics of the user (e.g. student, office worker) which could be obtained from contract data or analytics of movement and usage.
  • IP Internet Protocol
  • the event super-group types may also either be pre-defined or defined as part of the analysis, or a combination of both. When defined as part of the analysis, the event super-group types can be defined as part of creating the third time sequence from the second time sequence and according to the second time sequence.
  • each time series should be represented in a way in which a user can easily distinguish between them. Namely, in each time series each type is ascribed a different visual characteristic for the visualization, for example a different color picked from a color chart.
  • each event is ascribed a value of a QoS parameter, which may be a continuously variable scalar parameter which can adopt a value within a particular range.
  • the QoS parameter can be represented in the visualization such that the range of the QoS parameter values is represented by a range of values of a visualization parameter. For example, if different colors are used to distinguish different types, then color saturation could be used as the visualization parameter. Alternatively, luminosity or brightness could be used according to the HSL (Hue-Saturation-Lightness) or HSB (Hue- Saturation-Brightness) color representations respectively. Another alternative would be to use opacity as the visualization parameter.
  • the visualization can be performed by a custom graphical user interface (GUI) which receives the time series which processes the time series data to present images in a timeline which have any one or more of the following attributes: real-time, animated, searchable, zoomable in and out in time and/or space (which may be real space or space in a network topology of interlinked nodes), contain icons for elements that are linked to a particular location, contain alert messages.
  • GUI graphical user interface
  • the kinds of occurrences that can be monitored and analyzed are for example: changes in QoS during the morning/evening rush hour on a work day/Satu rday/Su nday, lunch period on a work day/Saturday/Sunday, during stadium usage event for sport/concert on a work day/evening or Saturday daytime/evening or Sunday daytime/evening.
  • a computer program stored on a computer readable medium and loadable into the internal memory of a computer, comprising software code portions, when said program is run on a computer, for performing the method of the above aspects.
  • a computer program product may also be provided for storing the computer program.
  • Figure 1 shows a multi-level hierarchical analytics and visualization method and system as
  • FIG 2 shows aspects of the method and system of Figure 1 in more detail.
  • Figure 3 shows an example of event types and data feeds with three vertical levels and three horizontal levels.
  • Figure 4 shows a hierarchical visualization of a multi-level quality of service variation analysis.
  • Figure 5 shows a structure of a computer system and computer program code that may be used to implement the disclosed methods.
  • Figure 6 is a flow diagram of an embodiment of a computer-automated method of an embodiment of the disclosure for analyzing quality of service data.
  • Figure 7 is a block schematic diagram showing a computer system of an embodiment of the
  • disclosure configured to analyze quality of service data.
  • Memory may comprise any known computer-readable storage medium, which is described below.
  • cache memory elements of memory provide temporary storage of at least some program code (e.g., program code) in order to reduce the number of times code needs to be retrieved from bulk storage while instructions of the program code are carried out.
  • program code e.g., program code
  • memory may reside at a single physical location, comprising one or more types of data storage, or be distributed across a plurality of physical systems in various forms.
  • the proposed computer system is able to operate as a standalone entity taking a data set as input, i.e. the quality of service parameter values, and presenting its analysis as a visualization on a display that is part of the computer system and/or outputting the rendered data in a format suitable presenting the visualization on an external display.
  • the computer system will include one or more of the following features: a data collection interface for inputting the data; data integration functionalities for domain-specific parameters; and a graphical user interface possibly having different implementation variants for different display devices. The details of these additions are well known to a person skilled in the art.
  • the proposed computer system is also suitable to be integrated into an existing reporting and analytics computer system.
  • Figure 1 shows a multi-level implementation of hierarchical analytics and visualization method and system as envisaged by embodiments of the disclosure.
  • there are three analytics levels but it will be appreciated from the following that four, five, six or more analytic levels could be provided following the teachings provided.
  • input module 10 receives session data from individual users, which collectively constitute an input data set for analysis.
  • the input data set contains, for each of a plurality of users, a log of a first time sequence of events. (A time sequence of events may be referred to as a timeline elsewhere in this document.)
  • Each user event may be associated with one or more network locations, e.g. by being associated with a network entity such as a user equipment or network data repository.
  • Basic analytics are applied by a basic-level, basic analytics module 12 to classify the events into one of a plurality of event types.
  • the basic events timeline is rendered in a basic-level visualization module 13 by the method into a visualization which may be displayed to a user via a graphical user interface.
  • Mid-level analytics are then applied by a mid-level analytics module 14 which has the role of creating a second time sequence from the first time sequence obtained from the basis analytics by aggregating the network events into event groups.
  • Each event group is defined as a plurality of events which are in a specific sequence of event types.
  • Each event group is classified into one of a plurality of event group types.
  • the mid-level analytics may employ a pattern recognition algorithm such that the event groups are created when predefined patterns are recognized or uncovered for the first time.
  • the mid-level events timeline is rendered by a mid-level visualization module 15 into a visualization which may be displayed to a user via a graphical user interface.
  • Top-level analytics are then applied by a top-level analytics module 16 which has the role of creating a third time sequence from the second time sequence by aggregating the event groups into higher level event groups, which is referred to herein as super-groups to distinguish them from the mid-level analytics groups and to reflect the fact that they are supra-ordinate to the mid-level groups.
  • Each event super-group is defined as a plurality of event groups which are in a specific sequence of event group types. Moreover, each event super-group is classified into one of a plurality of event supergroup types.
  • the top-level analytics may employ a pattern recognition algorithm such that the event super-groups are created when predefined patterns are recognized or uncovered for the first time.
  • the top-level events timeline is rendered by a top-level visualization module 17 into a visualization which may be displayed to a user via a graphical user interface.
  • Each of the three timelines are co-rendered by a visualization module 18 which combines the outputs of the individual visualization modules 13, 15, 17 so that the three levels of timeline are presented alongside each other in a single visualization of all three analytics levels, so that their juxtaposition allows for direct visual comparison by a domain expert user.
  • the rendering of the basic, mid and top-level events is done in such a way that within each visualization level the different types are visually distinct from each other.
  • the visualizations may create the timelines with events being color-coded (i.e. hue) by look up to a customizable color chart, and a quality of service measure being coded within each color by another color parameter (saturation or brightness in HSB, or saturation or lightness in HSL).
  • the quality of service value could be coded by opacity.
  • the visualization may allow user customization, for example: a real-time sliding window view in which the user can set the time window to be visualized; zooming in or out in time after the session has finished; and enabling predefined time-warping techniques in which event types deemed to have higher importance are highlighted by extra horizontal (i.e. time axis) extension.
  • Any of the basic, mid and top-level analytics may use automatic classification to classify the events, event groups and event super-groups respectively, wherein the automatic classification may use pattern recognition as described in any of the following publications, the entire contents of which are incorporated herein by reference:
  • a characteristic may be an amount of time that is needed for the stadium to fill with spectators or an amount of time needed for the spectators to leave the stadium after an event.
  • Such a characteristic may be a scalar quantity which is continuously variable to reflect what proportion of the stadium capacity will be utilized, e.g. sell out to negligible ticket sales.
  • the value of the characteristic will be different in case of music concerts of national and international stars, or local football matches and international cup matches as presented in, for example, the previously-mentioned publication: I. Godor, P. Jonsson, Z. Kail us, D. Kondor - The Digital Signatures of Sport, Ericsson Mobility Report, pages 20-23, June 2015, the entire contents of which is incorporated herein by reference.
  • the characteristic is useful to predict how the network should adapt to avoid quality of service issues arising, or to solve existing quality of service issues, and whether and how the spreading of the event might be expected throughout the network beyond the neighborhood of the stadium.
  • the analysis is thus performed in a hierarchical manner to give three levels of network performance evaluation, by feeding the output time series as input for mid-level pattern detection and feeding the mid-level output as input for top-level pattern detection.
  • the three resulting timelines are displayed together in a hierarchical diagram, e.g. a cake diagram.
  • the pattern detection algorithm applied at the mid and top-levels can be trained by experts and/or by applying machine learning to historic datasets.
  • the first event analytics level is constituted by the events that have been logged in the network which may be actual events, i.e. raw network data, or pre-processed network data, e.g. the events may be peaks in particular actual events.
  • Such events are referred to as basic events which form a basic timeline, e.g. from a single user or user equipment, being formed by events identified in the raw measurement stream or streams that have been collected for analysis.
  • the basic events may be a specific type of event as seen on load measurements of a part of the network, such as a cell, or for a particular class of users.
  • the second event analytics level is constituted by specific basic event patterns which are each formed by specific combinations of types of basic events (from the first level). Each specific combination of basic event types may be termed a mid-level event type.
  • the third event analytics level is constituted by specific mid-level event type patterns, i.e. specific combinations of mid-level event types, which may be termed top-level event types.
  • the raw measurements may come from a single stream or multiple streams which are combined into the first hierarchical event analytics level. Since the analytics levels and the multi-stream network data are functionally orthogonal to each other, the levels of the hierarchical event analytics are referred to as vertical levels and the tiers of combination of the network data (if present) as horizontal levels,
  • the first event analytics level represents the raw physical measurable data with high granularity.
  • the first level data can originate from
  • the third event analytics level represents the directly observable consequences creating the user experience and defining the quality of services.
  • the second level represents the abstract space of key performance indicators (KPIs) and events identified by the patterns of first level measurements.
  • the following visualizations can be considered: expert-defined relations and alerts view; relations and correlations uncovered by the ML modules; and mixed view for comparing the various explanations of the same top-level result.
  • measures can be taken when designing the layout and graphic elements including: color choice, dynamic transparency, fading, blinking, icons, hover overs and layout.
  • the basic timeline of a single user is formed by all the events collected for that specific user (e.g., signaling events, probe events, quality measures, etc).
  • Specific mid-level event patterns define the top-level events (e.g., bad video quality or numerous video freezes during a session).
  • a simple mapping is creating event hierarchy using only a single user timeline.
  • a unified mapping is creating event hierarchy using the merged timelines of two or more users. This union is achieved by using the time ordering or matching of patterns and can be created on the basic or mid-level timelines.
  • a differential mapping is creating an event hierarchy where relative performance of two users or groups of users - e.g., based on demographic or socioeconomic classification - for comparative study. Matching of patterns before differentiation can be useful to look for event-related performance, and simple time ordering to look for session-level performance comparison.
  • Figure 1 shows the single-user timeline functions corresponding to the hierarchical event analytics and visualization method on three vertical levels.
  • Basic analytics in module 12 is performed on the raw data received from module 10.
  • the output from the basic analytics module 12 is provided to the basic level visualization module 13 and to the mid-level analytics module 14.
  • the output from the mid-level analytics module 14 is provided as input for mid-level visualization module 15 and the top-level analytics module 16.
  • the three levels together serve to form the 3-level hierarchical visualization of a given user's (or group of users) performance timeline.
  • Each created event timeline can serve as the basis of new pattern definition. New definitions can, in return, improve the respective analytics logic.
  • FIG 2 shows the workflow of the basic analytics module of Figure 1 in more detail.
  • ML machine learning
  • FIG. 2 shows the workflow of the basic analytics module of Figure 1 in more detail.
  • Each box labeled "ML” is a machine learning (ML) algorithm module which applied its ML algorithm, based on pattern recognition, to the input data received as input to that analysis level.
  • Each of the storage icons shows a reference database which at each hierarchical level has the role of hosting a reference database referenced by the ML algorithm during processing the data, and also of providing storage for the input data to that level and the output from the ML algorithm.
  • Time-stamped raw sensor data 20 from the network is received at a first analysis level and stored in a reference database 20a.
  • a ML module 20b processes the raw sensor data by using pattern recognition to detect raw events in the raw sensor data and produce a time series of raw events.
  • the processed data is output to a raw event detection module 21.
  • Each raw event in the raw events time series is then transmitted to a quality metric addition module 22 and stored in a reference database 22a.
  • a ML module 22b processes the raw event detection data to deduce one or more quality metrics from each event, or at least those events deemed to be significant for the particular analysis being performed (so-called use case).
  • a quality metric is a discretely or continuously variable quantity that is a measure of goodness for performance, either objective from a network measurement such as packet loss, or subjective as set by quality of experience feedback from a user. The quality metrics are then associated with the relevant raw event from which they have been derived.
  • the processed data comprising the raw events with their quality metrics are then supplied to an event classifier module 23 which applies pattern recognition to classify each event into one of a plurality of event types based on a performance quality measure.
  • the classifier module 23 can be pre-configured with pre-defined event types or generate event types on the fly based on analyzing the raw event data and optionally also their associated quality metrics.
  • the event classifier outputs the time series, now appended with event classifications, to a duration measurement module 24 and stored in a reference database 24a.
  • a ML module 24b processes the time series and event classifications so as to associate each event with a time span, i.e. duration.
  • the processed data is output to an event processor module 25.
  • the event processor module 25 has the role of processing the time series data and associated data output from the multi-level analysis performed by the preceding stages in order to produce a characteristic measure (i.e. score) and performance quality indicators from the data. Namely, the event processor module 25 processes the multi-level analysis data from the preceding stages to add a quality measure to each event that is to be kept as a relevant event for the particular user experience, i.e. QoS, of interest.
  • This performance quality is the final assessment of the characteristic measure (or score), i.e. it is characteristic of an event which has meaning in the context of the use case and also a time dependency.
  • the events are color-coded, or otherwise visually tagged, according to event type.
  • the events are additionally visually tagged according to the value of a quality measure by saturation or luminosity/brightness of the event-type color, or some other suitable visual tagging which is distinct from, and preferably complementary to, the event type visual tagging.
  • the events with various scores at given times and associated time-dependent performance quality measures are parameters that the visualization module 28 can use for generating the visualization though suitable coloring, icon choice and so forth to generate the final GUI.
  • the user that is the person or persons tasked with evaluation and decision making based on their interpretation of the visualization, e.g. in a live system, need to know the performance quality measure(s) that are being rendered, but do not need any knowledge of how the analysis is performed in detail in order to be able to understand the visualization, e.g. to identify the root cause behind a current quality of service problem, or to predict a future quality of service problem, though the visualized timelines of events and their scores at the various levels.
  • definition of event types may be pre-defined by an expert, or defined on-the-fly by the ML algorithm, or any combination of the two at any given analysis level.
  • the proposed approach therefore permits a duality at each step of expert vs machine learning for definition of event types based on previously observed or expected patterns.
  • the same point can also be made for the performance quality measures, i.e. they can be defined by experts in the field, or as part of the ML.
  • the whole process can be pre-programmed by field experts using pre-defined event types, event quality measures and ultimately the resulting
  • performance measure of the monitored system (calculated from events of various levels at specific times/places: put into context of the use case).
  • the implementation can heavily rely on automation where machine learning will find and define specific events and at most minimal settings of learning parameters are set by experts.
  • the learning can be performed on historic datasets and the learnt information can be used to fix the parameters that are used to process real-time data streams.
  • the learning could be done in real-time on the real-time data streams, so that the analysis stages user real-time ML algorithms that learn and detect on the incoming data stream in parallel and continuously.
  • the system can then improve its event detection and classification, and also define new event types, as more measured data is presented.
  • the analysis has the task of identifying, i.e. detecting events, in the incoming data stream, e.g. in real-time, of the system being monitored. Further the analysis will compare the incoming data to, for example, a baseline, or predefined thresholds of quality measures, which are relevant in the system being monitored, where this analysis may take account of location and time when assigning a characteristic quality measure to each detected event. The events are then filtered based on the quality measure so that events that are deemed to be relevant, are tagged as such, since these will be the events that are taken forward by the process for visualization. Events filtered out may be retained despite their tagging as non- relevant, or may be deleted, or at least not passed on to the next processing stage. The quality measure is used as a score for placing the event on a quality scale that is to be used for performance evaluation of an expert via the visualization.
  • the workflow of Figure 2 thus performs a number of tasks in series to create, from an input time series, a performance visualization timeline. These tasks can be summarized in order to be: pattern detection in time series of basic events identifying time and type of complex events in the raw data from the network; performance metric calculation of found complex events including event classification; and visualization according to timing, type and performance metrics.
  • Figure 3 shows an example of event types and data feeds with three vertical levels VI, V2, V3 and three horizontal levels HI, H2, H3 noting that the combined basic event data is both the top horizontal level H3 and the bottom vertical level VI.
  • the horizontal level HI could for example be single user measurements from individual UEs in neighboring cells, where the UE data from each cell is unified, so that in horizontal level H2 there is a unified timeline for each cell.
  • the unified timelines of the basic events could then be processed to generate horizontal level H3 by differentiation to find peaks in the basic event activity. This peak data is then the data input to the hierarchical analysis, i.e. data set H3 is data set VI.
  • events are characterized by basic parameters defining event type (e.g. for video streaming, loss of a data packet, change in radio signal strength) and event quality (e.g. for perceived quality by a user, MOS score, packet loss percentage in dB).
  • event type e.g. for video streaming, loss of a data packet, change in radio signal strength
  • event quality e.g. for perceived quality by a user, MOS score, packet loss percentage in dB.
  • the events, event groups and event super-groups are visualized in different colors through look up to a color coding chart and the quality measure by another color parameter or opacity as described above according to a scale defined by a preset.
  • a preset a definition is meant which maps a range of values of the quality measure to a range of values of the parameter used to visualize the quality measure, where the mapping may be linear or non-linear.
  • the visualization is suitable to allow domain experts to make a performance evaluation, and also to recognize when new patterns occur.
  • the QoS for a single user can be characterized based on summarized event quality over a linear timeline of a session. Differential mapping of the current timelines against stored historic timelines (or summaries thereof) can highlight trends in current performance.
  • a group of users can be analyzed by using unified mappings (i.e. mid level) and/or differential mappings (i.e. top-level) where mid and top-level quality measures can be based on simple or relative requirements.
  • unified mappings i.e. mid level
  • differential mappings i.e. top-level
  • the basic 'horizontal' hierarchy level is the events coming from single users.
  • the mid-level groups events from different groups of users.
  • the extent to which a QoS degradation is common to a certain group of users or network area can thus be identified both in temporal and spatial dimensions at the different 'horizontal' hierarchical levels. Examples of such errors are: a) topological errors even on physical or logical level, b) saturation of the network (which may be as detected in the user plane and/or the control plane), and c) user-specific errors which occur repetitively, e.g., in a given region of the network or in given time periods of the day, etc.
  • new mid and top-level event groups and super-groups can be performed by the domain expert without any data engineering knowledge as the visual tool facilitates intuitive understanding by inspection.
  • new groups and super-groups can be automatically proposed by self-learning pattern recognition algorithms to give a powerful combination of machine learning and domain expert knowledge. This in return facilitates the continuous updating of rule engines using the defined events for automating actions.
  • Outlier recognition by domain experts is similarly highly facilitated by the visual tool and its customization options.
  • automated higher-level analytical tasks, such as prediction, prevention, or outlier detection can be performed using machine learning techniques on historical mid or top-level timelines of users or user equipments with similar functions.
  • Figure 4 shows an example of hierarchical visual analysis illustrating service quality variation.
  • This illustration is an oversimplification in order to show the principles and has three analysis levels. In other embodiments, four or more analysis levels could be provided.
  • Figure 4 a specific embodiment is described which shows event analytics for quality of service monitoring in a telecommunications network. The time evolution of network loading is visualized with timelines at three levels: base level (one user), mid level (group of users) and top-level.
  • Level 1 On the single user level, the measurements of UE capabilities, radio properties and cell parameters are used.
  • Example radio properties with their labels in the figure in brackets are RSCP (a), ECNO (b), RSRQ (e) and RSRP (f).
  • Example cell parameters with their labels in the figure in brackets are total load (c), number of users (d) and transport properties (g). All of these are visualized according to their exact time and the corresponding measured quality of service parameter values.
  • Level 2 On the mid-level, complex events can be identified, e.g., video freezes (A) or pixelating of image (B), or buffer length drop (C). These are the compound effect of basic events such as: a series of packet drops, or an increased latency.
  • video freezes A
  • pixelating of image B
  • buffer length drop C
  • Level 3 On the top-level of this particular use-case one can see the decreasing video QoE (a - alpha) or VoLTE call drop (b - beta) as a result of the degradation of the radio service properties.
  • monitoring systems can learn what the normal behavior is, when the network should be upgraded to handle traffic increase tendencies or could react upon events that increase the traffic in given cells or regions, thereby can be assumed to spread through the network like in case of a marathon running in a city, where a large crowd follow the competition or "pop up" at major milestones of the competition.
  • root cause discovery one can think of a similar complaint from the user for loss of video quality.
  • this could be identified as a complex event of type 'alpha', where the root cause is overload of traffic on the cell, and a surrounding crowd effect
  • this could be identified as a complex event of type 'beta', where the root cause was an individual user's faulty device, and so was not accompanied by simultaneous complaints from other users in close proximity.
  • the visualization could present pixels covering different areas with an opacity which scales with usage volume in that area, or how current quality of service values differ from expected values for a recognized event type.
  • the proposed computer system and method relates to processing telecommunications network data to generate quality and performance analytics visualizations. It can be used to visualize a series of events identified during a session. It is suitable for various domains where performance is related to ordering and timing of specific event configurations, and pattern recognition is useful for evaluation of performance. Variations and hierarchical pattern formation, differential and unifying views are also proposed. Using input from real-time data streams, the proposed visualizations make it easy to derive insights and recognize complex patterns at a glance. In particular, the proposed visualizations are suitable for users who are experts of the underlying telecommunications network, and thus have extensive domain knowledge, but who are not necessarily experts either in analytics or the reporting system which collected the network data.
  • the multi-level pattern embedded in the proposed visualization can enable a domain expert to make an intuitive visual evaluation of performance during a session.
  • the timelines of different network entities such as users or groups of users, can be presented alongside each other in a single view to allow for direct visual comparison.
  • the visualization facilitates detection of trends, patterns, or anomalies in a series of events which in turn may facilitate improvement of existing rule engines.
  • the proposed visualizations are also adaptable to any domain where a series of events reoccurs in a dynamical manner, e.g. the morning or evening rush hour, or when a stadium hosts concerts or sporting events.
  • the proposed analysis is usable as part of a real-time reporting system to monitor how events are unfolding, as well being useable for analyzing historical data sets.
  • Video quality degradation may be caused by signaling and packet loss events.
  • the root cause analysis is a difficult task even for domain experts.
  • the reason behind it is that various patterns relating to respective errors can overlap each other and also it is often the case that given types of errors or issues do not have an exact timing, e.g. duration, associated with them. It is therefore not straightforward to recognize sequences of the basic (level 1) events that are characteristic of a given type of network problem.
  • the sequence of events might be incomplete for reasons such as some events might be lost, others may not be triggered, or the measurements may be live and the sequence of events may not yet have finished.
  • Such errors can be signaling errors or user-specific errors that occur repeatedly, e.g., in a given region of the network such as a cell or in given time periods of the day, e.g. in the morning rush hour.
  • An example hierarchical visual analysis was discussed above in relation to Figure 4.
  • a second example relates to handover in a high-speed train which shows how that, in a
  • the user experience can be affected by several factors that are outside of the common areas that are covered by the knowledge of a domain expert.
  • the user experience might be degraded even if all of the specific raw events carried out on the network were successful. Namely, if a user is travelling on a high-speed train the user experience might be degraded due to frequent handovers between cells. In this scenario, the expert would only see that multiple successful handovers have taken place, e.g. as measured through a good handover KPIs. In other words, everything is working fine from a network operation point of view. Nevertheless, the user experience will have been degraded due to the short time interval between handovers.
  • the ML can highlight the location specific constraints: the high speed of the train forces the frequent handovers.
  • Figure 5 shows a structure of a computer system and computer program code that may be used to implement any of the disclosed methods.
  • computer system 501 comprises a processor 503 coupled through one or more I/O Interfaces 509 to one or more hardware data storage devices 511 and one or more I/O devices 513 and 515.
  • Processor 503 may also be connected to one or more memory devices or memories 505.
  • At least one memory device 505 contains stored computer program code 507, which is a computer program that comprises computer-executable instructions.
  • the stored computer program code includes a program that implements the method and method aspects presented herein.
  • the data storage devices 511 may store the computer program code 507.
  • Computer program code 507 stored in the storage devices 511 is configured to be executed by processor 503 via the memory devices 505.
  • Processor 503 executes the stored computer program code 507.
  • Memory 505 may comprise any known computer-readable storage medium, which is described below.
  • cache memory elements of memory 505 provide temporary storage of at least some program code (e.g., program code 507) in order to reduce the number of times code needs to be retrieved from bulk storage while instructions of the program code are carried out.
  • program code 507 e.g., program code 507
  • memory 505 may reside at a single physical location, comprising one or more types of data storage, or be distributed across a plurality of physical systems in various forms.
  • I/O interface 509 comprises any system for exchanging information to or from an external source.
  • I/O devices 513, 515 comprise any known type of external device, including a display device (e.g., monitor), keyboard, etc.
  • a bus provides a communication link between each of the components in computer system 501, and may comprise any type of transmission link, including electrical, optical, wireless, etc.
  • I/O interface 509 also allows computer system 501 to store information (e.g., data or program instructions such as program code 507) on and retrieve the information from computer data storage unit 511 or another computer data storage unit (not shown).
  • Computer data storage unit 511 may comprise any known computer-readable storage medium.
  • computer data storage unit 511 may be a non-volatile data storage device, such as a semiconductor memory, a magnetic disk drive (i.e., hard disk drive) or an optical disc drive (e.g., a CD-ROM drive which receives a CD-ROM disk).
  • An implementation of the present disclosure may take the form of a computer program product embodied in one or more computer-readable storage medium(s) (e.g., memory 505 and/or computer data storage unit 511) having computer-readable program code (e.g., program code 507) embodied or stored thereon.
  • computer-readable storage medium(s) e.g., memory 505 and/or computer data storage unit 5111
  • program code e.g., program code 507
  • Program code (e.g., program code 507) embodied on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, radio frequency (RF), etc., or any suitable combination of the foregoing.
  • RF radio frequency
  • Step S61 there is received a data set containing a log of a first time sequence of user session events performed on a user equipment. A value of a quality of service parameter is recorded for each user session event. Moreover, each user event has previously been classified as belonging to one of a plurality of event types.
  • Step S62 there is created a second time sequence from the first time sequence by aggregating the events into event groups, wherein each event group is defined as a plurality of events which are in a specific sequence of event types, each event group being classified into one of a plurality of event group types.
  • Step S63 there is created a third time sequence from the second time sequence by aggregating the event groups into event super-groups, wherein each event super-group is defined as a plurality of event groups which are in a specific sequence of event group types, each event super-group being classified into one of a plurality of event super-group types.
  • Step S64 there is rendered into a visualization some desired combination of the first time sequence, the second time sequence and the third time sequence such that in the visualization each of said types is visually distinct from other types in the same time sequence. If all combinations are rendered, then a desired combination can be picked out by a user when the visualization is displayed.
  • Step S65 the visualization is displayed on a display.
  • Figure 7 is a block schematic diagram showing a computer system 70 of an embodiment of the disclosure configured to analyze quality of service data.
  • a basic-level analytics unit 71 is configured to pre-process input quality of service data to generate a data set containing a log of a first time sequence of user session events performed on a user equipment. A value of a quality of service parameter is recorded for each user session event.
  • each user event has previously been classified as belonging to one of a plurality of event types.
  • a mid-level analytics unit 72 is configured to create a second time sequence from the first time sequence by aggregating the events into event groups, wherein each event group is defined as a plurality of events which are in a specific sequence of event types, each event group being classified into one of a plurality of event group types.
  • a top-level analytics unit 73 is configured to create a third time sequence from the second time sequence by aggregating the event groups into event super-groups, wherein each event super-group is defined as a plurality of event groups which are in a specific sequence of event group types, each event super-group being classified into one of a plurality of event super-group types.
  • a rendering unit 74 is operable to render into a visualization a desired combination of the first time sequences, the second time sequence and the third time sequence such that in the visualization each of said types is visually distinct from other types in the same time sequence. If all combinations are rendered, then a desired combination can be picked out by a user when the visualization is displayed.
  • a display unit 75 is configured to receive the rendered visualization and display it for a user.
  • 4G is the 4th generation of mobile telecommunications technology as defined by the ITU
  • IMT International Mobile Telecommunication system
  • LTE Long Term Evolution
  • 5G is the 5 th generation of mobile telecommunications and wireless technology which is not yet fully defined, but in an advanced draft stage, e.g. in 3GPP TS 23.401 version 13.6.1 Release 13 of May 2016. For the purposes of this document LTE is considered to include 5G.
  • ECN0 Signal to noise ratio defined in 3GPP standard
  • ML machine learning
  • QoE Quality of Experience
  • QoS Quality of Service
  • RAN Radio Access Network
  • RSCP Received Signal Code Power defined in 3GPP standard
  • RSRP Reference Signal Received Power defined in 3GPP standard
  • RSRQ Reference Signal Received Quality defined in 3GPP standard RSS: Radio Signal Strength UA: is part of a UE and acts as a client in a transport protocol (TP) for communication with a server.
  • UE is a terminal that resides with the user which hosts a UA.
  • WiFi is an environment and interface allowing an electronic device, such as a UE, to wirelessly connect to, and form part of, a wireless LAN (WLAN).
  • WLAN wireless LAN

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Human Computer Interaction (AREA)
  • Telephonic Communication Services (AREA)

Abstract

L'invention concerne un système informatique et un procédé informatisé permettant d'analyser la qualité d'un service fourni à un utilisateur d'un équipement utilisateur dans un réseau de télécommunications. Un ensemble de données contient un journal d'une première séquence temporelle d'événements de session d'utilisateur effectuées sur un équipement d'utilisateur dans lequel une valeur d'un paramètre de qualité de service est enregistrée pour chaque événement de session d'utilisateur. Une deuxième séquence temporelle est générée à partir de la première séquence temporelle par agrégation des événements en groupes d'événements, et au moins une troisième séquence temporelle est générée par agrégation des groupes d'événements en super-groupes d'événements. Une hiérarchie d'événements de séquence temporelle multi-niveaux est ainsi créée, qui est rendue dans une visualisation. La visualisation révèle des modèles d'expert de domaine de comportement dans l'ensemble de données qui peuvent être utilisés pour détecter et comprendre des problèmes de qualité de service rencontrés par des utilisateurs.
PCT/EP2017/082632 2017-12-13 2017-12-13 Technique d'analyse de qualité de service dans un réseau de télécommunications Ceased WO2019114947A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/EP2017/082632 WO2019114947A1 (fr) 2017-12-13 2017-12-13 Technique d'analyse de qualité de service dans un réseau de télécommunications

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/EP2017/082632 WO2019114947A1 (fr) 2017-12-13 2017-12-13 Technique d'analyse de qualité de service dans un réseau de télécommunications

Publications (1)

Publication Number Publication Date
WO2019114947A1 true WO2019114947A1 (fr) 2019-06-20

Family

ID=60888388

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2017/082632 Ceased WO2019114947A1 (fr) 2017-12-13 2017-12-13 Technique d'analyse de qualité de service dans un réseau de télécommunications

Country Status (1)

Country Link
WO (1) WO2019114947A1 (fr)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363194A (zh) * 2021-12-28 2022-04-15 中国电信股份有限公司 网络设备的隐患分析方法、装置、电子设备及存储介质
CN114726749A (zh) * 2022-03-02 2022-07-08 阿里巴巴（中国）有限公司 数据异常检测模型获取方法、装置、设备、介质及产品
CN116595425A (zh) * 2023-07-13 2023-08-15 浙江大有实业有限公司杭州科技发展分公司 一种基于电网调度多源数据融合的缺陷识别方法
CN118395022A (zh) * 2024-06-25 2024-07-26 安徽思高智能科技有限公司 一种时空感知多组件图注意力时序QoS预测方法及系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110214020A1 (en) * 2010-03-01 2011-09-01 Microsoft Corporation Root cause problem identification through event correlation
US8656226B1 (en) * 2011-01-31 2014-02-18 Open Invention Network, Llc System and method for statistical application-agnostic fault detection
US20170149626A1 (en) * 2012-10-29 2017-05-25 T-Mobile, U.S.A., Inc. Remote driving of mobile device diagnostic applications

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110214020A1 (en) * 2010-03-01 2011-09-01 Microsoft Corporation Root cause problem identification through event correlation
US8656226B1 (en) * 2011-01-31 2014-02-18 Open Invention Network, Llc System and method for statistical application-agnostic fault detection
US20170149626A1 (en) * 2012-10-29 2017-05-25 T-Mobile, U.S.A., Inc. Remote driving of mobile device diagnostic applications

Non-Patent Citations (7)

* Cited by examiner, † Cited by third party
Title
D. KONDOR; P. THEBAULT; S. GRAUWIN; I. GODOR; S. MORITZ; S. SOBOLEVSKY; C. RATTI: "A Tale of Many Cities — Visualizing Signatures of Human Activity in Cities Across the Globe", LANDSCAPE ARCHITECTURE FRONTIERS, vol. 3/3, June 2015 (2015-06-01), pages 54 - 61
EAGLE; PENTLAND: "Reality mining: sensing complex social systems", PERSONAL AND UBIQUITOUS COMPUTING, vol. 10, no. 4, 2006, pages 255 - 268, XP058231399, DOI: doi:10.1007/s00779-005-0046-3
GONZALEZ, HIDALGO; BARABASI: "Understanding individual human mobility patterns", NATURE, vol. 453, 2008, pages 779 - 782, XP055011504, DOI: doi:10.1038/nature06958
GOODALL J R ET AL: "Preserving the big picture: visual network traffic analysis with TNV", VISUALIZATION FOR COMPUTER SECURITY, 2005. (VIZSEC 05). IEEE WORKSHOP ON MINNEAPOLIS, MN, USA OCT. 26, 2005, PISCATAWAY, NJ, USA,IEEE, PISCATAWAY, NJ, USA, 26 October 2005 (2005-10-26), pages 47 - 54, XP010852596, ISBN: 978-0-7803-9477-3, DOI: 10.1109/VIZSEC.2005.1532065 *
I. GODOR; P. JONSSON; Z. KALLUS; D. KONDOR: "The Digital Signatures of Sport", ERICSSON MOBILITY REPORT, June 2015 (2015-06-01), pages 20 - 23
I. GODOR; Z. KALLUS; D. KONDOR; S. GRAUWIN; S. SOBOLEVSKY: "The Signatures of City Life", ERICSSON MOBILITY REPORT, November 2014 (2014-11-01), pages 28 - 29
S. GRAUWIN; S. SOBOLEVSKY; S. MORITZ; I. GODOR; C. RATTI: "New York, London and Hong Kong, Computational Approaches for Urban Environments", 2015, SPRINGER, article "Towards a comparative science of cities: using mobile traffic records", pages: 363 - 387

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114363194A (zh) * 2021-12-28 2022-04-15 中国电信股份有限公司 网络设备的隐患分析方法、装置、电子设备及存储介质
CN114363194B (zh) * 2021-12-28 2024-04-30 中国电信股份有限公司 网络设备的隐患分析方法、装置、电子设备及存储介质
CN114726749A (zh) * 2022-03-02 2022-07-08 阿里巴巴（中国）有限公司 数据异常检测模型获取方法、装置、设备、介质及产品
CN114726749B (zh) * 2022-03-02 2023-10-31 阿里巴巴（中国）有限公司 数据异常检测模型获取方法、装置、设备及介质
CN116595425A (zh) * 2023-07-13 2023-08-15 浙江大有实业有限公司杭州科技发展分公司 一种基于电网调度多源数据融合的缺陷识别方法
CN116595425B (zh) * 2023-07-13 2023-11-10 浙江大有实业有限公司杭州科技发展分公司 一种基于电网调度多源数据融合的缺陷识别方法
CN118395022A (zh) * 2024-06-25 2024-07-26 安徽思高智能科技有限公司 一种时空感知多组件图注意力时序QoS预测方法及系统

Similar Documents

Publication Publication Date Title
EP3574611B1 (fr) Système et procédé d'analyse de données de performance de réseau
US11271796B2 (en) Automatic customer complaint resolution
US11005705B1 (en) Network fault detection and quality of service improvement systems and methods
US20190028909A1 (en) Adaptive health status scoring for network assurance
Wu et al. CellPAD: Detecting performance anomalies in cellular networks via regression analysis
CN104145500A (zh) 用于维持或优化移动电话网络的方法和系统
WO2019114947A1 (fr) Technique d'analyse de qualité de service dans un réseau de télécommunications
CN104683998A (zh) 业务质量的处理方法及装置
JP7036225B2 (ja) 推定装置、システム及び方法及びプログラム並びに学習装置及び方法及びプログラム
Garcia et al. Big data analytics for automated QoE management in mobile networks
US10778566B2 (en) Pattern discovery from high dimensional telemetry data using machine learning in a network assurance service
Yusuf-Asaju et al. Framework for modelling mobile network quality of experience through big data analytics approach
WO2023041051A1 (fr) Procédé et dispositif d'évaluation de perception d'utilisateur, support de stockage et dispositif électronique
Gokcesu et al. Qoe evaluation in adaptive streaming: Enhanced mdt with deep learning
Panek et al. Modeling configuration-performance relation in a mobile network: a data-driven approach
Muelas et al. Facing network management challenges with functional data analysis: Techniques & opportunities
Zhohov et al. One step further: Tunable and explainable throughput prediction based on large-scale commercial networks
Pejović et al. RICERCANDO: Data mining toolkit for mobile broadband measurements
US20250016050A1 (en) Computerized systems and methods for application-based qoe network configuration and implementation
US12475134B2 (en) Time disambiguation for natural language queries in network monitoring systems
Osee et al. Quality of Experience Management in Mobile Networks: Techniques, Constraints, and Emerging Trends for Value-Added Services
Fowdur et al. Machine Learning For Network Traf f ic and Video Quality Analysis
Tarrías et al. Toward Zero-Touch Cellular Networks via next-Generation Crowdsourcing
Wassermann Machine learning for network traffic monitoring and analysis: Application to internet qoe assessment and network security
Al-Shehri et al. Metrics for broadband networks in the context of the digital economies

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17822616

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17822616

Country of ref document: EP

Kind code of ref document: A1