[go: up one dir, main page]

Kuehn et al., 2014 - Google Patents

Monitoring data streams at process level in scientific big data batch clusters

Kuehn et al., 2014

Document ID
11570329646126117513
Author
Kuehn E
Fischer M
Jung C
Petzold A
Streit A
Publication year
Publication venue
2014 IEEE/ACM International Symposium on Big Data Computing

External Links

Snippet

The operation of scientific big data centres requires an overall monitoring and perception of system components. Insights into internal and external network traffic is of high importance for understanding specific data flows regarding storage accesses, firewall configurations …
Continue reading at ieeexplore.ieee.org (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for programme control, e.g. control unit
    • G06F9/06—Arrangements for programme control, e.g. control unit using stored programme, i.e. using internal store of processing equipment to receive and retain programme
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061—Partitioning or combining of resources
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/34—Recording or statistical evaluation of computer activity, e.g. of down time, of input/output operation; Recording or statistical evaluation of user activity, e.g. usability assessment
    • G06F11/3466—Performance evaluation by tracing or monitoring
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/30286—Information retrieval; Database structures therefor; File system structures therefor in structured data stores
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/30—Monitoring
    • G06F11/3065—Monitoring arrangements determined by the means or processing involved in reporting the monitored data
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F11/00—Error detection; Error correction; Monitoring
    • G06F11/07—Error detection; Error correction; Monitoring responding to the occurence of a fault, e.g. fault tolerance
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F2201/00—Indexing scheme relating to error detection, to error correction, and to monitoring
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/14—Arrangements for maintenance or administration or management of packet switching networks involving network analysis or design, e.g. simulation, network model or planning
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/50—Network service management, i.e. ensuring proper service fulfillment according to an agreement or contract between two parties, e.g. between an IT-provider and a customer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/02—Arrangements for maintenance or administration or management of packet switching networks involving integration or standardization
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/12—Arrangements for maintenance or administration or management of packet switching networks network topology discovery or management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance or administration or management of packet switching networks
    • H04L41/06—Arrangements for maintenance or administration or management of packet switching networks involving management of faults or events or alarms
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing packet switching networks
    • H04L43/10—Arrangements for monitoring or testing packet switching networks using active monitoring, e.g. heartbeat protocols, polling, ping, trace-route
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06Q—DATA PROCESSING SYSTEMS OR METHODS, SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL, SUPERVISORY OR FORECASTING PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management, e.g. organising, planning, scheduling or allocating time, human or machine resources; Enterprise planning; Organisational models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00—Network-specific arrangements or communication protocols supporting networked applications
    • H04L67/10—Network-specific arrangements or communication protocols supporting networked applications in which an application is distributed across nodes in the network

Similar Documents

Publication Publication Date Title
CN110855473B (en) Monitoring method, device, server and storage medium
US11829233B2 (en) Failure prediction in a computing system based on machine learning applied to alert data
US10558920B2 (en) Machine learning classification with confidence thresholds
US20190065738A1 (en) Detecting anomalous entities
WO2021041039A1 (en) Computational instance batching and automation orchestration based on resource usage and availability
US11394750B1 (en) System and method for generating network security policies in a distributed computation system utilizing containers
US20210365762A1 (en) Detecting behavior patterns utilizing machine learning model trained with multi-modal time series analysis of diagnostic data
US11663515B2 (en) Machine learning classification with model quality prediction
Alqahtani et al. End-to-end service level agreement specification for iot applications
Brandt et al. Network Performance Counter Monitoring and Analysis on the Cray XC Platform.
US11106525B2 (en) Systems and methods for classifying and predicting the cause of information technology incidents using machine learning
Kuehn et al. Monitoring data streams at process level in scientific big data batch clusters
Usman et al. SmartX Multi‐View Visibility Framework for unified monitoring of SDN‐enabled multisite clouds
Pan et al. Octopus: Experiences with a hybrid event-driven architecture for distributed scientific computing
Raghunandan et al. Digital twinning for microservice architectures
Yazdanov et al. Dolen: User-side multi-cloud application monitoring
US20250219894A1 (en) Determining critical logs for network applications
US12437250B2 (en) Multi-dimensional process mining and analysis
Lin Monarch: Scalable monitoring and analytics for visibility and insights in virtualized heterogeneous cloud infrastructure
Ravindhren et al. CCMA—cloud critical metric assessment framework for scientific computing
Rathore et al. Spatio-temporal summarized visualization of smartx multi-view visibility in cloud-native edge boxes
Mesiti et al. An editable live ETL system for Ambient Intelligence environments
Brim et al. Monitoring extreme-scale Lustre toolkit
Wang et al. Resolving Packets from Counters: Enabling Multi-scale Network Traffic Super Resolution via Composable Large Traffic Model
Kuehn et al. Analyzing data flows of WLCG jobs at batch job level