RU2851979C1 - Method and system for optimising cold start infrastructure for transport layer applications in serverless computing mode - Google Patents

Abstract

FIELD: computing.

SUBSTANCE: invention relates to means for implementing serverless computing (SLC). The claimed result is achieved by implementing the steps of: activating an eBPF agent in a Kubernetes environment for a virtual machine (VM) to prepare a transmission control protocol; configuring virtual interfaces and protocols in the Kubernetes environment for an application and assigning it an external IP address and port; intercepting an incoming network packet, using the eBPF agent, to launch a first serverless function (SLF) and identifying the requested SLF by the key-value pair protocol+IP address - port; routing requests for the prepared protocol and until the first launch of the SLF; launching in response to the received request the first SLF on the VM, using the prepared first protocol; if a request is received via a second protocol, launching in response to the received request the second SLF on the VM, using the prepared second protocol; and routing requests according to the protocols and ports corresponding to the SLFs; reading metrics from the network flow of the server application by means of the eBPF agent; and transmitting the metrics to the infrastructure event-driven scaling system (KEDA), which makes a decision on scaling the server application.

EFFECT: optimisation of cold start and expansion of the list of supported protocols.

8 cl, 8 dwg

Description

The invention relates to the field of computer engineering and information technology, in particular, to means for implementing serverless computing.

State of the art

Serverless computing was first introduced to a wider audience in 2014, pioneered by Amazon with its AWS Lambda platform. The breakthrough idea behind this platform is the concept of serverless computing, which only charges for the resources actually used, and when an application is idle (no client requests), the resources are freed up and not billed.

The serverless computing launch mode has been implemented in the form of platforms by popular cloud computing providers - Google Cloud Platform (Cloud Run), Microsoft Azure (Container Apps), Huawei Cloud (FunctionGraph), Yandex Cloud (Serverless Containers), Cloud.ru (Container Apps) and others.

Existing platforms for running serverless computing assume that the server application uses the HTTP protocol and its extensions, such as HTTPS, GRPC, WebSocket, and others. This limits the range of applications that can be published on such platforms. Essentially, the only type of such application is a web server. A web server is far from the only type of client-server application; there are also a number of applications that implement their own L7 protocol for client-server interactions—for example, database management systems such as PostgreSQL and MySQL, data caching systems (Redis, Memcached), data transfer systems (FTP, SCP), and applications for multiplayer online games (Minecraft and others). Game servers (for online gaming) also use proprietary protocols that no one will modify for serverless computing/processing.

An example of a specialized service that partially addresses the limitation of using only the HTTP protocol in serverless cloud computing services is Neon Serverless PostgreSQL, a modern solution for running PostgreSQL in serverless mode, allowing the use of a relational database without traditional server resource provisioning. When a database query is launched, the system automatically allocates short-lived compute nodes (instances) that access data stored in a central, scalable storage. These compute nodes scale up or down depending on the load and can be quickly launched and terminated, allowing you to pay only for the time actually used. Thus, Neon's architecture enables high scalability and flexibility: persistent storage ensures reliability and long-term data retention, and the compute layer automatically adapts to changing workloads, making the solution truly serverless.

However, the problem remains unresolved for other types of L7 systems and protocols in general. Thus, there is no unified solution that expands both the range of protocols used in serverless computing and the range of products.

Another example of a specialized service implementation is Knative, another of the most popular serverless computing solutions used at Cloud.ru, for example, in Evolution App Services. However, like Neon Serverless PostgreSQL, it is limited to the HTTP protocol and has several drawbacks. It only supports HTTP traffic, as it was originally designed for web services and APIs, so it doesn't support other L7 protocols (DNS, PostgreSQL Wire Protocol, MQTT, etc.) without additional applications, custom adapters, or sidecar proxies, which complicates the overall architecture. Furthermore, this solution has a number of disadvantages:

- lack of built-in support for long-running processes (Knative is optimized for short-term HTTP requests, while tasks with long execution times (for example, queue processing or streaming) require additional solutions (for example, Kombinator in Cloud.ru));

- limited event model (Knative Eventing provides mechanisms for triggers, integration with non-HTTP event sources (e.g. Kafka, RabbitMQ), but requires additional components (Brokers, Triggers), which increases overhead).

Knative also contains a special component (activator) that accepts incoming connections and requests to the user load, and temporarily buffers requests until the user load is scaled from zero to the required number of active containers (scale from zero process).

This approach allows for the implementation of a scaling algorithm from scratch, but requires increased resources for its execution, as there are overhead costs associated with running such an activator service—increased consumption of CPU, RAM, network, file descriptors, etc.

Also known is patent application US 2022/0156097 A1, published November 24, 2022, which discloses a method and system for managing serverless functions in a cloud environment. Before launching a function, a virtual machine creates a TCP socket (socket activation). After socket activation, upon receiving a request from a client, the specified function is launched using the already created socket, reducing latency and improving efficiency. The system also collects and analyzes virtual machine (VM) performance metrics, enabling the selection of optimal resources for executing functions. Thus, this method and system are aimed at solving the performance problem when working with a virtualized infrastructure by reducing connection latency.

Overall, an analysis of existing solutions shows that they utilize proxying and buffering components in one form or another, which leads to increased consumption of computing resources. These components, such as gateway activators, ensure hot startup and scaling of serverless application instances, necessitating the availability of active, hot application instances and components that proxy to these hot instances upon incoming connections, ready to accept incoming requests 24/7.

Our proposed solution differs in that, instead of creating sockets through a VM, it uses a system of eBPF agents that intercept network packets before they reach the operating system (OS) network layer. This reduces platform operating overhead and optimizes cold starts. It also expands the list of supported protocols, enabling the simultaneous use of previously unavailable serverless software solutions and the launch of any TCP/UDP/SCTP applications, not just HTTP, thereby completely eliminating the problem of protocol unification in a serverless infrastructure. This ensures the efficient operation of complex multi-component systems in a system layer that provides versatility, based on our serverless platform, while optimizing computing resources.

In our invention, a cold start refers to the process of scaling an application without the presence of active "hot" instances upon incoming connections. Existing HTTP-based solutions utilize various proxy components that are ready to accept incoming requests 24/7. In our case, these components are absent, and this allows for infrastructure optimization, eliminating the need to allocate resources for the proxy component's operation, such as CPU/RAM, etc.

Patent CN 116016644 A, published April 25, 2023, discloses a method for processing service requests in a serverless architecture. Upon receiving request information from the API gateway, the serverless application management service determines the application's targets and the corresponding connection information to the BaaS (Backend as a Service) service. The system then creates a virtual instance of the application and launches a BaaS proxy component, which facilitates network communication between the running application and the target service. Additionally, the system can use information from the application's storage to refine the configuration and, if the standard proxy component is unavailable, create a connection pool to ensure reliable access. Thus, this method enables dynamic and secure execution of service tasks in an isolated serverless environment using BaaS. The method also aims to solve the problem of accelerating the processing of the first requests to the serverless computing infrastructure; to eliminate delays, a proxy balancer is used in a virtualized environment based on information about the target serverless computing application in the request.

The proposed invention is distinguished by its use of an eBPF agent system instead of a BaaS proxy. This solution addresses a fundamentally different problem: expanding the range of supported protocols in serverless computing, enabling the simultaneous use of previously unavailable software solutions. This ensures the efficient operation of complex multi-component systems within a system layer that provides versatility, based on our serverless platform, while optimizing computing resources. The solution is designed to apply the serverless computing model to databases (Database as a Service/DBaaS). Consequently, the protocol set is limited to popular databases, while our solution is universal in the types of protocols it supports.

Patent application US 2019/0377604 A1, published December 12, 2019, describes a system for delivering Function as a Service (FaaS) in the cloud. The system consists of a master node, multiple worker nodes, and operational nodes. The system operates by dynamically deploying containers (pods) running serverless functions, managed by a master node that schedules container launches, balances the load, and scales them based on current demand. The system also supports the migration of serverless computing functions between different FaaS platforms, enabling the transfer of code, configurations, and load data for optimal resource allocation across a distributed cloud computing infrastructure. This system thus addresses the problem of horizontally scaling FaaS computing resources in a serverless computing platform, taking into account dynamically changing loads.

The proposed invention is distinguished by its solution to a fundamentally different problem: expanding the range of supported protocols in serverless computing, enabling the simultaneous use of various previously unusable software solutions. It also facilitates scaling of serverless applications without the need for active "hot" instances and associated proxy and buffering components that rely on incoming connections. In our case, these components are eliminated, thereby optimizing the infrastructure and eliminating the need to allocate additional computing resources to proxy components. This ensures the efficient operation of complex multi-component systems in a system layer that provides versatility, based on our serverless platform, while optimizing computing resources.

Patent application EP 4432094 A1, published September 18, 2024, discloses a method for automatically analyzing the structure and flows of an application in the cloud. The method involves collecting serverless computing activity logs and constructing an event graph in which nodes represent individual entities (functions, resources, users), and edges record interactions between them. The system then applies community detection algorithms and centrality measures to identify application flows and determine their intersections, enabling the reconstruction of the application's logic and analysis of its structure. Thus, this method aims to address the problem of security monitoring in a serverless computing cloud environment by using static analysis (intersection detection) of function call graphs (object-event) from activity log data.

The proposed invention is distinguished by its solution to a fundamentally different problem: expanding the range of supported protocols in serverless computing, enabling the simultaneous use of various previously unusable software solutions. It also facilitates scaling of serverless applications without the need for active "hot" instances and associated proxy and buffering components that rely on incoming connections. In our case, these components are eliminated, thereby optimizing the infrastructure and eliminating the need to allocate additional computing resources to proxy components. This ensures the efficient operation of complex multi-component systems in a system layer that provides versatility, based on our serverless platform, while optimizing computing resources.

Thus, in order to implement a new method and obtain a new technical result, in comparison with the previous level of technology, it is necessary to implement a number of tasks:

- ensure maximum and unlimited expansion of the list of supported protocols, and, due to this, the universality of publishing the application "as is", without the need to modify its architecture;

- ensure processing of requests that take an unusually long time to execute compared to the typical load (long-running requests) in a serverless environment;

- ensure adaptive scaling of the infrastructure at the L4 level (TCP/UDP/SCTP) to the load on the cloud platform;

- ensure routing of protocols to target components of the published application;

- eliminate proxying and buffering solutions and "hot" application instances, which allows for optimization of cold start infrastructure and reduction of operating costs for the serverless computing platform.

The objective of the proposed invention is to expand the list of supported protocols in the context of serverless computing, as well as to completely abandon the buffering of incoming requests using a proxy and/or caching gateway-activator and to perform scaling from scratch based on the fact of receiving a connection request, while ensuring the stable operation of existing application protocols - without interruptions, connection errors, timeouts, etc.

The technical result of the proposed invention is to enable the execution of applications in serverless computing mode without the limitation of only the HTTP protocol, removing restrictions on the support of long-running requests, as well as reducing the requirements for published applications in terms of cloud deployment, and eliminating proxying and buffering solutions, which allows for the optimization of the cold start infrastructure.

The technical result is ensured by the serverless technology itself—the ability to save on resources, since serverless computing mode is used: resources are used only when there is an actual user request.

The specified technical result is achieved by implementing a method for executing client-server applications of the transport layer in a serverless computing mode, comprising the steps of:

activate an eBPF agent that performs real-time read and write functions to and from the kernel to pre-stage the transmission control protocol before the virtual machine runs any serverless function associated with the pre-staged protocol;

configure virtual interfaces and protocols in the Kubernetes environment for the published application and assign it an external IP address and port;

intercept the incoming network packet using the eBPF agent to launch the first serverless function;

using the eBPF agent, identify the requested serverless function using a key-value pair (IP address - port);

In response to the received request, launch the first serverless function on the virtual machine using the pre-prepared first protocol;

if a request is received via the second protocol, a second serverless function is launched in response to the received request on the virtual machine using the previously prepared second protocol;

route requests according to protocols and ports to the corresponding serverless functions,

read metrics from the network flow of the client-server application using the eBPF agent; and

transmit metrics to the KEDA system, which makes decisions about scaling the client-server application.

An additional feature is that it initializes the eBPF agent with a routing table for published client-server applications.

An additional feature is that the routing table is updated when the states of the client-server applications change, in particular when scaling to zero and/or removing the client-server application from the system.

An additional feature is that the metrics define, in particular, the number of active connections and/or the number of packets transmitted per second.

An additional feature is that the requested serverless function is identified in the eBPF agent: by the incoming protocol and/or IP address and/or destination port.

An additional feature is that the protocols used are TCP/IP and/or SCTP and/or UDP.

An additional feature is that as a result of determining the set of (first and/or second) protocols, routing and transmission order, data packets are sent to the first and second serverless functions being launched, using the first protocol, for example, TCP/IP or SCTP or UDP, and the second protocol, for example, TCP/IP or SCTP or UDP, respectively.

An additional feature is that the method for executing client-server applications of the transport layer in serverless computing mode is implemented without the use of proxying and buffering components.

A system for executing client-server applications of the transport layer in the serverless computing mode is proposed, implementing the mentioned method, wherein the system contains: at least one processor, at least one memory, input/output interfaces, a network interaction means, which implement the following functionally interconnected means: at least one virtual machine with a deployed Kubernetes environment; a Kubernetes system for managing external - public IP addresses Cilium (IPAM); an infrastructure event scaling system (KEDA); at least one serverless application deployed in Kubernetes; at least one eBPF agent on each of at least one virtual machine within the Kubernetes environment.

The conducted analysis of the state of the art allows us to determine that the proposed technical solution is new and has an inventive step, and the possibility of its use in industry determines it as industrially applicable.

These and other aspects will become apparent and will be explained with reference to the drawings and embodiments described hereinafter.

The invention is explained by the following graphic materials:

Fig. 1a shows the general component diagram of serverless computing in a cloud environment using the example of Evolution App Services Cloud.ru.

Fig. 1b - shows an example of the architecture of a system for executing client-server applications of the transport layer in serverless computing mode.

Fig. 2 shows the announcement of VM IP addresses in the Kubernetes environment.

Fig. 3 - shows the session initialization process using TCP as an example.

Fig. 4 shows the algorithm for accepting or ignoring a packet at the eBPF agent level (kernel level).

Fig. 5 - shows the algorithm for repeating the initialization of a TCP session (TCP SYN retransmission).

Fig. 6 - shows the algorithm for scaling an application based on the TCP protocol.

Fig. 7 - shows the algorithm for scaling an application based on the UDP protocol.

Fig. 8 shows the stages of request processing with infrastructure scaling (for the growth and decrease in the number of requests).

Implementation of the invention

Let us introduce a number of definitions and concepts that will be used in describing the claimed invention.

A user application (UA) is a program that runs during serverless computing.

A client-server application is a network application architecture in which there is a client, such as a physical user of a service and his or her personal computer, and a server located in a remote environment, such as a data center.

OSI network model - An ISO standard that describes the layers of network protocols.

The transport layer is the fourth layer of the OSI network model (OSI/L4), designed for data delivery. Higher-level protocols, such as HTTP or DNS, are implemented on the transport layer.

The application network layer is the 7th layer of the OSI network model (OSI/L7), and implements a protocol for interaction between a specific client-server application.

TCP (TCP/IP) is a protocol used for transmitting data on the internet. A key feature of the protocol is its SYN/ACK connection-enforcing mechanisms.

UDP (UDP/IP) is a protocol used for data transfer on the internet. It differs from TCP/IP in that it offers increased performance due to the lack of connection monitoring mechanisms.

HTTP is an application-level protocol used for client-server interactions within the Internet and its extensions, such as HTTPS, GRPC, Websocket, and others.

A web server is a server application that uses the HTTP protocol to interact with a client application.

DNS server is a server application that converts a domain name into an IP address.

Serverless computing is a method of program execution in which computing power is provided based on the actual use of the application, rather than in a permanently allocated (paid) mode.

Serverless computing platform (PaaS) is a software package that allows you to run applications in serverless computing mode.

Docker image is an open format for distributing applications.

Kubernetes (k8s) is an open-source software platform for orchestrating containerized applications, automating their deployment, scaling, and coordination within a cluster. It supports major containerization technologies, including Docker and rkt, and can also support hardware virtualization technologies. Kubernetes assembles one or more computers (VMs) into a cluster that can run containerized workloads. It works with various container runtime environments, such as containerd and CRI-O. Kubernetes defines a set of building blocks ("primitives") that, when combined, provide mechanisms for deploying, maintaining, and scaling applications based on CPU, memory, or user-defined metrics. Its suitability for running and managing workloads of all sizes and styles has led to its widespread adoption in clouds and data centers.

The Kubernetes Event-Driven Autoscaler (KEDA) is an open-source solution for reactive application scaling. KEDA responds to events and increases or decreases the number of active application instances. KEDA can be added to any Kubernetes cluster. KEDA can expand functionality without rewriting or duplicating. With KEDA, applications requiring event-driven scaling can be isolated while other applications continue to function, making KEDA a flexible and secure option for running alongside any number of other Kubernetes applications or frameworks.

The Cilium Network Plugin (CNI) is an open-source solution for implementing networking within a Kubernetes cluster. It is used to advertise a dedicated IP address for a published application. Cilium allows flexible configuration of networking restrictions within the cluster using network policies.

eBPF (Extended Berkeley Packet Filter) is a Linux kernel technology that allows specialized routines to process network packets to be run at the OS kernel level or, in some cases, at the network device (network card) level.

An eBPF agent is a software application consisting of two subroutines: a user subroutine running in user space and a kernel subroutine. Using eBPF technology, two-way communication can be implemented between these subroutines, allowing for reading and writing data to and from the kernel.

XDP (eXpress Data Path) is the earliest point in the path of any network packet passing through an eBPF program. It allows the connection of an eBPF program that will perform control (PASS, DROP, REJECT) of the incoming network packet. If at this point the packet is marked PASS, i.e., permitted for further processing, it is then passed by the Linux kernel to other subsystems—the network stack, the application-level stack, etc.

The proposed method and system for routing requests in serverless environments implements direct interception of network packets at the transport layer, bypassing standard processing by the OS network stack and without the use of proxy components, socket pre-initialization, or intermediate transformations. The technology provides support for multiprotocol communications such as TCP/IP, UDP, and SCTP, reducing resource costs by eliminating redundant infrastructure and minimizing latency through optimized data delivery to the target function without unnecessary network hops.

Figure 1a shows a general component diagram of serverless computing in a cloud environment, using Evolution App Services at Cloud.ru as an example. It is implemented as a public cloud based on open-source components with IaaS and PaaS services. The cloud is used by those who need to rent computing power and storage, for example, to launch a home project, test a new development, or build a cloud-native application.

Control Plane 1 contains the following participating components: Control API 10, Common Public API 20, User Layer API 30, Core API 40, Kubernetes Control Layer 50, Monitoring API 60, Usage API for viewing usage and costs 70, Kafka Message Broker 80, Database 90.

Data Plane 2 contains the following participating components: Kubernetes runtime 100, serverless client application 110, data and event sources 120.

Cloud service 3 contains the following participating components: IAM functions 130, MonAAS 140, LogAAS 150, Billing 160, Product Catalog 170, Event Bus 180.

User space 4 contains the following participating components: container register 190, user event bus 200, API gateway 210.

Fig. 1b shows an example of the architecture of a system for executing client-server applications of the transport layer in serverless computing mode.

Each VM with a deployed Kubernetes environment and/or KEDA infrastructure event scaling system, and/or Cilium's IPAM (Kubernetes system for managing external public IP addresses), and/or eBPF agent can be implemented as a computing host node 201, which facilitates the implementation of the claimed method, for example, as a storage module/node, or as part of a computing cluster processing the necessary data to support the cloud environment. Host 201 is a device, such as a general-purpose computer system or server, that hosts, for example, a client and Kubernetes, under whose control the VMs are executed.

In general, the host system 201 comprises the following components: one or more processors 101, at least one memory 105, data storage means 103, input/output interfaces 102, input/output means 106, and network interaction means 107, which are connected via a universal bus 104. The processor 101 performs the basic computing operations required for processing data during the method implementation. The processor 101 executes the necessary machine-readable instructions contained in the random access memory 105. The memory 105 is typically implemented in the form of RAM and contains the necessary program logic to provide the required functionality. The data storage means 103 can be implemented in the form of HDD, SSD disks, RAID array, flash memory, optical storage devices (CD, DVD, Blue-Ray disks). The 103 tools enable long-term storage of various types of information, such as data blocks, data structures, metadata, and test scripts. Various types of input/output interfaces 102 are used to organize the operation of system components and the operation of externally connected devices. The selection of the appropriate interfaces depends on the specific design of the computing devices, which may include, but are not limited to: PCI, AGP, PS/2, IrDA, FireWire, LPT, COM, SATA, Lightning, USB, TRS/Audio jack, HDMI, DVI, VGA, Display Port, RJ45, and RS232. The selection of interfaces 102 depends on the specific design of the system, which may be implemented on a wide range of devices, such as a personal computer, a server cluster, or a server. The following may be used as input/output data means 106: keyboard, joystick, display (touch display), monitor, touch display, touchpad, mouse, light pen, touch panel, speakers, microphone, augmented reality means, optical sensors, tablet, projector, camera, biometric identification means.

Network interaction means 107 are selected from devices that provide network data reception and transmission, such as an Ethernet card, a WLAN/Wi-Fi module, a Bluetooth module, a BLE module, an NFC module, an IrDA module, an RFID module, or a GSM modem. Means 106 facilitate data exchange between, for example, a system represented as a server and a user's computing device, which can display received data via a wired or wireless data transmission channel, such as a WAN, PAN, LAN, Intranet, Internet, WLAN, WMAN, or GSM.

In general, a network request routing system in a serverless environment consists of a set of components, including, for example, hosts—physical or VMs with an OS installed—with a Kubernetes environment deployed on them and instances of serverless applications running in this environment, and a software switch that connects all VMs to Kubernetes into a single system. A unique feature of the system is that the Kubernetes network layer includes the Cilium plugin, an example of which is described in the article https://habr.com/ru/companies/kts/articles/825136/.

Cilium allows flexible configuration of network communication restrictions within the cluster using network policies. Their implementation in Cilium is similar to standard Kubernetes network policies: by default, a pod has no restrictions on ingress and egress traffic, but if it matches the selector of at least one policy, it switches to "default deny" mode. This blocks all traffic except that permitted by the policies. Response traffic is implicitly allowed. Cilium also offers an "always" mode, in which the "default deny" rule applies to all pods regardless of whether they match the network policy selector. In this mode, network policies are disabled, and traffic within the cluster is transmitted without restrictions.

The Kubernetes IPAM system, Cilium, for managing external (public) IP addresses includes the Cilium toolkit, a plugin that replaces standard network components (kube-proxy, iptables) by integrating eBPF programs into the OS kernel. A Cilium agent on each host interacts with the Kubernetes API to synchronize network policies and endpoint state. Information about pods and services is cached in local memory. Configuration state (security policies, pod IP addresses, and internal request routing tables) can be stored, for example, in etcd or CiliumNetworkPolicy. Traffic processing is accomplished by loading eBPF programs into the OS kernel, which process traffic between Kubernetes pods directly, bypassing the network stack. The proposed solution implements a component (eBPF agent) that intercepts incoming network packets before the OS network stack rejects the connection.

Data Plane with Kubernetes shell.

1. Initialization of the routing table.

In the proposed invention, a virtual machine within the Kubernetes environment, using the Cilium network plugin, which is part of the Kubernetes network layer, informs the upstream network equipment, including the servicing software switch, that it is ready to accept requests on one of the assigned public IP addresses or subnets of IP addresses, using the L2 ARP protocol, as shown in Fig. 2.

Fig. 2 shows the stages of announcing IP addresses of VMs in the Kubernetes environment, where each VM, for example, 220 with the address 100.72.0.1 and 230 with the address, for example, 100.72.0.2, bind to a router (network device, router) 210. User 200 makes a request through router 210 and selects the required VM 220 with the address 100.72.0.1 and then transmits packets from user 200 via router 210 to VM 220 with the address 100.72.0.1.

No socket creation occurs at the VM level where the serverless application instance is deployed, nor at the network device level, eliminating the need for additional resources such as file descriptors and RAM. Furthermore, each hosted server application is assigned a unique combination of server protocol, IP address, and destination port at the eBPF agent level.

Based on the combination of protocol, IP address and destination port, the system builds an internal request routing table, for example:

Protocol IP address Port of destination Application Application status TCP 100.72.0.1 5432 Appendix A There is an active copy UDP 100.72.0.1 53 Appendix B There are no active instances.

The routing table is then replicated, via the Kubernetes API, to all VMs within the Kubernetes environment for subsequent use by the eBPF agent when initializing a connection.

When the application state changes (scaling to zero - when the user load is not scaled to the required number of active containers, deleting the application from the system, etc.), the routing table is updated.

2. Intercepting the incoming connection and applying the routing table.

Using the TCP protocol as an example, if there are no active instances of a serverless application, the VM OS network stack will reject the incoming connection with the Connection Refused error as shown in Fig. 3, the "negative scenario" block.

Fig. 3 shows the session initialization process using TCP as an example, where in a positive scenario, user application (serverless application instance) 340 (UA) initializes (allocates) a socket via OS network stack 330. User 300 initializes a connection (TCP SYN) via OS network stack 330, which checks for the existence of the requested port and sends a positive response (TCP SYN + ACK) to user 300, since the port was allocated earlier and the system can accept the request. User 300 confirms the connection (TCP ACK) to OS network stack 330. Packets are exchanged within the TCP session between user 300 and UA 340. In a negative scenario, user 300 initializes a connection (TCP SYN) via OS network stack 330, which checks for the existence of the requested port and sends a connection rejection (TCP RST + ACK) to user 300, since The port has not been allocated previously, the system cannot accept the request.

If the user, having sent a TCP SYN packet when attempting to establish a connection, has not received either a positive or negative response from the server, for example, due to problems at the network level, then after a timeout, the standard mechanism for retrying to initialize the TCP session is activated, as shown in Fig. 5, and see https://datatracker.ietf.org/doc/html/rfc793#section-3.4

Fig. 5 shows the algorithm for repeating the initialization of a TCP session (TCP SYN retransmission), where the user 500 transmits a TCP SYN request to the network device (router) 510 to attempt to transmit a packet to the network device server (NIC) 520. The packet was not delivered to the requested address, for example, due to problems at the network infrastructure level and the TCP ASK was not returned. The TCP SYN retransmission cycle is repeated from one time to a specified number of times. Waiting for a response from the network device server (NIC) 520. The user 500 retransmits the TCP SYN request to the network device 510 to attempt to transmit a packet to the network device server (NIC) 520. Repeated attempt to initialize the session.

In the case of the UDP protocol, since, unlike the TCP protocol, the UDP protocol does not guarantee the delivery of messages to the requested server (due to this, increased throughput is achieved), UDP-based protocols often implement a message relay mechanism, for example, one of the most popular UDP-based protocols - DNS, uses a mechanism for repeating requests after a timeout, see https://datatracker.ietf.org/doc/html/rfc1035#section-4.2.1

In the "positive scenario," when at least one instance of the serverless application is already running, the application initiates (allocates) an open socket on the OS network stack. Next, the eBPF agent, upon intercepting an incoming packet, extracts the protocol (TCP), the requested IP address, and the destination port from the incoming request, checks the state of the server application from the replicated routing table, and takes further action according to the algorithm, depending on the application state, as shown in Figure 4.

Figure 4 shows the algorithm for accepting or ignoring a packet at the eBPF agent level (kernel level), where at step 400 the incoming packet is received by the network device, and at step 410 the incoming packet is intercepted by the eBPF agent. At step 420 the protocol, requested IP address, and destination port are extracted from the incoming packet. At step 430 the application state is queried in the eBPF map and the packet is passed into the network stack (step 440) if the application is active, or the packet is dropped (step 450) if the application is inactive.

3. Algorithm for scaling a serverless application (from zero active instances).

TCP. An example of interception of an incoming message by an eBPF agent for the TCP protocol is shown in Fig. 6.

In general, if there is no active serverless application instance, the packet is ignored and the eBPF agent sends a connection open event request to the KEDA system. During a subsequent attempt to initialize a TCP session, the eBPF agent checks the application state from the routing table and, if the application is active (the application has an open socket on the network stack) and is ready to accept incoming requests, it forwards the packets to the requested application unchanged.

Figure 6 shows an algorithm for scaling an application based on the TCP protocol, where Kubernetes, via API, through KEDA 620, launches the eBPF agent (user space) 640 at system startup, synchronizes IP:PORT+STATUS pairs for the PP 630, and transmits a message to eBPF (user space) 640. The IP:PORT+STATUS pairs are recorded in the eBPF map and are available for reading/writing in the agent components. eBPF (user space) 640 loads the eBPF program into the eBPF kernel (kernel) 650, which binds the intercepting logic to the network interface (with a public IP) 610.

User 600 sends a TCP SYN packet (attempt N1) to network interface (with public IP) 610.

The operating algorithm for an active serverless application (SA) instance is as follows: network interface (with public IP) 610 transmits an XDP event with a TCP SYN packet to eBPF (kernel) 650, which checks the availability of SA 630 in eBPF (user space) 640. If the response is negative, eBPF (kernel) 650 transmits a "drop" message to network interface (with public IP) 610. The system does not respond to the TCP SYN request because there is currently no available container, and the user will receive a response to the SYN request in a subsequent request, as part of the TCP SYN retransmission mechanism. eBPF (kernel) 650 transmits an event about an attempt to initialize a TCP session via eBPF (user space) 640, which in turn transmits an event for scaling in Kubernetes via the API through KEDA 620. Kubernetes API/KEDA 620 then transmits a message - launch and wait for availability on PP 630.

User 600 transmits a TCP SYN packet (attempt N2) to network interface (with public IP) 610. Since no response was received to the previous connection attempt, the standard TCP SYN RETRY mechanism is used. Network interface (with public IP) 610 transmits an XDP event with a TCP SYN packet to eBPF (kernel) 650, which checks the availability of PP 630 in eBPF (user space) 640. If the response is positive, eBPF (kernel) 650 transmits a message to network interface (with public IP) 610 about passing the packet further, to the OS network stack (pass). Network interface (with public IP) 610 transmits a connection acknowledgment packet to user 600 - a standard TCP packet with the SYN+ACK flags. User 600 exchanges packets within the established TCP session with PP 630.

SCTP. The protocol has the same connection initialization algorithm as TCP, so the scaling algorithm is the same as for TCP.

UDP. An example of interception of an incoming eBPF message by an agent for the UDP protocol is shown in Fig. 7.

In general, in the UDP protocol, the eBPF agent also ignores network messages. If there is no active application instance, the KEDA system is notified, and the application is expected to launch and become available. Subsequent messages sent via the retry delivery mechanism are passed on by the eBPF agent to the requested application. This solves the problem of correctly establishing a connection in the absence of an active application instance, without changing the specific application protocol, eliminating the need for an intermediate component such as a proxy and/or caching gateway-activator.

Figure 7 shows an application scaling algorithm based on the UDP protocol, where Kubernetes, via API through KEDA 720, launches eBPF agent (user space) 740 at system startup, synchronizes IP:PORT+STATUS pairs for PP 730, and transmits a message to eBPF (user space) 740. IP:PORT+STATUS pairs are written to the eBPF map and are available for reading/writing in agent components. eBPF (user space) 740 loads the eBPF program into eBPF kernel (kernel) 750, which binds the intercepting logic to network interface (with public IP) 710.

User 700 transmits a UDP packet (attempt N1) to network interface (with public IP) 710. The operating algorithm for an active serverless application (SA) instance is as follows: network interface (with public IP) 710 transmits an XDP event with a UDP packet to eBPF (kernel) 750, which checks the availability of SA 730 in eBPF (user space) 740. If the response is negative, eBPF (kernel) 750 transmits a "drop" message to network interface (with public IP) 710. The system does not respond to the incoming UDP request because there is currently no available container, and the user will receive a response to the UDP request in a subsequent request, as part of the retry mechanism. eBPF (kernel) 750 transmits an event about receiving a UDP packet via eBPF (user space) 740, which in turn transmits an event for scaling to Kubernetes via the API through KEDA 720. Kubernetes, via the API through KEDA 720, then transmits a message - launch and wait for availability on PP 730.

User 700 transmits a UDP packet (attempt N2) to network interface (with public IP) 710, since no response was received to the previous connection attempt, the standard retry mechanism is used. Network interface (with public IP) 710 transmits an XDP event with a UDP packet to eBPF (kernel) 750, which checks the availability of PP 730 in eBPF (user space) 740. If the response is positive, eBPF (kernel) 750 transmits a message to network interface (with public IP) 710 to pass the packet further into the OS network stack (pass). Network interface (with public IP) 710 forwards the packet to the requested container for PP 730. The user exchanges packets with PP 730 using the UDP protocol.

4. Algorithm for scaling a serverless application to accommodate increases and decreases in load (number of requests).

Fig. 8 shows a block diagram of request processing with infrastructure scaling (increasing and decreasing the number of requests).

During eBPF execution, the agent reads metrics from the serverless application's network stream, such as:

1. Number of active connections.

2. The number of packets transmitted per second.

These metrics are transmitted to the KEDA system, which makes decisions about scaling the application.

If there are no active connections, metrics synchronization occurs. KEDA 820, in conjunction with eBPF agent 810, establishes a connection to the metrics service. In response, eBPF agent 810 transmits metrics to KEDA in streaming mode. The number of connections at this point is zero, and no action is required.

Next, the number of connections increases from zero to one. At this point, metrics are synchronized, and KEDA 820, in conjunction with eBPF agent 810, establishes a connection to the metrics service. In response, eBPF agent 810 streams metrics to KEDA. User 800 opens a connection to the VM and eBPF agent 810 running on it, which informs KEDA 820 that the number of connections has increased to one. KEDA 820 sends a command to Kubernetes system 830 to scale the serverless application from zero to one instance. A health check then occurs for the application. eBPF agent 810 approves a further connection attempt with server application 840, after which a two-way TCP stream is transmitted between user 800 and server application 840.

User 800 closes the connection to the VM and the eBPF agent 810 running on it. In doing so, eBPF agent 810 sends a message to KEDA 820 informing it that the number of connections has become zero. KEDA 820 sends a request to Kubernetes system 830 to scale the application to zero instances and sends a message to server application 840 to stop the application, after which the application is stopped.

The method and system of the present invention may take the form of at least program code (instructions, software applications) embodied on tangible media such as CD-ROM, hard drives, RAM, or any other computer-readable storage medium. The instructions may be provided in the form of software or firmware and/or may be implemented in whole or in part in hardware components such as ASICs, FPGAs, DSPs, or other similar devices. The instructions may be configured for execution by one or more processors, which, when executing a series of computer instructions, perform or facilitate the execution of all or part of the disclosed methods and procedures. When the instructions are loaded into a computer and executed, it becomes a device for practical implementation of the invention. Software instructions recorded on a computer-readable medium may be configured for the present invention or may be known and used by a person skilled in the art in computer software. Examples of software instructions may include: machine language codes, such as those generated by a compiler, as well as high-level language codes that are executed by a computer using an interpreter and the like.

The submitted application materials disclose preferred examples of the implementation of the technical solution and should not be interpreted as limiting other, particular examples of its implementation that do not go beyond the scope of the requested legal protection, which are obvious to specialists in this field of technology.

Claims (17)

1. A method for executing client-server applications of the transport layer in serverless computing mode, implementing the steps of: perform eBPF agent activation to pre-stage the transmission control protocol before the virtual machine starts the serverless function associated with the pre-staged protocol; configure virtual interfaces and protocols in the Kubernetes environment for the application and assign it an external IP address and port; intercept the incoming network packet using the eBPF agent to launch the first serverless function; using the eBPF agent, identify the requested serverless function using the protocol+IP address-port key-value pair; In response to the received request, launch the first serverless function on the virtual machine using the pre-prepared first protocol; if a request is received via the second protocol, a second serverless function is launched in response to the received request on the virtual machine using the previously prepared second protocol; route requests according to protocols and ports to the corresponding serverless functions; read metrics from the network flow of client-server applications using the eBPF agent; and transmit metrics to the infrastructure event-driven scaling system (KEDA), which makes decisions about scaling client-server applications. 2. The method according to paragraph 1, characterized in that the eBPF agent is initialized with a routing table for client-server applications. 3. The method according to paragraph 2, characterized in that the routing table is updated when the states of the client-server application change, in particular, such as scaling to zero and/or deleting the client-server application. 4. The method according to paragraph 1, characterized in that the metrics determine, in particular, the number of active connections and/or the number of transmitted packets per second. 5. The method according to paragraph 1, characterized in that the requested serverless function is identified in the eBPF agent by the incoming protocol and/or IP address and/or destination port. 6. The method according to paragraph 1, characterized in that the pre-prepared protocols are TCP/IP, and/or SCTP, and/or UDP protocols. 7. The method according to paragraph 6, characterized in that as a result of determining the protocols and routing, data packets are sent to the first and second serverless functions being launched, using the first and second protocols. 8. A system for executing client-server applications of the transport layer in serverless computing mode, implementing the method according to any of paragraphs. 1-7, consisting of functionally interconnected: at least one virtual machine with a deployed Kubernetes environment; a Kubernetes system for managing external - public IP addresses Cilium (IPAM); a KEDA system; at least one serverless application deployed in Kubernetes; at least one eBPF agent on each of at least one virtual machine within the Kubernetes environment.