[go: up one dir, main page]

WO2023016415A1 - 运行容器组的节点，容器组的管理系统和方法 - Google Patents

运行容器组的节点，容器组的管理系统和方法 Download PDF

Info

Publication number
WO2023016415A1
WO2023016415A1 PCT/CN2022/110895 CN2022110895W WO2023016415A1 WO 2023016415 A1 WO2023016415 A1 WO 2023016415A1 CN 2022110895 W CN2022110895 W CN 2022110895W WO 2023016415 A1 WO2023016415 A1 WO 2023016415A1
Authority
WO
WIPO (PCT)
Prior art keywords
sidecar
container group
service
service container
node
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2022/110895
Other languages
English (en)
French (fr)
Inventor
张伟
姜宇
黄毽
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Cloud Computing Technologies Co Ltd
Original Assignee
Huawei Cloud Computing Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Cloud Computing Technologies Co Ltd filed Critical Huawei Cloud Computing Technologies Co Ltd
Priority to JP2024508076A priority Critical patent/JP7755045B2/ja
Priority to EP22855396.2A priority patent/EP4369181A4/en
Publication of WO2023016415A1 publication Critical patent/WO2023016415A1/zh
Priority to US18/435,772 priority patent/US20240250918A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/70—Admission control; Resource allocation
    • H04L47/80—Actions related to the user profile or the type of traffic
    • H04L47/803—Application aware
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0806—Configuration setting for initial configuration or provisioning, e.g. plug-and-play
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0893—Assignment of logical groups to network elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0894—Policy-based network configuration management
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/50—Network service management, e.g. ensuring proper service fulfilment according to agreements
    • H04L41/5041—Network service management, e.g. ensuring proper service fulfilment according to agreements characterised by the time relationship between creation and deployment of a service
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00—Traffic control in data switching networks
    • H04L47/10—Flow control; Congestion control
    • H04L47/20—Traffic policing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533—Hypervisors; Virtual machine monitors
    • G06F9/45558—Hypervisor-specific management and integration aspects
    • G06F2009/4557—Distribution of virtual machine instances; Migration and load balancing
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0803—Configuration setting
    • H04L41/0813—Configuration setting characterised by the conditions triggering a change of settings
    • H04L41/082—Configuration setting characterised by the conditions triggering a change of settings the condition being updates or upgrades of network functionality
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
    • H04L41/08—Configuration management of networks or network elements
    • H04L41/0895—Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L43/00—Arrangements for monitoring or testing data switching networks
    • H04L43/10—Active monitoring, e.g. heartbeat, ping or trace-route

Definitions

  • the present application relates to the field of cloud computing, and in particular to a node running a container group, a container group management system and method.
  • the service mesh technology strips the non-functional traffic-oriented service governance logic in the distributed application of the microservice architecture from the business process into the sidecar container, in a non-invasive way Provide connection, security, flow control, grayscale release, and observation capabilities between services to realize business lightweight and service governance infrastructure.
  • IP Internet protocol
  • the embodiment of the present application provides a node running a container group, a container group management system and method, which can select a sidecar for a service container group, so as to perform better traffic management on the service container group.
  • the embodiment of the present application provides a node running a container group.
  • the node runs a connection control module, a sidecar cluster, and a first service container group.
  • the sidecar cluster includes at least two sidecars, wherein the connection
  • the control module is used to receive the sidecar allocation strategy sent by the console connected to the node, select the first sidecar in the sidecar cluster according to the sidecar allocation strategy, and forward the data packet sent by the first service container group to the second sidecar.
  • a sidecar; the first sidecar is used for flow management of the data packets sent by the first service container group.
  • connection control module can select a sidecar for the first service container from at least two sidecars according to the sidecar allocation strategy sent by the console, and use the selected sidecar to
  • the traffic management of the data packets can realize the flexible management of the first business container group, can perform better traffic management for the first business container group, and ensure the high availability of the business of the first business container group.
  • the node also includes a second service container group; the connection control module is also configured to select a second sidecar in the sidecar cluster according to the sidecar allocation strategy, and send the second service container group The data packets are forwarded to the second sidecar; the second sidecar is used for traffic management on the data packets sent by the second service container group.
  • the second sidecar and the first sidecar may be the same redundant sidecar, or may be different sidecars.
  • connection control module can select a sidecar for the second service container from at least two sidecars according to the sidecar allocation strategy, and use the selected sidecar to perform traffic management on the data packets sent by the second service container group , so that flexible management of the second service container group can be realized, better traffic management can be performed on the second service container group, and high availability of services of the second service container group can be ensured.
  • the specifications of the hardware resources allocated by the first sidecar are higher than those allocated by the second sidecar, and the sidecar allocation strategy includes the first strategy, which is used to indicate that the first service container group has priority Using the first sidecar; connecting the control module, specifically for selecting the first sidecar in the sidecar cluster according to the first strategy.
  • hardware resources with different hardware resource specifications can be set, and when the sidecar allocation policy instructs the first service container group to preferentially use sidecars with high hardware resource specifications, use high hardware resources
  • the standard sidecar manages the flow of data packets sent by the first business container group, ensuring the service quality of the first business container group.
  • the node further includes a second service container group
  • the sidecar allocation strategy further includes a second strategy, which is used to indicate that the number of service objects of the first sidecar does not exceed the upper limit
  • the control module is also used to determine the number of service objects of the first sidecar, and forward the data packets sent by the second service container group to the first sidecar when the number of service objects of the first sidecar does not exceed the upper limit
  • the car the first side car is also used for simultaneous flow management of the data packets sent by the first service container group and the data packets sent by the second service container group.
  • the upper limit of the number of service objects of the sidecar can be set, and when the current number of service objects of the sidecar does not exceed the upper limit, the sidecar will continue to allocate data packet, so that the sidecar performs traffic management on the data packet, thereby improving the resource utilization rate of the sidecar while avoiding the overload of the sidecar.
  • connection control module is configured to select a third sidecar from the sidecar cluster or notify the console to create a third sidecar at the node after the first sidecar fails, and the first sidecar Another data packet sent by the service container group is forwarded to the third sidecar; the third sidecar is used for traffic management on another data packet sent by the first service container group.
  • the third sidecar can be reselected for the first service container group, and the third sidecar can continue to manage the flow of data packets sent by the first service container group , so as to ensure the high availability of the first business container group business.
  • the third sidecar is a new version with upgraded functions based on the first sidecar, or the third sidecar is a copied version of the first sidecar.
  • a sidecar with the same function as the first sidecar can be selected for the first business container group, or a sidecar with a higher level than the first sidecar can be selected , so as to continue to perform traffic management on the data packets sent by the first service container group, thereby ensuring the high availability of the first service container group.
  • the first sidecar is configured to send the data packets to the backend container group after performing traffic management on the data packets sent by the first service container group.
  • the first sidecar may send the traffic-managed data packets to the back-end container group, so as to invoke services of the back-end container group or provide services for the back-end container group.
  • the first sidecar is also used to generate a session ID and send the session ID to the first service container group and the connection control module;
  • the connection control module is used to record the connection between the session ID and the backend container group Corresponding relationship;
  • the third sidecar is used to obtain the session ID from the first business container group and determine the back-end container group in the corresponding relationship recorded by the connection control module according to the session ID.
  • another packet is sent to the backend container group.
  • connection control module can record the corresponding relationship between the session identifier generated by the first sidecar and the backend container group, and the third sidecar can send other The data packets are sent to the back-end container group after traffic management, thus avoiding the problem that different data packets sent by the same container group are sent to different back-end container groups by different sidecars.
  • the sidecar allocation strategy includes a third strategy, and the third strategy is used to indicate that the sidecar in the sidecar cluster is preferentially used when the number of service objects is 0; the connection control module is also used to determine The number of service objects of the first sidecar, if the number of service objects of the first sidecar is 0, the data packet sent by the first service container group is forwarded to the first sidecar.
  • connection control module is also used to monitor the working status of each sidecar in the sidecar cluster, and when an offline sidecar is found, send the information of the offline sidecar to the console .
  • the downlink sidecar can be fed back to the console, so that the console can update the running sidecar, and formulate a sidecar allocation strategy accordingly to realize effective management of the container group.
  • traffic management includes: traffic control, traffic security, and traffic observation.
  • the node is a virtual machine, a computer or a bare metal server.
  • the embodiment of the present application provides a container group management system, including a console and the node provided in the first aspect.
  • the embodiment of the present application provides a method for managing a container group in a node.
  • the node runs a connection control module, a sidecar cluster, and a first service container group.
  • the sidecar cluster includes at least two sidecars.
  • the method includes: the connection control module receives the sidecar allocation strategy sent by the console connected to the node, selects the first sidecar in the sidecar cluster according to the sidecar allocation strategy, and forwards the data packet sent by the first service container group to the second One side car: the first side car performs flow management on the data packets sent by the first service container group.
  • the node also runs a second service container group
  • the method further includes: the connection control module selects the second sidecar in the sidecar cluster according to the sidecar allocation strategy, and assigns the second service container group The sent data packets are forwarded to the second sidecar; the second sidecar performs traffic management on the data packets sent by the second service container group.
  • the specifications of the hardware resources allocated by the first sidecar are higher than those allocated by the second sidecar
  • the sidecar allocation strategy includes the first strategy, which is used to indicate that the first service container group has priority Using the first sidecar; selecting the first sidecar in the sidecar cluster according to the sidecar allocation strategy includes: selecting the first sidecar in the sidecar cluster according to the first strategy.
  • the node also runs a second service container group
  • the sidecar allocation strategy also includes a second strategy, which is used to indicate that the number of service objects of the first sidecar does not exceed the upper limit
  • method It also includes: the connection control module determines the number of service objects of the first sidecar, and forwards the data packets sent by the second service container group to the first sidecar when the number does not exceed the upper limit value; The data packets sent by the first service container group and the data packets sent by the second service container group are managed at the same time.
  • the method further includes: connecting the control module, selecting a third sidecar from the sidecar cluster or notifying the console to create a third sidecar at the node after the first sidecar fails, Another data packet sent by a service container group is forwarded to the third sidecar; the third sidecar performs flow management on another data packet sent by the first service container group.
  • the third sidecar is a new version with upgraded functions based on the first sidecar, or the third sidecar is a copied version of the first sidecar.
  • the method further includes: after the first sidecar performs traffic management on the data packets sent by the first service container group, sending the data packets to the backend container group.
  • the method further includes: the first sidecar generates a session ID and sends the session ID to the first service container group and the connection control module; the connection control module records the correspondence between the session ID and the backend container group ; The third sidecar obtains the session ID from the first service container group and determines the backend container group in the corresponding relationship recorded by the connection control module according to the session ID, after performing traffic management on another data packet sent by the first service container group , to send another packet to the backend container group.
  • the sidecar allocation strategy includes a third strategy, and the third strategy is used to indicate that the number of service objects of the sidecar in the sidecar cluster is preferentially used; according to the sidecar allocation strategy, the sidecar Selecting the first sidecar in the cluster, and forwarding the data packet sent by the first service container group to the first sidecar includes: the connection control module determines the number of service objects of the first sidecar, and the number of service objects of the first sidecar If the number is 0, the data packets sent by the first service container group are forwarded to the first sidecar.
  • the method further includes: the connection control module monitors the working status of each sidecar in the sidecar cluster, and when an offline sidecar is found, sends the information of the offline sidecar to the control tower.
  • traffic management includes: traffic control, traffic security, and traffic observation.
  • the embodiment of the present application provides a node running a container group, including a processor and a memory, and the processor is configured to execute instructions stored in the memory to execute the method provided in the second aspect.
  • an embodiment of the present application provides a computer-readable storage medium, including computer program instructions.
  • the computer program instructions When the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices executes the method provided in the first aspect.
  • the embodiment of the present application provides a computer program product containing instructions, and when the instructions are executed by a cluster of computer equipment, the cluster of computer equipment executes the method as provided in the first aspect.
  • the node running the container group provided by the embodiment of the present application, the container group management system and method can select a sidecar for a service container from at least two sidecars according to the sidecar allocation strategy sent by the console, and use the selected The sidecar manages the flow of data packets sent by the service container group, so as to realize the flexible management of the service container group, perform better traffic management for the service container group, and ensure the high availability of the business container group ability.
  • FIG. 1 is a schematic structural diagram of a container group management system provided in an embodiment of the present application
  • FIG. 2 is a schematic structural diagram of a node running a container group provided in an embodiment of the present application
  • FIG. 3A is a schematic diagram of a container group management framework provided by an embodiment of the present application.
  • FIG. 3B is a flow chart of a container group management method provided in an embodiment of the present application.
  • FIG. 4A is a schematic diagram of a container group management framework provided by an embodiment of the present application.
  • FIG. 4B is a flow chart of a container group management method provided in an embodiment of the present application.
  • FIG. 5A is a schematic diagram of a container group management framework provided by an embodiment of the present application.
  • FIG. 5B is a schematic diagram of a container group management framework provided by the embodiment of the present application.
  • FIG. 5C is a flow chart of a container group management method provided in the embodiment of the present application.
  • FIG. 6A is a schematic diagram of a container group management framework provided by an embodiment of the present application.
  • FIG. 6B is a schematic diagram of a container group management framework provided by the embodiment of the present application.
  • FIG. 6C is a flow chart of a container group management method provided in the embodiment of the present application.
  • FIG. 7 is a flow chart of a container group management method provided in an embodiment of the present application.
  • FIG. 8 is a schematic block diagram of a node provided by an embodiment of the present application.
  • first and second are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, a feature defined as “first” and “second” may explicitly or implicitly include one or more of these features.
  • the terms “including”, “comprising”, “having” and variations thereof mean “including but not limited to”, unless specifically stated otherwise.
  • the microservice architecture is a service-oriented architecture (service oriented architecture, SOA), which divides a complex system into multiple small services or applications. This small service or application can be called a microservice.
  • SOA service oriented architecture
  • Each microservice is responsible for implementing an independent business logic.
  • Microservices are built around business functionality and can be deployed independently. Microservices rely on each other to provide a series of functions. Microservices are easy to understand and modify, bringing flexibility in choice of language and framework.
  • Microservices can run in containers. Among them, multiple containers with high interdependence may form a container group. For example, in the K8S (kubernetes) system, multiple containers can be encapsulated into a pod, that is, the container group is a pod.
  • a container running a microservice may be called a service container.
  • a sidecar can be implemented by a container or a process. When a sidecar is implemented by a container, the sidecar can also be called a sidecar container.
  • service governance also known as SOA governance (service oriented architecture governance, SOA governance)
  • SOA governance is the process used to manage the adoption and implementation of the microservice architecture.
  • service governance can be performed by sidecars.
  • the sidecar is a container with a sidecar function, and the sidecar provides traffic service management functions for the business container.
  • pod-level sidecars are deployed. That is, each pod corresponds to a sidecar, which is used for service governance of the pod.
  • different sidecars correspond to different pods.
  • the failure of a sidecar only affects the pod corresponding to the sidecar and does not affect other pods. It can be said that this solution helps to ensure the high availability of the business.
  • this solution if the number of deployed pods is large, there are also many sidecars deployed, so that the computing resources occupied by the sidecars cannot be ignored, which may lead to problems such as large link delays.
  • node-level sidecars are deployed. That is, a node deploys a sidecar for service management of multiple pods on the node. In this solution, the failure of the sidecar will affect multiple pods on the node, which will have a great impact on the high availability of the business.
  • the embodiment of the present application provides a container group management system, as shown in Figure 1, the management system may include multiple nodes such as node 100 and node 200, and may also include a Console 300.
  • the console 300 can manage the multiple nodes, for example, can send a sidecar allocation policy to any node in the multiple nodes, so that the node can allocate a sidecar to the service container group in the node according to the sidecar allocation policy, so as to The data packets sent by the business container group perform better traffic management to achieve high availability of the business.
  • the allocation policy may be configured on the console 300 by a manager or a tenant of the data center where the node resides, thereby enabling flexible management of the container group.
  • the nodes in the management system shown in FIG. 1 involved in the embodiment of the present application are introduced as examples.
  • the node 100 may be a virtual machine (Virtual Machine, VM).
  • VM Virtual Machine
  • the node 100 may have various hardware components, such as one or more processors (such as a central processing unit (central processing unit, CPU)), one or more memories, and the like.
  • the hardware components of the node 100 can endow the node 100 with data computing and data storage capabilities.
  • node 100 may be a computer.
  • node 100 may be a bare metal server.
  • a node 100 may run a connection control module 110 , a sidecar cluster 120 and a service container group cluster 130 .
  • the sidecar cluster 120 may include at least two sidecars such as a sidecar 121 , a sidecar 122 , and a sidecar 123 .
  • a sidecar is also called a sidecar.
  • the service container group cluster 130 may include a service container group 131 .
  • the service container group 131 may include one or more containers, where each container may run an application or a microservice.
  • the connection control module 110 may receive the sidecar allocation policy A sent by the console 200 .
  • the connection control module 110 can select a sidecar for the service container group 131 in the sidecar cluster 120 according to the allocation strategy A, for example, the sidecar 121 can be selected. Afterwards, the connection control module 110 may forward the data packet sent by the service container group 131 to the sidecar 121 .
  • the sidecar 121 can perform flow management on the data packet.
  • the traffic management may specifically be traffic control.
  • the traffic control may include any one or a combination of traffic passing, traffic denying, traffic duplication, and traffic coloring.
  • traffic management may specifically be traffic security.
  • traffic security may include encrypting or decrypting data packets, and the like.
  • traffic management may specifically be traffic observation. Wherein, the traffic observation may include drawing a call chain in the console 300 and the like.
  • connection control module may be configured with a sidecar list.
  • the list of data plane proxies can include information about running sidecars.
  • the information of the sidecar may include an identifier of the sidecar (for example, a process identity (identity, ID)) and a listening port of the sidecar.
  • the listening port of the sidecar may also be called the inbound port of the sidecar.
  • the specifications of the hardware resources allocated by the sidecar 121 are higher than those allocated by the sidecar 122, and the sidecar allocation policy A includes policy A1, and the policy A1 is used to instruct the service container group 131 to use the sidecar 121 preferentially; thus , the connection control module 110 can select the sidecar 121 in the sidecar cluster 120 according to the policy A1, so that the sidecar 121 performs service governance (ie traffic management) on the data packets sent by the container group 131 .
  • service governance ie traffic management
  • the service container group cluster 130 may further include a service container group 132 .
  • the connection control module 110 can also select the sidecar 122 in the sidecar cluster 120 according to the allocation strategy A, and forward the data packet sent by the service container group 132 to the sidecar 122 . Afterwards, the sidecar 122 can perform traffic management on the data packet.
  • the same sidecar in the sidecar cluster 120 can simultaneously perform flow management on data packets sent by multiple service container groups.
  • the service container group cluster 130 may also include the service container group 132 .
  • the sidecar allocation strategy A also includes a strategy A2, which can be used to indicate that the number of service objects of the sidecar does not exceed the upper limit.
  • the service object of the sidecar refers to the business container group that accepts the traffic management service of the business container group. That is to say, if the sidecar performs traffic management on the data packets sent by the service container group, then the service container group is the service object of the sidecar.
  • the upper limit of the sidecar refers to the maximum number of service container groups that can be managed by the sidecar at the same time.
  • the connection control module can determine the number of service objects of the sidecar 121, and forward the data packet sent by the service container group 132 to the sidecar 121 under the condition that the number of service objects of the sidecar 121 does not exceed the upper limit of the sidecar 121 Car 121. Afterwards, the sidecar 121 can simultaneously perform traffic management on the data packets sent by the service container group 131 and the data packets sent by the service container group 132 .
  • a sidecar after a sidecar fails, for example, after a sidecar crashes or restarts, a sidecar can be reselected for the service object of the sidecar, so that the newly selected sidecar can continue to support the failed sidecar.
  • the service object continues with traffic management. Still taking the sidecar 121 as an example, as mentioned above, before the sidecar 121 fails, it can perform traffic management on the data packets sent by the service container group 131 .
  • the connection control module 110 may redefine the sidecar for the service container group 131 .
  • the re-determined sidecar can manage the flow of data packets sent by the service container group 131 . Exemplarily, the sidecar is re-determined for the service container group 131.
  • the connection control module 110 may reselect from the sidecar cluster 120 the sidecar that performs traffic management for the data packets of the service container group 131.
  • the sidecar cluster 120 includes a sidecar 123
  • the connection control module 110 selects the sidecar 123 from the sidecar cluster 120 , so that the sidecar 123 performs traffic management on the data packets sent by the service container group 131 .
  • the data packet sent by the service container group 131 may be sent to the sidecar 123, so that the sidecar 123 performs traffic management on the data packet.
  • the connection control module may create the sidecar 123 in the node 100 through the console 300, and then send the data packet sent by the service container group 1131 to the sidecar 123, This enables the sidecar 123 to perform traffic management on the data packet.
  • the specific process of creating the sidecar 123 in the node 100 will be specifically introduced in Embodiment 1 below, and will not be repeated here.
  • the sidecar 123 can be a new version of the sidecar whose functions are upgraded on the basis of the sidecar 121 .
  • the version upgrade process of the sidecar will be introduced in detail below, and will not be repeated here.
  • sidecar 123 is a duplicate version of sidecar 121 . That is, the console 300 can again create a sidecar in the node 100 that has the same function as the sidecar 121 .
  • the sidecar 121 sends the data packets to the backend container group after performing traffic management on the data packets sent by the service container group 131 .
  • the backend container group may be a service container group on other nodes, such as node 200 .
  • the backend container group may be other service container groups on the node 100 except the service container group 131 .
  • the sidecar 121 may also generate a session ID and send the session ID to the service container group 131 and the connection control module 110 .
  • the session identifier may refer to a session identifier between the service container group 131 and the backend container group.
  • the connection control module 110 may record the correspondence between the session identifier and the backend container group.
  • the sidecar 123 When the sidecar 123 performs traffic management on the data packets sent by the service container group 131, it can obtain the session identifier from the service container group 131, and then can use the corresponding relationship between the session identifier and the backend container group and the information obtained from the service container group 131 The session identifier determines the backend container group, so that after the sidecar 123 performs traffic management on the new data packet sent by the service container group 131, the new data packet is sent to the backend container group.
  • the new data packet may refer to the data packet sent by the service container group 131 after the sidecar 123 determines the back-end container group according to the corresponding relationship between the session identifier and the backend container group and the session identifier obtained from the service container group 131 . Specifically, it will be specifically introduced in Embodiment 4 below, and will not be repeated here.
  • the sidecar allocation strategy A may include a strategy A3, and the strategy A3 is used to indicate that the sidecar in the sidecar cluster 120 is preferentially used when the number of service objects is 0.
  • the connection control module 110 can determine the number of service objects of the sidecar 121 , and forward the data packet sent by the service container group 131 to the sidecar 121 when the number of service objects of the sidecar 121 is 0.
  • the service object refers to the service container group served by the sidecar, which can be implemented with reference to the content introduced in Embodiment 2 below, and will not be repeated here.
  • connection control module 110 is also used to monitor the working status of each sidecar in the sidecar cluster 120 , and send the information of the offline sidecar to the console 300 when an offline sidecar is found. The details will be introduced in Embodiment 1 below, and will not be repeated here.
  • the node provided by the embodiment of the present application can run multiple sidecars, and select a sidecar for the service container group among the multiple sidecars according to the allocation strategy, and then use the selected sidecar to process the data packets sent by the service container group Traffic management, so that better traffic management can be performed on the data packets of the business container group, and the high availability of the business can be realized.
  • this embodiment provides a node 100 that can run a service container group cluster 130 , including multiple service container groups such as a service container group 131 and a service container group 132 .
  • the node 100 also runs a sidecar cluster 120, including multiple sidecars such as sidecar121, sidecar122, and sidecar123.
  • each sidecar in the plurality of sidecars can be bound to different hardware resources, for example, can be bound to one or more central processing units (central processing unit, CPU), and can also be bound to other computing modules (for example, dedicated to Encryption and decryption hardware, graphics processing unit (graphics processing unit, GPU, etc.).
  • Node 100 also runs a connection management module 110 .
  • the connection management module 110 may include a protocol stack 111 and a sidecar startup manager 112 .
  • the microservice node 100 can access the network, and the network can include multiple backend container groups such as the backend container group 210 and the backend container group 220.
  • the backend container group may run on other nodes in the network except node 100 .
  • a group of backend containers may run on nodes 100 .
  • the data packets sent by the service container group in the node 100 can be sent to the backend container group after being managed by the sidecar, so as to call the service of the backend container group or provide services for the backend container group.
  • the network may further include a console 300 .
  • the console 300 can receive the operation of the management personnel or operation and maintenance personnel, and control the node 100 and other nodes in the network in response to the operation.
  • the console 300 may send the service list to the sidecar to synchronize the service list, so that the sidecar can perform traffic management according to the service list.
  • the console 300 may send the sidecar allocation policy A to the connection management module 110, so that the connection management module 110 selects a sidecar for the service container group in the sidecar cluster according to the sidecar allocation policy A. The selected sidecar performs traffic management on the data packets sent by the service container group.
  • the connection management module 110 may manage multiple sidecars in the node 100 in an all-active manner. That is, the multiple sidecars can be in a running state or working state at the same time.
  • the same sidecar can provide traffic management for links serving multiple service container groups at the same time.
  • sidecar 121 can provide traffic management for link 1311 , link 1312 and link 1321 at the same time.
  • link 1311 and link 1312 are links from the service container group 131
  • link 1321 is a link from the service container group 132 .
  • different sidecars can simultaneously provide traffic management for different links of the same service container group.
  • sidecar 121 can provide traffic management for link 1321 originating from service container group 132 .
  • the sidecar 123 may provide traffic management for the link 1322 originating from the traffic container group 132 .
  • providing traffic management for the link of the service container group may specifically refer to providing traffic management for the data packets sent by the service container group through the link.
  • flow control, flow security, flow observation, etc. are performed on the data packet.
  • the traffic control may include any one or a combination of traffic passing, traffic denying, traffic duplication, and traffic coloring.
  • the node 100 may also include a protocol stack 111, which may be used to allocate a sidecar for the connection request from the sidecar cluster 120 when the service container group generates a connection request, thereby establishing a relationship between the service container group and the allocated sidecar. The link between them, so that the allocated sidecar can provide traffic management for the data packets sent by the service container group through the link.
  • a protocol stack 111 which may be used to allocate a sidecar for the connection request from the sidecar cluster 120 when the service container group generates a connection request, thereby establishing a relationship between the service container group and the allocated sidecar. The link between them, so that the allocated sidecar can provide traffic management for the data packets sent by the service container group through the link.
  • the protocol stack 111 can call or read the sidecar list.
  • the sidecar list may be set in the protocol stack 111 .
  • the sidecar list may include information of sidecars in a running state.
  • the information of the sidecar may include an identifier of the sidecar (for example, a process identity (identity, ID)), version information of the sidecar, and a listening port of the sidecar.
  • the sidecar version information is used to represent the sidecar version.
  • the version information of the sidecar may be a version number.
  • the size of the version number represents the level of the version.
  • the listening port of the sidecar can also be the inbound port of the sidecar. Through the listening port of the sidecar, a connection request can be sent to the sidecar, so that a link between the source service container group of the connection request and the sidecar can be established.
  • the sidecar information may also include the current connection number of the sidecar.
  • the number of current connections is used to represent the number of links currently connected to the sidecar. Taking sidecar 121 shown in FIG. 3A as an example, it is connected to link 1311 , link 1312 , and link 1321 , so the current number of connections is three.
  • the sidecar information may also include the identifier of the service container group that the sidecar is currently connected to.
  • the identifier of the service container group is used to uniquely identify the service container group. Therefore, the protocol stack 111 can determine the service container group currently connected to the sidecar through the identifier of the service container group currently connected to the sidecar.
  • the sidecar-connected service container group refers to the service container group that has a link with the sidecar. For example, as shown in FIG. 3A , the service container group 131 belongs to the currently connected service container group of the sidecar 121 .
  • the information of the sidecar may also include hardware resource information bound to the sidecar (such as CPU information, whether to enable hardware acceleration, etc.) and the like.
  • the sidecar list may also include other information, which will not be listed here.
  • connection request P1 may be sent to the protocol stack 111 .
  • the protocol stack 111 can read the sidecar information recorded in the sidecar list, and allocate a sidecar to the connection request P1 according to the sidecar information. Next, a specific description will be given.
  • the protocol stack 111 may determine whether the service container group P is the service container group currently connected to the sidecar, that is, determine that one or more links already exist between the service container group P and the sidecars in the sidecar list.
  • the protocol stack 111 may send the connection request P1 to the sidecar B1.
  • the protocol stack 111 can send the connection request P1 through the listening port of the sidecarB1, so that the connection request P1 can be sent to the sidecarB1.
  • sidecarB1 receives the connection request P1
  • another link can be established between the sidecarB1 and the service container group P according to the connection request P1.
  • Service container group P can send data packets to sidecarB1 through this link. After receiving the data packet, sidecarB1 can perform traffic management on the data packet.
  • the sidecar with the least current connection number may be determined according to the current connection number of each sidecar in the sidecar list. It can be set that sidecarB2 in the sidecar list has the least number of current connections.
  • SidecarB2 may be assigned to connection request P1. When sidecarB2 receives the connection request P1, another link is established between sidecarB2 and the service container group P according to the connection request P1. Service container group P can send data packets to sidecarB2 through this link. After receiving the data packet, sidecarB2 can perform traffic management on the data packet.
  • the protocol stack 111 may determine whether a sidecar of a new version exists in the sidecar list. Specifically, it can be determined whether all sidecar versions in the sidecar list are the same. If the version of one or more sidecars in the sidecar list is higher than that of other sidecars, it may be determined that the one or more sidecars are sidecars of a new version. If the versions of all the sidecars in the sidecar list are the same, it can be determined that there is no new version of the sidecar in the sidecar list.
  • the sidecar of the new version may be assigned to the connection request P1.
  • the sidecar of the new version receives the connection request P1, another link is established between the sidecar of the new version and the service container group P according to the connection request P1.
  • the service container group P can send data packets to the sidecar of the new version through this link. After receiving the data packet, the sidecar of the new version can perform traffic management on the data packet.
  • a sidecar in node 100 may be hot upgraded. That is, the upgrade of the sidecar can be completed without the user's awareness.
  • Operation and maintenance personnel can configure sidecar creation information on the console 300 for creating a new sidecar in the node 100 .
  • the sidecar creation information may include version information and hardware resource information.
  • the sidecar creation information may also include replaced sidecar information, which is used to indicate which sidecars in the node 100 the newly created sidecar is used to replace.
  • the sidecar to be replaced may be a sidecar already in running state in the node 100.
  • the information of the replaced sidecar may include the identifier of the replaced sidecar, the remaining runtime of the replaced sidecar, and the like. Exemplarily, when there are multiple sidecars to be replaced, different remaining runtimes may be set for different sidecars to be replaced, so as to prevent or alleviate the problem of sudden increase in connection requests caused by simultaneous offline of multiple sidecars to be replaced.
  • the sidecar creation information may also be called sidecar upgrade information, that is, the sidecar creation information may be used to upgrade the sidecar.
  • the console 300 may execute step 401 to send sidecar creation information to the sidecar startup manager 111321 .
  • the sidecar startup manager 112 can create a new sidecar based on the sidecar creation information.
  • the sidecar launch manager 112 creates a sidecar conforming to the version information. That is, the version of the newly created sidecar conforms to the version information.
  • the sidecar startup manager 112 may also perform step 402 to allocate a listening port for the new sidecar.
  • the sidecar startup manager 112 may allocate a listening port to a new sidecar according to the occupancy of the listening port in the node 100 . When allocating the listening port, the listening port number can be assigned to the new sidecar to complete the allocation of the listening port.
  • the sidecar startup manager 112 may execute step 403 to start a new sidecar. In this way, a new sidecar is created on the sidecar side.
  • the new sidecar can execute step 404, and the new sidecar can be initialized and enter the running state.
  • the sidecar startup manager 112 may execute step 407 to update the sidecar list, and send the updated sidecar list to the protocol stack 111 .
  • updating the sidecar list includes: adding new sidecar information (such as listening port, version information, current connection number, currently connected service container group identifier, hardware resource information, etc.) to the sidecar list.
  • the sidecar startup manager 112 can carry the updated sidecar list in the Ebpf (extended BPF) map, and then send the Ebpfmap to the protocol stack 111, so as to send the updated sidecar list to the protocol stack 111.
  • the sidecar startup manager 112 may execute step 406 to monitor the running status of the sidecar.
  • the sidecar startup manager 112 can establish a domain socket (domain socket) persistent connection between itself and the sidecar, and monitor the running status of the sidecar through the domain socket persistent connection. And according to the monitored result, execute step 407 and execute step 408 .
  • the sidecar startup manager 112 may send sidecar update information to the console 300, including the running status and identification information of the new sidecar.
  • the new sidecar may also start domain socket (domain socket) monitoring in step 404.
  • the new sidecar may also perform step 405, and the new sidecar is connected to replace the sidecar through the domain socket.
  • the new sidecar can monitor the running status of the replaced sidecar through the domain socket between it and the replaced sidecar, and send an offline command to the replaced sidecar to instruct the replaced sidecar to go offline.
  • the new sidecar starts timing after initialization is completed, and when the timing reaches the remaining running time, the new sidecar can execute step 409, and the replaced sidecar will be offline .
  • the new sidecar can send an offline command to the replaced sidecar through the domain socket between it and the replaced sidecar.
  • the replaced sidecar can be offline in response to this command.
  • one or more sidecars in the node 100 may execute step 410, crash or restart.
  • the crash or restart of the sidecar can be collectively referred to as offline.
  • the offline of the sidecar may lead to the disconnection of the link between the sidecar and the service container group, and the disconnection of the domain socket between the sidecar and the sidecar startup manager 112 . That is, in step 411, the link between the offline sidecar and the service container group is disconnected. In step 412, the domain socket between the offline sidecar and the boot manager 112 is disconnected. When the domain socket between the offline sidecar and the data plane startup manager 112 is disconnected, the sidecar startup manager 112 can confirm that the sidecar is offline, and then execute step 413 to send sidecar update information to the console 300, including the downlink The identification information of the sidecar of the line, etc.
  • the sidecar startup manager 112 may also perform step 414 to update the sidecar list, and send the updated sidecar list to the protocol stack 111 .
  • updating the sidecar list may include deleting an offline sidecar from the sidecar list or listing it as a non-running state.
  • the protocol stack 111 may execute step 415 to clear the connection information of the offline sidecar.
  • the connection information may include information such as a service container group historically connected to the offline sidecar.
  • the service container group may generate a connection request to request to connect to the sidecar again. Specifically, the service container group may send a connection request to the protocol stack 111 in step 416 .
  • the service container group sending the connection request may be referred to as the service container group to be connected.
  • the protocol stack 111 may execute step 417 to select a sidecar for the service container group to be connected.
  • the sidecar can be selected according to the following rules.
  • the sidecar with the least number of current connections may be determined from the sidecar list. And the sidecar with the least number of current connections is selected as the sidecar.
  • the protocol stack 111 can execute step 418 to send the connection request to the selected sidecar, so as to create a link between the sidecar and the service container group to be connected, so that the sidecar can connect the service container The group sends packets over the link for traffic management.
  • the sidecar may be selected through the rules provided in the embodiment shown in FIG. 2 .
  • the node provided in embodiment 1 can support multi-instance operation of sidecar, and can perform load balancing on each available sidecar instance through the number of connections.
  • the node can also support dynamic expansion of sidecar instances and hot upgrade of existing instances. New connections will preferentially use the hot-upgraded sidecar, and control the offline time window of the replaced sidecar to reduce the problem of connection queue accumulation caused by instantaneous connection switching.
  • the protocol stack and the sidecar startup manager can automatically manage the listening port of each sidecar. When the sidecar is down due to its own problems, the business container group connected to the sidecar will be relocated to other available sidecars On the one hand, without manual intervention, the overall availability of the system is improved.
  • the embodiment of the present application provides a node 100, which runs a sidecar cluster 120, and may include multiple types of sidecars, wherein different types of sidecars have different performances. It may be set that the sidecar cluster 120 includes a sidecar121 and a sidecar122, wherein the sidecar121 is a high-performance sidecar, and the sidecar122 is a low-performance sidecar.
  • high-performance sidecars are higher than that of ordinary sidecars.
  • high-performance sidecars are configured with more hardware resources, such as a larger number of CPUs, acceleration hardware, and so on.
  • high-performance sidecars allocate higher hardware resources than ordinary performance sidecars. Therefore, high-performance sidecars have stronger data processing capabilities than ordinary sidecars.
  • QoS Quality of Service
  • the high-performance sidecar provides traffic management for the link of the business container group, it can guarantee the service quality (Quality of Service, QoS) of the business container group. Therefore, for a business container group with quality of service requirements, a link can be created between it and a high-performance sidecar, so that the high-performance sidecar can provide traffic management for the link to ensure the service quality of the business container group.
  • QoS Quality of Service
  • Operation and maintenance personnel can configure a priority list through the console 300, and the priority list can include various types of connection requests.
  • different types of connection requests correspond to different types of sidecars.
  • the node 100 is provided with a high-performance sidecar and a normal sidecar.
  • the priority list may include connection requests of the first type and connection requests of the second type.
  • the first type of connection request corresponds to a high-performance sidecar
  • the second type of connection request corresponds to a normal performance sidecar.
  • each type of connection request may correspond to at least one service container group label (label). Different types of connection requests correspond to different service container group labels. That is, each type of connection request can be characterized by a service container group label.
  • a business container group tag is used to represent a type of business container group. Wherein, when the business container group is pod, the business container group label is podlabel.
  • the priority list records the correspondence between the service container group label corresponding to the connection request and the sidecar type. For example, a type of connection request corresponds to a service container group label Q1 and a service container group label Q2, and this type of connection request corresponds to a high-performance sidecar. Then, the priority record list may record the corresponding relationship between the service container group label Q1 and the high-performance sidecar, and also record the corresponding relationship between the service container group label Q2 and the high-performance sidecar.
  • each type of connection request may correspond to at least one service type. Different types of connection requests correspond to different service types. That is, each type of connection request can be characterized by a service type.
  • the priority list records the corresponding relationship between the service type corresponding to the connection request and the sidecar type. For example, a type of connection request corresponds to a service type S, and this type of connection request corresponds to a high-performance sidecar. Then, the priority record list can record the corresponding relationship between the service type S and the high-performance sidecar.
  • connection control module 110 may include a protocol stack 111 and a control plane proxy 113 .
  • the console 300 may send the configured priority list to the control plane proxy 113 .
  • the control plane agent 113 can save the priority list and send the priority list to the protocol stack 111 .
  • the control plane agent may send the priority list to the protocol stack 111 through Ebpfmap.
  • the protocol stack 111 is also configured with a sidecar list, which records information such as the type of the sidecar of the node 100, the listening port of the sidecar, and the like.
  • a sidecar list which records information such as the type of the sidecar of the node 100, the listening port of the sidecar, and the like.
  • the protocol stack 111 can record the sidecar type in the sidecar list according to the correspondence between the sidecar identifier and the sidecar type.
  • the protocol stack 111 may also be configured with a corresponding relationship between a service container group label and a service container group identifier.
  • the service container group identifier may be cgroup.ns(control group namespace).
  • the service container group identifier is used to represent a service container group
  • the service container group label is used to represent a type of service container group. That is to say, multiple service container groups have multiple service container group identifiers, wherein the multiple service container groups are in one-to-one correspondence with the multiple service container group identifiers.
  • the service container group labels of the multiple service container groups are the same. The manner of determining the correspondence between the service container group label and the service container group identifier will be described below in the embodiment shown in FIG. 4B , and will not be repeated here.
  • the priority list records the correspondence between service container group labels and sidecar types.
  • the protocol stack 111 can determine the service container group identifier of the service container group 131 according to the source address (such as the source IP address) of the connection request . Then, the service container group label of the service container group 131 may be determined according to the service container group identifier of the service container group 131 and the corresponding relationship between the service container group label and the service container group identifier. After the service container group label of the service container group 131 is determined, the sidecar type corresponding to the service container group label of the service container group 131 may be determined according to the priority list.
  • the protocol stack 111 sends the connection request to the determined data agent of the sidecar type.
  • the sidecar type corresponding to the service container group label of the service container group 131 can be set as a high-performance sidecar, and then the protocol stack 111 can send the connection request from the service container group 131 to the high-performance sidecar.
  • the high-performance sidecar is made to provide flow management for data packets from the service container group 131 .
  • the priority list records the correspondence between service types and sidecar types.
  • the protocol stack 111 may determine a target service of the connection request. Then, the sidecar type corresponding to the target service may be determined according to the service type and the priority list of the target service. Then, the protocol stack 111 may send the connection request to the determined data proxy of the sidecar type. For example, the sidecar type corresponding to the target service of the connection request sent by the business container group 132 can be set as a normal performance sidecar, and then the protocol stack 111 can send the connection request from the business container group 132 to the normal performance sidecar. The normal performance sidecar is made to provide traffic management for data packets from the business container group 132 .
  • Embodiment 2 is described in more detail.
  • the service container group monitor module (for example, pod monitor module (monitor)) in the node 100 can monitor the service container group creation module through step 501 to monitor the service container group creation.
  • the business container group creation module may specifically be kubelet.
  • Service container group creation module In step 502, a service container group can be created. In this way, the service container group monitoring module can detect the service container group creation event through step 503 . Through the service container group creation event, the service container group monitoring module can obtain the service container group identifier and service container group label of the created service container group. Afterwards, the service container group monitoring module may perform step 504 to establish a corresponding relationship between the service container group label and the service container group identifier.
  • the service container group monitoring module may send the corresponding relationship between the service container group label and the service container group identifier to the protocol stack 111 through step 505 .
  • the service container group monitoring module may send the corresponding relationship between the service container group label and the service container group identifier to the protocol stack 111 through the Ebpf map.
  • the operation and maintenance personnel can configure the priority list on the console 300 .
  • the console 300 may send the priority list to the control plane agent 113 through step 507 .
  • the control plane proxy 113 may send the priority list to the protocol stack 111 through step 508 .
  • the protocol stack 111 may save the priority list through step 509 .
  • the protocol stack 111 may receive the connection request sent by the service container group through step 510 .
  • the service container group that sends the connection request may be referred to as the service container group to be connected.
  • the protocol stack 111 may analyze the source address of the connection request, and determine the ID of the service container group according to the source address of the connection request in step 511 .
  • the protocol stack 111 may determine the service container group label in step 512 according to the determined service container group identifier. Further, the protocol stack 111 can determine the sidecar type corresponding to the service container group label according to the priority list in step 513, so as to determine the sidecar type corresponding to the service container group to be connected.
  • the protocol stack 111 may select a sidecar for the service container group to be connected in step 514 .
  • the priority list is empty, that means there is no correspondence between connection requests and sidecar types configured. From the sidecar list, select the sidecar with the least number of current connections.
  • the high-performance sidecar can provide traffic management for the service container group to be connected, so that the high-performance sidecar can Performance When the sidecar is idle, the overall resource utilization of the sidecar is improved.
  • the service container group corresponds to a high-performance sidecar, select the high-performance sidecar.
  • the protocol stack 111 may execute step 515 to send the connection request to the selected sidecar.
  • the protocol stack 111 may execute step 515 to send the connection request to the selected sidecar.
  • the sidecar In order to create a link between the sidecar and the service container group to be connected, so that the sidecar can perform traffic management on the data packets sent by the service container group through the link.
  • Embodiment 2 supports the use of sidecars with different performances to manage traffic for different types of business container groups or target services when Embodiment 1 supports multi-active and high availability, so as to ensure that some services require high quality of service.
  • the connection requests use independent or more hardware resources, and can improve the overall resource utilization of the sidecar when the high-performance sidecar is idle.
  • the service change may include service online, service offline, and the like.
  • a long socket (socket) connection is established between the sidecars of nodes such as node M and node N in the service grid system and the control plane in the service grid system.
  • the control plane can generate a service change message.
  • the service change message may be used to describe the service change of the backend container group, or the service change message may include an updated service list due to the service change of the backend container group.
  • the control plane sends service change messages to each sidecar through the long connection between it and each sidecar.
  • the backend container group may refer to a container group capable of providing corresponding services in response to the service call of the node M or the node N.
  • the control plane needs to poll all sidecars it is connected to and send service change messages.
  • polling sidecars and sending service change messages will consume a large amount of computing resources on the control plane and put a lot of pressure on the network.
  • the control plane performs polling and sends service change messages according to the startup sequence of the sidecar or a random sequence. This may lead to large differences in the time when different sidecars in the same node receive service change messages. In turn, the time window for inconsistent service instances perceived by the sidecar in the same node becomes larger, which may cause business exceptions.
  • the console 300 can establish a connection (for example, a long socket connection) with the control plane agent of each node. In this way, the console 300 can send the service change message to the control plane agent of each node.
  • the control plane sends a service change message to the control plane proxy 113 of node 100 and the control plane proxy 213 of node 200 .
  • the control plane proxy 113 may send the service change message to each sidecar such as the sidecar 121 and the sidecar 122 in the node 100 .
  • the control plane proxy 213 may send the service change message to each sidecar such as the sidecar 221 and the sidecar 222 in the node 200 .
  • the control plane performs polling and sends service change messages at the node level, which reduces the consumption of computing resources on the control plane and the pressure on the network.
  • the control plane agent of the node sends the service change message to each sidecar in the node through the internal communication mechanism of the node. In this way, the time for the service instance inconsistency perceived by the sidecar in the same node is greatly reduced window.
  • control plane agent when the network between the control plane and the control agent fails, can serve as an offline service center to provide the reading function for the sidecar in the node where it is located, and realize the communication between the sidecar and the control side .
  • control side includes a control plane agent and a control plane.
  • Embodiment 3 will be described in more detail.
  • the console 300 may monitor the service change of the backend container group through step 601 . And in step 602 , a connection between the console 300 and the control plane proxy 113 within the node 100 is established.
  • the connection can be a socket long connection.
  • the control plane agent 113 may establish a connection with the sidecar in the node where it is located.
  • the console 300 When the console 300 senses that a service change occurs in the backend container group, it may generate a service change message in step 604 , and send the service change message to the control plane agent through step 605 .
  • control plane proxy may send the service change message to the sidecar through step 606 .
  • the sidecar may update routing information according to the service change message.
  • the server When the client requests a session from the server for the first time, the server will create a session for the client, and a session identity (sessionID) calculated by a special algorithm is used to identify the session.
  • the server can return the session identifier to the client.
  • the client can save the session ID in a local cookie.
  • the session identifier can be sent to the server.
  • the server will use the session corresponding to the session ID again to provide the corresponding service for the client.
  • the client is a business container group that can call the service.
  • the server can also be called a backend container group, which can provide services for the business container group.
  • the server can include multiple service instances, and use one or more of the service instances to provide services to the client.
  • the service container group C can be connected to the sidecar M1 through the link C1, and connected to the sidecar M2 through the link C2.
  • the service container group C can send a connection request to the backend container group through the link C1, and the connection request can carry the session identifier E.
  • sidecar M1 After sidecar M1 receives the connection request, it can extract the session ID E, and then use the hash (hash) algorithm to determine the routing policy according to the session ID E, so as to send the connection request to the service instance according to the routing policy.
  • the routing policy may be set to send the connection request to the service instance D1 in the backend container group.
  • the service container group C can also send a connection request to the backend container group through the link C2, and the connection request can carry the session identifier E.
  • sidecar M2 After sidecar M2 receives the connection request, it can extract the session identifier E, and then use the hash (hash) algorithm to determine the routing strategy according to the session identifier E, so as to send the connection request to the service instance according to the routing strategy.
  • the routing policy may be set to send the connection request to the service instance D2 in the backend container group.
  • Embodiment 4 provides a solution.
  • the connection requests of the service container group through different links can be sent to the same service instance.
  • this solution will be illustrated with reference to FIG. 4B and FIG. 4C .
  • node 100 may include sidecar121 and sidecar122.
  • the node 100 may further include a connection control module 110 .
  • the service container group 131 may be connected to the sidecar 121 through a link 1311 and connected to the sidecar 122 through a link 1313 .
  • the backend container group can be configured with multiple service instances such as service instance G1 and service instance G2.
  • the node 100 may be configured with a mapping relationship between a session identifier F and a service instance.
  • a sidecar such as sidecar121
  • the sidecar121 can establish a mapping relationship between the session ID F and the service instance.
  • sidecar121 can share the mapping relationship with sidecar122.
  • the sidecar 121 may send the mapping relationship to the connection control module 110 .
  • the sidecar 122 can read the mapping relationship between the session identifier F and the service instance in the connection control module 110, and can also save the mapping relationship in a local cache.
  • the sidecar 121 When the sidecar 121 receives the connection request containing the session identifier F sent by the service container group 131 through the link 1311 , it can determine the service instance according to the correspondence between the session identifier F and the service instance. In turn, the connection request is routed to the service instance.
  • the sidecar 122 receives the connection request containing the session identifier F sent by the service container group 131 through the link 1313 , it can determine the service instance according to the correspondence between the session identifier F and the service instance. In turn, the connection request is routed to the service instance. In this way, when different sidecars provide traffic management for different links of the same service container group, the connection requests of the service container group through different links can be sent to the same service instance.
  • Embodiment 4 will be described in more detail.
  • the service container group 131 may send a session request to the backend container group 210 through step 701 .
  • the backend container group 210 may assign a session identifier F to the service container group 131 in response to the session request in step 702 , and send the session identifier F to the service container group 131 in step 703 .
  • the service container group 131 may save the session identifier F.
  • the service container group 131 may send a connection request including the session identifier F to the sidecar 121 through step 705 .
  • the sidecar 121 when the sidecar 121 receives the connection request containing the session ID F, it can query the service instance corresponding to the session ID F in the connection control module 110 through step 706a.
  • the sidecar 121 when the sidecar 121 receives the connection request containing the session identifier F, it may query the service instance corresponding to the session identifier F in the local cache of the sidecar 121 through step 706b. That is to say, in this example, the correspondence between the session identifier F and the service instance may be stored in the local cache of the sidecar 121 .
  • the service container group 131 may send a connection request including the session identifier F to the sidecar 122 through step 707 .
  • the sidecar 122 when the sidecar 122 receives the connection request containing the session ID F, it can query the service instance corresponding to the session ID F in the connection control module 110 through step 708a.
  • the sidecar 122 when the sidecar 122 receives the connection request containing the session identifier F, it may query the service instance corresponding to the session identifier F in the local cache of the sidecar 122 through step 708b. That is to say, in this example, the correspondence between the session identifier F and the service instance may be stored in the local cache of the sidecar 122 .
  • sidecar 121 may execute step 709 , and if no service instance is found, perform a hash operation according to the session identifier F to obtain a service instance corresponding to the session identifier F. That is to say, through the above step 706a and step 706b, no service instance corresponding to the session identifier F is found.
  • step 706a the connection control module 110 does not save the correspondence between the session identifier F and the service instance; and when sidecar121 executes step 706b, the local cache of sidecar121 does not save the correspondence between the session identifier F and the service instance.
  • sidecar 121 performs a hash operation according to the session ID to obtain a service instance corresponding to the session ID F.
  • the sidecar 121 may execute step 710 to create a mapping relationship between the session ID F and the service instance, and send the mapping relationship between the session ID F and the service instance to the connection control module 110 .
  • the connection control module 110 may save the mapping relationship between the session identifier F and the service instance.
  • sidecar121 may execute step 712 to update the local cache, so as to save the mapping relationship between the session identifier F and the service instance in the local cache.
  • sidecar121 may execute step 713 to create a connection according to the service instance. That is, the connection request is sent to the service instance corresponding to the session identifier F, so as to establish the connection between the service container group 131 and the service instance.
  • sidecar 122 may execute step 714 , and if no service instance is found, perform a hash operation according to the session identifier F to obtain a service instance corresponding to the session identifier F. That is to say, through the above step 708a and step 708b, no service instance corresponding to the session identifier F is found.
  • step 708a the connection control module 110 does not save the correspondence between the session identifier E and the service instance; and when sidecar122 executes step 708b, the local cache of sidecar122 does not save the correspondence between the session identifier F and the service instance.
  • sidecar 122 performs a hash operation according to the session ID to obtain a service instance corresponding to the session ID F.
  • the sidecar 122 may execute step 715 to create a mapping relationship between the session ID F and the service instance, and send the mapping relationship between the session ID F and the service instance to the connection control module 110 .
  • the connection control module 110 can execute step 716 to determine that there is already a mapping relationship between the session identifier F and the service instance in the connection control module 110 (the mapping relationship between the session identifier F and the service instance is saved in the above step 711), and no longer send The mapping relationship between the session ID E and the service instance is updated.
  • the connection control module 110 may also execute step 717 to send the mapping relationship between the session identifier F and the service instance saved in step 711 to the sidecar 122 .
  • the sidecar 122 may execute step 718 to update the local cache, so as to save the mapping relationship between the session identifier F received from the connection control module 110 and the service instance in the local cache.
  • the sidecar 122 may determine the service instance corresponding to the session identifier F according to the mapping relationship between the session identifier F and the service instance received from the connection control module 110 . Then, sidecar122 may execute step 719 to create a connection according to the service instance. That is, a connection request is sent to the service instance corresponding to the session identifier F, so as to establish a connection between the service container group 131 and the service instance.
  • the lifetime of a session ID can be configured.
  • the connection control module 110 may determine that the session ID is expired when the life cycle of the session ID ends, and execute step 720 to clear the expired session ID. Wherein, in step 720, the mapping relationship between expired session identifiers and service instances is cleaned up.
  • the connection control module 110 may execute step 721a, and send a session ID clearing command to the sidecar 121 .
  • the sidecar 121 may respond to the session identifier clearing command and execute step 722a to clear the cache, so as to clear the expired mapping relationship between the session identifier and the service instance from the local cache.
  • the connection control module 110 may execute step 721b, and send the session ID clearing command to the sidecar 122 .
  • the sidecar 122 may respond to the session identifier clearing command and execute step 722b to clear the cache, so as to clear the expired mapping relationship between the session identifier and the service instance from the local cache.
  • connection requests of the service container group through different links can be sent to the same service instance , so that the service instance can provide services for the service container group through different links.
  • the node runs a connection control module, a sidecar cluster, and a first service container group, and the sidecar cluster includes at least two sidecars.
  • the method includes the following steps.
  • Step 7001 the connection control module receives the sidecar allocation strategy sent by the console connected to the node, selects the first sidecar in the sidecar cluster according to the sidecar allocation strategy, and forwards the data packet sent by the first service container group to the second sidecar side car.
  • Step 7002 the first sidecar performs traffic management on the data packets sent by the first service container group.
  • the node also runs a second service container group
  • the method further includes: the connection control module selects a second sidecar in the sidecar cluster according to the sidecar allocation policy, and assigns the second sidecar to The data packets sent by the second service container group are forwarded to the second sidecar; the second sidecar performs flow management on the data packets sent by the second service container group.
  • the specifications of hardware resources allocated by the first sidecar are higher than those allocated by the second sidecar
  • the sidecar allocation strategy includes a first strategy, and the first strategy uses Instructing the first service container group to preferentially use the first sidecar; selecting the first sidecar in the sidecar cluster according to the sidecar allocation strategy includes: Select the first sidecar in the cluster.
  • the node also runs a second service container group
  • the sidecar allocation strategy further includes a second strategy, the second strategy is used to indicate that the number of service objects of the first sidecar does not exceed upper limit value
  • the method further includes: the connection control module determines the number of service objects of the first sidecar, and when the number does not exceed the upper limit value, group the second service container The sent data packets are forwarded to the first sidecar; the first sidecar simultaneously performs traffic management on the data packets sent by the first service container group and the data packets sent by the second service container group.
  • the method further includes: the connection control module, after the failure of the first sidecar, selects a third sidecar from the cluster of sidecars or notifies the console to Create the third sidecar, and forward another data packet sent by the first service container group to the third sidecar; the third sidecar forwards another data packet sent by the first service container group package for traffic management.
  • the third sidecar is a new version with upgraded functions based on the first sidecar, or the third sidecar is a copied version of the first sidecar.
  • the method further includes: after the first sidecar performs traffic management on the data packets sent by the first service container group, sending the data packets to the backend container Group.
  • the method further includes: the first sidecar generates a session identifier and sends the session identifier to the first service container group and the connection control module; the connection control module records The corresponding relationship between the session ID and the backend container group; the third sidecar acquires the session ID from the first service container group and records the session ID in the connection control module according to the session ID
  • the back-end container group is determined in the corresponding relationship, and after traffic management is performed on another data packet sent by the first service container group, the other data packet is sent to the back-end container group.
  • the sidecar allocation strategy includes a third strategy, and the third strategy is used to indicate that the sidecar cluster in the sidecar cluster is preferentially used when the number of service objects is 0; according to the The sidecar allocation policy selects the first sidecar in the sidecar cluster, and forwards the data packet sent by the first service container group to the first sidecar, including: the connection control module determines that the first The number of service objects of the sidecar, if the number of service objects of the first sidecar is 0, the data packet sent by the first service container group is forwarded to the first sidecar.
  • the method further includes: the connection control module monitors the working status of each sidecar in the sidecar cluster, and when an offline sidecar is found, sends the offline sidecar information to the console.
  • the traffic management includes: traffic control, traffic security, and traffic observation.
  • the container group management method provided by the embodiment of the present application can select a sidecar for a business container from at least two sidecars according to the sidecar allocation strategy sent by the console, and use the selected sidecar to send the service container group
  • the traffic management of the data packets can realize the flexible management of the business container group, perform better traffic management for the business container group, and ensure the high availability of the business container group.
  • the sidecar can also perform traffic processing on data packets sent from other service container groups outside the node or within the node to the service container group bound to the sidecar, and the traffic processing strategy is also It can be set with reference to the above manner, which is not limited in this embodiment of the present application.
  • the embodiment of the present application also provides a node 800 , including a processor 810 and a memory 820 .
  • the processor 810 is configured to execute instructions stored in the memory 820 , so that the node 800 can execute the method shown in FIG. 7 .
  • the embodiment of the present application also provides a computer-readable storage medium, including computer program instructions.
  • the computer program instructions When the computer program instructions are executed by a cluster of computing devices, the cluster of computing devices executes the method shown in FIG. 7 .
  • the embodiment of the present application also provides a computer program product containing instructions, and when the instructions are run by a cluster of computer equipment, the cluster of computer equipment executes the method shown in FIG. 7 .
  • the processor in the embodiments of the present application may be a central processing unit (central processing unit, CPU), and may also be other general processors, digital signal processors (digital signal processor, DSP), application specific integrated circuits (application specific integrated circuit, ASIC), field programmable gate array (field programmable gate array, FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof.
  • a general-purpose processor can be a microprocessor, or any conventional processor. It can be understood that the various numbers involved in the embodiments of the present application are only for convenience of description, and are not used to limit the scope of the embodiments of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Software Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Mobile Radio Communication Systems (AREA)

Abstract

本申请涉及数据处理技术领域，具体涉及一种运行容器组的节点，容器组的管理系统和方法。其中，该节点运行有连接控制模块、边车集群、和第一业务容器组，边车集群包括至少两个边车，其中，连接控制模块，用于接收与该节点相连的控制台发送的边车分配策略，根据边车分配策略在边车集群中选择第一边车，并将第一业务容器组发出的数据包转发至第一边车；第一边车，用于对第一业务容器组发出的数据包进行流量管理。在本申请中，可以为业务容器组选择边车，以对业务容器组进行更优的流量管理。

Description

运行容器组的节点，容器组的管理系统和方法
本申请要求于2021年8月9日提交中国国家知识产权局、申请号为202110910594.X、申请名称为“运行容器组的节点，容器组的管理系统和方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及云计算领域，且特别涉及一种运行容器组的节点，容器组的管理系统和方法。
背景技术
服务网格(service mesh)技术将微服务架构(microservice architecture)的分布式应用中非功能性的针对流量的服务治理逻辑从业务进程中剥离到边车(sidecar)容器中，以无侵入的方式提供服务间的连接、安全、流控、灰度发布、观测能力，实现业务轻量化和服务治理基础设施化。另外，由于服务网格技术是基于传统互联网协议(internet protocol，IP)网络之上的应用网络技术。因此，在服务网格技术中，服务之间的发现与路由不再直接基于IP地址，而是基于服务的元数据信息(包括但不限于服务名称、版本等)。
随着用户需求的发展，微服务的规模和调用复杂度快速增长。如何在持续运行阶段高效的治理微服务、降低运维成本，是服务网格技术演进的一个重要问题。
发明内容
本申请实施例提供了一种运行容器组的节点，容器组的管理系统和方法，可以为业务容器组选择边车，以对业务容器组进行更优的流量管理。
第一方面，本申请实施例提供了一种运行容器组的节点，该节点运行有连接控制模块、边车集群、和第一业务容器组，边车集群包括至少两个边车，其中，连接控制模块，用于接收与该节点相连的控制台发送的边车分配策略，根据边车分配策略在边车集群中选择第一边车，并将第一业务容器组发出的数据包转发至第一边车；第一边车，用于对第一业务容器组发出的数据包进行流量管理。
在该方案中，连接控制模块可以按照控制台发送的边车分配策略，从至少两个边车中为第一业务容器选择边车，并使用选择出的边车对第一业务容器组发出的数据包进行流量管理，从而可以实现对第一业务容器组的灵活管理，可以为对第一业务容器组进行更优的流量管理，保障第一业务容器组业务的高可用能力。
在一种可能的实现方式中，该节点还包括第二业务容器组；连接控制模块，还用于根据边车分配策略在边车集群中选择第二边车，并将第二业务容器组发出的数据包转发至第二边车；第二边车，用于对第二业务容器组发出的数据包进行流量管理。其中，第二边车和第一边车可以为同一边车冗余，也可以为不同的边车。
也就是说，连接控制模块可以按照边车分配策略，从至少两个边车中为第二业务容器选择边车，并使用选择出的边车对第二业务容器组发出的数据包进行流量管理，从而可以实现对第二业务容器组的灵活管理，可以为对第二业务容器组进行更优的流量管理，保障第二业务容器组的业务的高可用能力。
在一种可能的实现方式中，第一边车分配的硬件资源规格高于第二边车分配的硬件资源，边车分配策略包括第一策略，第一策略用于指示第一业务容器组优先使用第一边车；连接控制模块，具体用于根据第一策略在边车集群中选择第一边车。
也就是说，在该实现方式中，可以设置硬件资源规格高低不同的硬件资源，并在边车分 配策略指示第一业务容器组优先使用高硬件资源规格的边车的情况下，使用高硬件资源规格的边车对第一业务容器组发出的数据包进行流量管理，保障了第一业务容器组业务的服务质量。
在一种可能的实现方式中，该节点还包括第二业务容器组，边车分配策略还包括第二策略，第二策略用于指示第一边车的服务对象数量不超过上限值；连接控制模块，还用于确定第一边车的服务对象的数量，在第一边车的服务对象的数量不超过上限值的情况下将第二业务容器组发出的数据包转发至第一边车；第一边车，还用于对第一业务容器组发出的数据包和第二业务容器组发出的数据包同时进行流量管理。
也就是说，在该实现方式中，可以设置边车的服务对象数量的上限值，并在该边车当前的服务对象的数量不超过上限值的情况下，继续为该边车分配数据包，使得该边车对该数据包进行流量管理，从而在避免边车过载的同时，提高了边车的资源利用率。
在一种可能的实现方式中，连接控制模块，用于在第一边车失效后，从边车集群中选择第三边车或通知控制台在所述节点创建第三边车，将第一业务容器组发送的另一数据包转发至第三边车；第三边车，用于对第一业务容器组发出的另一数据包进行流量管理。
也就是说，在该实现方式中，第一边车失效后，可以重新为第一业务容器组选择第三边车，第三边车可以继续对第一业务容器组发出的数据包进行流量管理，从而保障了第一业务容器组业务的高可用能力。
在一种可能的实现方式中，第三边车是基于第一边车进行功能升级的新版本，或者第三边车是第一边车的复制版本。
也就是说，在该实现方式中，在第一边车失效后，可以为第一业务容器组选择与第一边车的功能相同的边车，或者选择比第一边车更高级的边车，以继续对第一业务容器组发出的数据包进行流量管理，从而保障了第一业务容器组业务的高可用能力。
在一种可能的实现方式中，第一边车，用于在对第一业务容器组发出的数据包进行流量管理之后，将数据包发送至后端容器组。
也就是说，在该实现方式中，第一边车可以将经过流量管理后的数据包发送至后端容器组，以调用后端容器组的服务或为后端容器组提供服务。
在一种可能的实现方式中，第一边车，还用于产生会话标识并发送会话标识至第一业务容器组和连接控制模块；连接控制模块，用于记录会话标识与后端容器组的对应关系；第三边车，用于从第一业务容器组获取会话标识并根据会话标识在连接控制模块记录的对应关系中确定后端容器组，在对第一业务容器组发出的另一数据包进行流量管理之后，将另一数据包发送至后端容器组。
也就是说，在该实现方式中，连接控制模块可以记录第一边车产生的会话标识和后端容器组的对应关系，第三边车可以根据该对应关系，将第一容器组发出的其他数据包进行流量管理后发送至该后端容器组，从而避免了同一容器组发出的不同数据包被不同边车发送至不同后端容器组的问题。
在一种可能的实现方式中，边车分配策略包括第三策略，第三策略用于指示边车集群中的边车的服务对象数量为0时被优先使用；连接控制模块，还用于确定第一边车的服务对象的数量，在第一边车的服务对象的数量为0的情况下将第一业务容器组发出的数据包转发至第一边车。
在一种可能的实现方式中，连接控制模块，还用于监控边车集群中每个边车的工作状态，在发现存在下线的边车时，发送下线的边车的信息至控制台。
也就是说，在该实现方式中，可以向控制台反馈下行的边车，以便控制台更新正在运行的边车，并据此制定边车分配策略，实现对容器组的有效管理。
在一种可能的实现方式中，流量管理包括：流量控制、流量安全以及流量观测。
在一种可能的实现方式中，节点为虚拟机、计算机或裸金属服务器。
第二方面，本申请实施例提供了一种容器组的管理系统，包括控制台和第一方面所提供的节点。
第三方面，本申请实施例提供了一种节点中的容器组的管理方法，节点运行有连接控制模块、边车集群、和第一业务容器组，边车集群包括至少两个边车，该方法包括：连接控制模块接收与节点相连的控制台发送的边车分配策略，根据边车分配策略在边车集群中选择第一边车，并将第一业务容器组发出的数据包转发至第一边车；第一边车对第一业务容器组发出的数据包进行流量管理。
在一种可能的实现方式中，节点还运行有第二业务容器组，该方法还包括：连接控制模块根据边车分配策略在边车集群中选择第二边车，并将第二业务容器组发出的数据包转发至第二边车；第二边车对所述第二业务容器组发出的数据包进行流量管理。
在一种可能的实现方式中，第一边车分配的硬件资源规格高于第二边车分配的硬件资源，边车分配策略包括第一策略，第一策略用于指示第一业务容器组优先使用第一边车；根据边车分配策略在边车集群中选择第一边车包括：根据第一策略在边车集群中选择第一边车。
在一种可能的实现方式中，节点还运行有第二业务容器组，边车分配策略还包括第二策略，第二策略用于指示第一边车的服务对象数量不超过上限值；方法还包括：连接控制模块确定第一边车的服务对象的数量，在数量不超过上限值的情况下将第二业务容器组发出的数据包转发至第一边车；第一边车对第一业务容器组发出的数据包和第二业务容器组发出的数据包同时进行流量管理。
在一种可能的实现方式中，该方法还包括：连接控制模块，在第一边车失效后，从边车集群中选择第三边车或通知控制台在节点创建第三边车，将第一业务容器组发送的另一数据包转发至第三边车；第三边车对第一业务容器组发出的另一数据包进行流量管理。
在一种可能的实现方式中，第三边车是基于第一边车进行功能升级的新版本，或者第三边车是第一边车的复制版本。
在一种可能的实现方式中，该方法还包括：第一边车在对第一业务容器组发出的数据包进行流量管理之后，将数据包发送至后端容器组。
在一种可能的实现方式中，该方法还包括：第一边车产生会话标识并发送会话标识至第一业务容器组和连接控制模块；连接控制模块记录会话标识与后端容器组的对应关系；第三边车从第一业务容器组获取会话标识并根据会话标识在连接控制模块记录的对应关系中确定后端容器组，在对第一业务容器组发出的另一数据包进行流量管理之后，将另一数据包发送至后端容器组。
在一种可能的实现方式中，边车分配策略包括第三策略，第三策略用于指示边车集群中的边车的服务对象数量为0时被优先使用；根据边车分配策略在边车集群中选择第一边车，并将第一业务容器组发出的数据包转发至第一边车包括：连接控制模块确定第一边车的服务对象的数量，在第一边车的服务对象的数量为0的情况下将所述第一业务容器组发出的数据包转发至所述第一边车。
在一种可能的实现方式中，该方法还包括：连接控制模块监控边车集群中每个边车的工作状态，在发现存在下线的边车时，发送下线的边车的信息至控制台。
在一种可能的实现方式中，流量管理包括：流量控制、流量安全以及流量观测。
第四方面，本申请实施例提供了一种运行容器组的节点，包括处理器和存储器，处理器用于执行存储器中存储的指令，以执行第二方面所提供的方法。
第五方面，本申请实施例提供了一种计算机可读存储介质，包括计算机程序指令，当计算机程序指令由计算设备集群执行时，计算设备集群执行如第一方面所提供的方法。
第六方面，本申请实施例提供了一种包含指令的计算机程序产品，当指令被计算机设备集群运行时，使得计算机设备集群执行如第一方面所提供的方法。
本申请实施例提供的运行容器组的节点，容器组的管理系统和方法，可以按照控制台发送的边车分配策略，从至少两个边车中为业务容器选择边车，并使用选择出的边车对该业务容器组发出的数据包进行流量管理，从而可以实现对该业务容器组的灵活管理，可以为对该业务容器组进行更优的流量管理，保障该业务容器组业务的高可用能力。
附图说明
图1为本申请实施例提供的一种容器组的管理系统的结构示意图；
图2为本申请实施例提供的一种运行容器组的节点的结构示意图；
图3A为本申请实施例提供的一种容器组管理框架示意图；
图3B为本申请实施例提供的一种容器组管理方法流程图；
图4A为本申请实施例提供的一种容器组管理框架示意图；
图4B为本申请实施例提供的一种容器组管理方法流程图；
图5A为本申请实施例提供的一种容器组管理框架示意图；
图5B为本申请实施例提供的一种容器组管理框架示意图；
图5C为本申请实施例提供的一种容器组管理方法流程图；
图6A为本申请实施例提供的一种容器组管理框架示意图；
图6B为本申请实施例提供的一种容器组管理框架示意图；
图6C为本申请实施例提供的一种容器组管理方法流程图；
图7为本申请实施例提供的一种容器组管理方法流程图；
图8为本申请实施例提供的一种节点的示意性框图。
具体实施方式
下面将结合附图，对本发明实施例中的技术方案进行描述。显然，所描述的实施例仅是本说明书一部分实施例，而不是全部的实施例。
在本说明书的描述中“一个实施例”或“一些实施例”等意味着在本说明书的一个或多个实施例中包括结合该实施例描述的特定特征、结构或特点。由此，在本说明书中的不同之处出现的语句“在一个实施例中”、“在一些实施例中”、“在其他一些实施例中”、“在另外一些实施例中”等不是必然都参考相同的实施例，而是意味着“一个或多个但不是所有的实施例”，除非是以其他方式另外特别强调。
其中，在本说明书的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，在本说明书实施例的描述中，“多个”是指两个或多于两个。
在本说明书的描述中，术语“第一”、“第二”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括一个或者更多个该特征。术语“包括”、“包含”、“具有”及它们 的变形都意味着“包括但不限于”，除非是以其他方式另外特别强调。
微服务架构为一种面向服务的架构(service oriented architecture，SOA)，其将复杂系统切分为多个小服务或者说应用。可以将该小服务或者说应用称为微服务。每个微服务负责实现一个独立的业务逻辑。微服务是围绕业务功能进行构建的，可以独立部署。微服务之间通过相互依赖，从而可以提供一系列的功能。微服务易于被理解和修改，带来了语言和框架选择灵活性。微服务可以运行在容器(container)中。其中，多个相互之间依赖性较高的容器可以构成一个容器组。例如，在K8S(kubernetes)系统中，多个容器可以封装到一个pod中，即容器组为pod。
在本申请实施例中，运行微服务的容器可称为业务容器。边车可通过容器或进程实现，在边车通过容器实现时，边车又可称为边车容器。
在微服务架构中，服务治理也称SOA治理(service oriented architecture governance，SOA governance)，是用来管理微服务架构的采用和实现的过程。在基于微服务架构的服务网格中，服务治理可由边车执行。其中，在K8S(Kubernetes)系统中，边车是设置有边车功能的容器，边车为业务容器提供流量服务治理功能。
在一个方案中，部署pod级别的边车。即，每个pod对应一个边车，该边车用于对该pod进行服务治理。在该方案中，不同的边车对应不同的pod。一个边车的故障，仅影响该边车对应的pod，而不影响其他pod。可以说，该方案有助于保障业务的高可用能力。但在该方案中，若部署的pod的数量较多，部署的边车也较多，使得边车所占用的计算资源是不可忽略的，可能会导致链路时延大等问题。
在另一个方案中，部署节点级别的边车。即一个节点部署一个边车，用于对该节点上的多个pod进行服务治理。在该方案中，边车的故障，会影响节点上的多个pod，从而对业务的高可用能力造成较大影响。
针对以上技术问题，本申请实施例提供了一种容器组的管理系统，如图1所示，该管理系统可以包括节点100、节点200等多个节点，还可以包括与该多个节点相连的控制台300。控制台300可以管理该多个节点，例如可以向多个节点中的任一节点发送边车分配策略，使得该节点可以根据边车分配策略为该节点中的业务容器组分配边车，以便对该业务容器组发出的数据包进行更优的流量管理，实现业务的高可用能力。
在一些实施例中，分配策略可以是节点所在数据中心的管理人员或租户在控制台300配置的，由此，可以实现对容器组的灵活管理。
接下来，以节点100为例，对本申请实施例涉及的图1所示管理系统中的节点进行示例介绍。
在一些实施例中，节点100可以为虚拟机(Virtual Machine，VM)。
在一些实施例中，节点100可以具有多种硬件组件，例如一个或多个处理器(例如中央处理单元(central processing unit，CPU))、一个或多个存储器等。节点100所具有的硬件组件可以赋予节点100数据计算以及数据存储能力。在一个例子中，节点100可以为计算机。在另一个例子中，节点100可以为裸金属服务器。
参阅图2，节点100可以运行有连接控制模块110、边车集群120以及业务容器组集群130。其中，边车集群120可以包括边车121、边车122、边车123等至少两个边车。示例性的，边车又称为sidecar。业务容器组集群130可以包括业务容器组131。业务容器组131可以包括一个或多个容器，其中每个容器可以运行有应用或者微服务。连接控制模块110可以接收控制台200发送的边车分配策略A。在业务容器组131具有流量管理需求时，连接控制 模块110可以根据该分配策略A在边车集群120中为业务容器组131选择边车，例如可以选择边车121。之后，连接控制模块110可以将业务容器组131发出的数据包转发至边车121。边车121可以对该数据包进行流量管理。
在一个例子中，流量管理具体可以为流量控制。其中，流量控制可以包括流量通过、流量拒绝、流量复制以及流量染色中的任一项或任意多项的组合。在一个例子中，流量管理具体可以为流量安全。其中，流量安全可以包括对数据包进行加密或解密等。在一个例子中，流量管理具体可以为流量观测。其中，流量观测可以包括在控制台300中绘制调用链等。
在一些实施例中，连接控制模块可以配置有边车列表。数据面代理列表可以包括处于运行状态的边车的信息。边车的信息可以包括边车的标识(例如，进程身份标识(identity，ID))、边车的监听端口。其中，边车的监听端口也可以称为边车的入方向端口。通过边车121的监听端口，业务容器组131发出的数据包可以被发送到边车121，从而使得边车121可以对该数据包进行流量治理。
在一些实施例中，边车121分配的硬件资源规格高于边车122分配的硬件资源，边车分配策略A包括策略A1，策略A1用于指示业务容器组131优先使用边车121；由此，连接控制模块110可以根据策略A1在边车集群120中选择边车121，使得边车121对容器组131发出的数据包进行服务治理(即流量管理)。该实施例具体实现过程将在下文实施例2中进行具体介绍，在此不再赘述。
在一些实施例中，继续参阅图2，业务容器组集群130还可以包括业务容器组132。在业务容器组132具有流量管理需求时，连接控制模块110还可以根据分配策略A，在边车集群120中选择边车122，并将业务容器组132发出的数据包转发至边车122。之后，边车122可以对该数据包进行流量管理。
在一些实施例中，边车集群120中的同一个边车可以同时对多个业务容器组发出的数据包进行流量管理。例如，可以设定业务容器组集群130还可以包括业务容器组132。边车分配策略A还包括策略A2，策略A2可以用于指示边车的服务对象数量不超过上限值。边车的服务对象是指接受业务容器组的流量管理服务的业务容器组。也就是说，若边车对业务容器组发出的数据包进行流量管理，那么该业务容器组是该边车的服务对象。边车的上限值是指边车在同一时刻，能够进行流量管理的业务容器组的最大数量。其中，连接控制模块可以确定边车121的服务对象的数量，并在边车121的服务对象的数量不超过边车121上限值的情况下，将业务容器组132发出的数据包转发至边车121。之后，边车121可以对业务容器组131发出的数据包和业务容器组132发出的数据包同时进行流量管理。
在一些实施例中，在边车失效后，例如边车发生崩溃(crash)或重启后，可以为该边车的服务对象重新选择边车，以便新选择的边车继续对失效的边车的服务对象继续进行流量管理。仍以边车121为例，如上所述，边车121失效前可以对业务容器组131发出的数据包进行流量管理。当边车121失效后，连接控制模块110可以为业务容器组131重新确定边车。重新确定的边车可以对业务容器组131发出数据包进行流量管理。示例性的，为业务容器组131重新确定边车，具体可以为连接控制模块110从边车集群120中重新选择为业务容器组131的数据包进行流量管理的边车。例如，可以设定边车集群120包括边车123，连接控制模块110从边车集群120中选择边车123，使边车123对业务容器组131发出的数据包进行流量管理。具体而言，可以将业务容器组131发出的数据包发送至边车123，使得边车123对该数据包进行流量管理。示例性的，为业务容器组131重新确定边车，具体可以为连接控制模块通过控制台300在节点100中创建边车123，然后，将业务容器组1131发出的数据包发 送至边车123，使得边车123可以对该数据包进行流量管理。其中，在节点100中创建边车123的具体过程将在下文的实施例1中进行具体介绍，在此不再赘述。
在一些实施例中，边车123可以在边车121的基础上进行功能升级的新版本边车。其中，边车的版本升级过程将在下文进行具体介绍，在此不再赘述。
在一些实施例中，边车123是边车121的复制版本。也就是说，控制台300可以再次在节点100中创建功能与边车121相同的边车。
在一些实施例中，边车121在对业务容器组131发出的数据包进行流量管理之后，将数据包发送至后端容器组。在一个例子中，后端容器组可以是其他节点，例如节点200，上的业务容器组。在一个例子中，后端容器组可以是节点100上的除业务容器组131之外的其他业务容器组。
在这些实施例的一个示例中，边车121还可以产生会话标识，并将该会话标识发送至业务容器组131和连接控制模块110。该会话标识可以是指业务容器组131和后端容器组之间的会话标识。连接控制模块110可以记录会话标识与后端容器组的对应关系。边车123在对业务容器组131发出的数据包进行流量管理时，可以从业务容器组131获取该会话标识，然后可以根据会话标识与后端容器组的对应关系以及从业务容器组131获取的会话标识，确定后端容器组，从而在边车123对业务容器组131发送的新的数据包进行流量管理之后，将新的数据包发送至后端容器组。其中，新的数据包可以是指在边车123根据会话标识与后端容器组的对应关系以及从业务容器组131获取的会话标识，确定后端容器组之后，业务容器组131发送的数据包。具体将在下文实施例4中进行具体介绍，在此不再赘述。
在一些实施例中，边车分配策略A可以包括策略A3，策略A3用于指示边车集群120中的边车的服务对象数量为0时被优先使用。连接控制模块110可以确定边车121的服务对象的数量，并且在边车121的服务对象的数量为0的情况下将业务容器组131发出的数据包转发至边车121。其中，服务对象是指边车所服务的业务容器组，具体可以参考下文对实施例2介绍的内容实现，在此不再赘述。
在一些实施例中，连接控制模块110还用于监控边车集群120中每个边车的工作状态，在发现存在下线的边车时，发送下线的边车的信息至控制台300。具体将在下文实施例1中进行具体介绍，在此不再赘述。
本申请实施例提供的节点可以运行多个边车，并根据分配策略，在多个边车中为业务容器组选择边车，然后，利用选择出的边车对业务容器组发出的数据包进行流量管理，从而可以对业务容器组的数据包进行更优的流量管理，可实现业务的高可用能力。
接下来，在具体实施例中，对本申请实施例提供的节点以及容器组管理方案进行具体说明。
实施例1
参阅图3A，本实施例提供了一种节点100，其可以运行有业务容器组集群130，包括业务容器组131、业务容器组132等多个业务容器组。节点100还运行有sidecar集群120，包括sidecar121、sidecar122、sidecar123等多个sidecar。示例性的，该多个sidecar中每个sidecar可以绑定不同的硬件资源，例如可以绑定一个或多个中央处理器(central processingunit，CPU)，还可以绑定其他计算模块(例如专门用于加密解密的硬件、图形处理器(graphics processing unit，GPU)等)。节点100还运行有连接管理模块110。示例性的，连接管理模块110可以包括协议栈111和sidecar启动管理器112。
微服务节点100可以接入网络，该网络可以包括后端容器组210、后端容器组220等多 个后端容器组。在一个示例中，后端容器组可以运行于指该网络中除节点100之外的其他节点。在另一个示例中，后端容器组可以运行于节点100。节点100中的业务容器组所发出的数据包在经过sidecar的流量管理后，可以发送到后端容器组，以调用后端容器组的服务或者为后端容器组提供服务。
其中，该网络还可以包括控制台300。其中，控制台300可以接收管理人员或者说运维人员的操作，并响应该操作对网络中的节点100以及其他节点进行控制。例如，控制台300可以向sidecar发送服务列表，以进行服务列表同步，使得sidecar可以根据服务列表进行流量管理。再例如，控制台300可以向连接管理模块110发送边车分配策略A，以便连接管理模块110根据边车分配策略A，在sidecar集群中为业务容器组选择sidecar。选择出的sidecar对业务容器组发出的数据包进行流量管理。
连接管理模块110可以采用多活方式管理节点100中的多个sidecar。即该多个sidecar可以同时处于运行状态或者说工作状态。
示例性的，同一sidecar可以为同时为多个业务容器组的链路提供流量管理。在一个例子中，如图3A所示，sidecar121可以同时为链路1311、链路1312以及链路1321提供流量管理。其中，链路1311和链路1312是源自业务容器组131的链路，链路1321是源自业务容器组132的链路。
示例性的，不同sidecar可以同时为同一业务容器组的不同链路提供流量管理。在一个例子中，如图3A所示，sidecar121可以为源自业务容器组132的链路1321提供流量管理。sidecar123可以为源自业务容器组132的链路1322提供流量管理。
其中，在本实施例中，为业务容器组的链路提供流量管理具体可以是指为业务容器组通过该链路发送的数据包提供流量管理。例如，对该数据包进流量控制、流量安全、流量观测等。其中，流量控制可以包括流量通过、流量拒绝、流量复制以及流量染色中的任一项或任意多项的组合。
继续参阅图3A，节点100还可以包括协议栈111，其可以用于在业务容器组产生连接请求时，从sidecar集群120中为该连接请求分配sidecar，从而建立该业务容器组和分配的sidecar之间的链路，使得分配的sidecar可以为该业务容器组通过该链路发送的数据包提供流量管理。
接下来，介绍为连接请求分配sidecar的方案。
协议栈111可以调用或者读取sidecar列表。示例性的，如图3A所示，sidecar列表可以设置在协议栈111中。sidecar列表可以包括处于运行状态的sidecar的信息。sidecar的信息可以包括sidecar的标识(例如，进程身份标识(identity，ID))、sidecar的版本信息、sidecar的监听端口。其中，sidecar的版本信息用于表征sidecar版本。在一个例子中，sidecar的版本信息可以为版本号。其中，版本号的大小代表了版本的高低。sidecar的监听端口也可以成为sidecar的入方向端口。通过sidecar的监听端口，连接请求可以被发送到该sidecar，从而可以建立该连接请求的源业务容器组和该sidecar之间的链路。
示例性的，sidecar的信息还可以包括sidecar的当前连接数。当前连接数用于表征sidecar当前所连接的链路的数量。以图3A所示的sidecar121为例，其连接链路1311、链路1312、链路1321，因此，当前连接数为3。
示例性的，sidecar的信息还可以包括sidecar当前连接业务容器组的标识等。业务容器组的标识用于唯一标识业务容器组，由此，协议栈111可以通过sidecar当前连接业务容器组的标识，确定sidecar当前所连接业务容器组。其中，sidecar连接业务容器组是指与 sidecar之间具有链路的业务容器组。例如，如图3A所示，业务容器组131属于sidecar121的当前连接业务容器组。
示例性的，sidecar的信息还可以包括sidecar绑定的硬件资源信息(例如CPU信息、是否开启硬件加速等信息)等。在其他实施例中，sidecar列表还可以包括其他信息，在此不再一一列举。
当业务容器组P产生连接请求P1时，该连接请求P1可以被发送至协议栈111。协议栈111可以读取sidecar列表中记录的sidecar信息，并根据sidecar信息，为该连接请求P1分配sidecar。接下来，进行具体说明。
在一个说明性示例中，协议栈111可以判断业务容器组P是否为sidecar当前连接业务容器组，即判断业务容器组P和sidecar列表中的sidecar之间已经存在了一条或多条链路。
若业务容器组P和sidecar列表中的sidecar之间已经存在了一条或多条链路。可以设定业务容器组P和sidecarB1之间已经存在了一条或多条链路。那么可以将sidecarB1分配给连接请求P1。进而，协议栈111可以将连接请求P1发送至sidecarB1。例如，协议栈111可以通过sidecarB1的监听端口发送连接请求P1，使得连接请求P1可以被发送至sidecarB1。sidecarB1在接收到连接请求P1时，可以根据连接请求P1，在sidecarB1和业务容器组P之间再建立一条链路。业务容器组P可以通过该链路向sidecarB1发送数据包。sidecarB1接收到该数据包后，可以对该数据包进行流量管理。
若业务容器组P和sidecar列表中的sidecar之间不存在链路，即业务容器组P不是sidecar列表中任一sidecar的当前连接业务容器组。可以根据sidecar列表中各sidecar的当前连接数，确定当前连接数最少的sidecar。可以设定sidecar列表中的sidecarB2的当前连接数最少。可以将sidecarB2分配给连接请求P1。sidecarB2在接收到连接请求P1时，根据基于连接请求P1，在sidecarB2和业务容器组P之间再建立一条链路。业务容器组P可以通过该链路向sidecarB2发送数据包。sidecarB2接收到该数据包后，可以对该数据包进行流量管理。
在一个说明性示例中，协议栈111在接收到连接请求P1时，可以判断sidecar列表中是否存在新版本的sidecar。具体而言，可以判断sidecar列表中的所有sidecar的版本是否相同。若sidecar列表中一个或多个sidecar的版本高于其他sidecar的版本，那么可以确定该一个或多个sidecar为新版本的sidecar。若sidecar列表中所有sidecar的版本均相同，那么可以确定sidecar列表中不存在新版本的sidecar。
当sidecar列表中存在新版本的sidecar时，可以将新版本的sidecar分配给连接请求P1。新版本的sidecar在接收到连接请求P1时，根据基于连接请求P1，在新版本的sidecar和业务容器组P之间再建立一条链路。业务容器组P可以通过该链路向新版本的sidecar发送数据包。新版本的sidecar接收到该数据包后，可以对该数据包进行流量管理。
在一个说明性示例中，可以对节点100中的sidecar进行热升级。即可以在用户不感知的情况下，完成sidecar的升级。
接下来，结合图3B，对实施例1提供的sidecar分配方案以及sidecar升级方案，进行更具体地说明。
运维人员可以在控制台300配置sidecar创建信息，用于在节点100中创建新的sidecar。其中，sidecar创建信息可以包括版本信息、硬件资源信息。
sidecar创建信息还可以包括被替换sidecar的信息，用于指示新创建的sidecar用于替换节点100中的哪些sidecar。其中，被替换sidecar可以是节点100中已处于运行状态 的sidecar。被替换sidecar的信息可以包括被替换sidecar的标识以及被替换sidecar的剩余运行时长等。示例性的，当被替换sidecar有多个时，可以为不同的被替换sidecar设置不同的剩余运行时长，以防止或缓解多个被替换sidecar同时下线而导致的连接请求突增的问题。另外，当sidecar创建信息包括被替换sidecar的信息时，该sidecar创建信息也可以称为sidecar升级信息，即该sidecar创建信息可用于升级sidecar。
经过上述配置后，控制台300可以执行步骤401，向sidecar启动管理器111321发送sidecar创建信息。
sidecar启动管理器112可以基于sidecar创建信息，创建新的sidecar。当sidecar创建信息包括版本信息时，sidecar启动管理器112创建符合版本信息的sidecar。即创建的新的sidecar的版本符合该版本信息。在创建sidecar的过程中，sidecar启动管理器112还可以执行步骤402，为新的sidecar分配监听端口。示例性的，sidecar启动管理器112可以根据节点100中监听端口的占用情况，为新的sidecar分配监听端口。在分配监听端口时，可以将监听端口号分配给新的sidecar，以完成监听端口的分配。sidecar启动管理器112可以执行步骤403，启动新的sidecar。如此，在sidecar侧创建了新的sidecar。
新的sidecar可以执行步骤404，新的sidecar可以进行初始化，进入运行状态。
在新的sidecar执行了步骤404之后，sidecar启动管理器112可以执行步骤407，更新sidecar列表，并将更新后的sidecar列表发送至协议栈111。其中，更新sidecar列表包括：将新的sidecar的信息(例如监听端口、版本信息、当前连接数、当前连接业务容器组的标识、硬件资源信息等信息)添加到sidecar列表中。示例性的，sidecar启动管理器112可以更新后的sidecar列表可以携带在Ebpf(extended BPF)map中，然后，向协议栈111发送Ebpfmap，从而将更新后的sidecar列表发送至协议栈111。
在一些实施例中，在新的sidecar执行了步骤404之后，sidecar启动管理器112可以执行步骤406，监听sidecar的运行状态。在一些实施例中，sidecar启动管理器112可以建立其和sidecar之间的域套接字(domain socket)长连接，并通过域套接字长连接监听sidecar的运行状态。并根据监听到的结果，执行步骤407以及执行步骤408。其中，在步骤408，sidecar启动管理器112可以向控制台300发送sidecar更新信息，其中包括新的sidecar的运行状态以及标识信息等。
另外，当sidecar创建信息包括被替换sidecar的信息时，在步骤404，新的sidecar在步骤404中，还可以启动域套接字(domain socket)监听。并且，新的sidecar还可以执行步骤405，新的sidecar通过域套接字连接被替换sidecar。新的sidecar可以通过其和被替换sidecar之间的域套接字，监听被替换sidecar的运行状态，并向被替换sidecar发送下线指令，以指示被替换sidecar下线。其中，当被替换sidecar的信息包括被替换sidecar的剩余运行时长时，新的sidecar在完成初始化后开始计时，当计时达到剩余运行时长时，新的sidecar可以执行步骤409，将被替换sidecar下线。具体而言，新的sidecar可以通过其和被替换sidecar之间的域套接字，向被替换sidecar发送下线指令。被替换sidecar可以响应该指令而下线。
另外，可以理解，节点100中的一个或多个sidecar可能会执行步骤410，发生崩溃(crash)或重启。其中，为方便描述，可以将sidecar的崩溃或重启，统称下线。
sidecar的下线可以导致sidecar和业务容器组之间的链路断开，以及导致sidecar和sidecar启动管理器112之间的域套接字断开。即在步骤411中，下线的sidecar和业务容器组之间的链路断开。在步骤412中，下线的sidecar和启动管理器112之间的域套接字断 开。当下线的sidecar和数据面启动管理器112之间的域套接字断开时，sidecar启动管理器112可以确认sidecar下线，进而执行步骤413，向控制台300发送sidecar更新信息，其中包括下线的sidecar的标识信息等。
sidecar启动管理器112在确认sidecar下线时，还可以执行步骤414，更新sidecar列表，并将更新后的sidecar列表发送至协议栈111。其中，更新sidecar列表可以包括将下线的sidecar从sidecar列表中删除或者列为不运行状态。在一些实施例中，协议栈111可以执行步骤415，清理下线sidecar的连接信息。该连接信息可以包括下线sidecar历史连接的业务容器组等信息。
当业务容器组和sidecar之间的链路断开(例如，因sidecar的下线而断开)时，业务容器组可以产生连接请求，以再次请求连接sidecar。具体而言，业务容器组可以在步骤416中，向协议栈111发送连接请求。为方便描述，可以将发出连接请求的业务容器组称为待连接业务容器组。协议栈111在接收到连接请求之后，可以执行步骤417，为待连接业务容器组选择sidecar。
在一些实施例中，可以通过如下规则，选择sidecar。
首先，判断sidecar列表中是否有存在新版本的sidecar。具体而言，可以判断sidecar列表中的所有sidecar的版本是否相同。若sidecar列表中一个或多个sidecar的版本高于其他sidecar的版本，那么可以确定该一个或多个sidecar为新版本的sidecar。当sidecar列表中存在新版本的sidecar时，可以将新版本的sidecar作为选择的sidecar。
当sidecar列表中不存在新版本的sidecar时，可以判断待连接业务容器组和sidecar列表中的sidecar之间是否已经存在了一条或多条链路。若待连接业务容器组和sidecar列表中的sidecarB1之间已经存在了一条或多条链路。那么可以将sidecarB1作为选择的sidecar。
若待连接业务容器组和sidecar列表中的sidecar之间不存在链路，那么可以从sidecar列表中确定当前连接数最少的sidecar。并将当前连接数最少的sidecar作为选择的sidecar。
通过上述规则，在选择出sidecar之后，协议栈111可以执行步骤418，将连接请求发送给选择的sidecar，以便在该sidecar和待连接业务容器组之间创建链路，使得该sidecar可以对业务容器组通过该链路发送的数据包进行流量管理。
在另一些实施例中，可以通过图2所示实施例所提供的规则，选择sidecar。
实施例1提供的节点可以支持sidecar的多实例运行，并可通过连接数对每个可用sidecar实例进行负载均衡。该节点还可以支持动态扩充sidecar实例及对已有实例热升级，新连接优先使用热升级后的sidecar，并控制被替换sidecar的下线时间窗口，降低瞬间连接切换导致的连接队列堆积问题。并且，在该节点中，协议栈与sidecar启动管理器可以自动管理每个sidecar的监听端口，当sidecar因本身问题导致的宕机时，该sidecar连接的业务容器组将被重新定位到其他可用sidecar上，无需人工干预，提升系统整体可用性。
实施例2
本申请实施例提供了一种节点100，其运行有sidecar集群120，可以包括多种类型的sidecar，其中，不同类型的sidecar的性能高低不同。可以设定sidecar集群120包括sidecar121和sidecar122，其中，sidecar121为高性能sidecar，sidecar122为低性能sidecar。
高性能sidecar的性能高于普通sidecar的性能。具体而言，相对于普通sidecar，高性能sidecar配置了更多的硬件资源，例如可以配置有更多数量的CPU，也可以配置有加速 硬件，等等。也就是说，高性能sidecar分配的硬件资源高于普通性能sidecar。由此，高性能sidecar比普通sidecar具有更强大的数据处理能力。当高性能sidecar为业务容器组的链路提供流量管理时，可以保障该业务容器组的服务质量(Quality of Service，QoS)。因此，对于具有服务质量要求的业务容器组，可以在其和高性能sidecar之间创建链路，使得高性能sidecar可以为该链路提供流量管理，以保障该业务容器组的服务质量。
运维人员可以通过控制台300配置优先级列表，优先级列表可以包括多种类型的连接请求。其中，不同类型的连接请求对应不同类型的sidecar。例如，节点100设置有高性能sidecar和普通sidecar。优先级列表可以包括第一类型的连接请求和第二类型的连接请求。其中，第一类型的连接请求对应高性能sidecar，第二类型的连接请求对应普通性能sidecar。
在一个例子中，每一类型的连接请求可以对应至少一种业务容器组标签(label)。不同类型的连接请求对应不同的业务容器组标签。也就是说，每一类型的连接请求可以通过业务容器组标签来表征。业务容器组标签用于表示一种类型的业务容器组。其中，当业务容器组为pod时，业务容器组标签为podlabel。优先级列表记录了连接请求对应的业务容器组标签和sidecar类型的对应关系。例如，一类型的连接请求对应业务容器组标签Q1和业务容器组标签Q2，并且该类型的连接请求对应高性能sidecar。那么，优先级记录列表可以记录业务容器组标签Q1和高性能sidecar的对应关系，也记录了业务容器组标签Q2和高性能sidecar的对应关系。
在一个例子中，每一类型的连接请求可以对应至少一种服务类型。不同类型的连接请求对应不同的服务类型。也就是说，每一类型的连接请求可以通过服务类型来表征。优先级列表记录了连接请求对应的服务类型和sidecar类型的对应关系。例如，一类型的连接请求对应服务类型S，并且该类型的连接请求对应高性能sidecar。那么，优先级记录列表可以记录服务类型S和高性能sidecar的对应关系。
参阅图4A，节点100还运行有连接控制模块110。连接控制模块110可以包括协议栈111和控制面代理113。
参阅图4A，控制台300可以将配置的优先级列表发送至控制面代理113。控制面代理113可以保存该优先级列表，以及将优先级列表发送至协议栈111。示例性的，控制面代理可以通过Ebpfmap将优先级列表发送至协议栈111。
协议栈111还配置有sidecar列表，该sidecar列表记录有节点100的sidecar的类型、sidecar监听端口等信息。其中，sidecar列表的获取以及更新方式可以参考上文对实施例1的介绍。其中，与实施例1不同的是，在实施例2中可以预先配置sidecar标识和sidecar类型的对应关系。由此，协议栈111可以根据sidecar标识和sidecar类型的对应关系，将sidecar的类型记录到sidecar列表中。
协议栈111还可以配置有业务容器组标签和业务容器组标识的对应关系。在一个例子中，业务容器组标识可以为cgroup.ns(control group namespace)。其中，业务容器组标识用于表示一个业务容器组，业务容器组标签用于表示一种类型的业务容器组。也就是说，多个业务容器组具有多个业务容器组标识，其中，多个业务容器组和多个业务容器组标识一一对应。当该多个业务容器组为同一类型的业务容器组时，该多个业务容器组的业务容器组标签相同。其中，业务容器组标签与业务容器组标识的对应关系的确定方式将在下文对图4B所示实施例的介绍，在此不再赘述。
在一个说明性示例中，优先级列表记录了业务容器组标签和sidecar类型的对应关系。当协议栈111接收到业务容器组，例如业务容器组131，发送的连接请求时，协议栈111可 以根据该连接请求的源地址(例如源IP地址)，确定业务容器组131的业务容器组标识。然后，可以根据业务容器组131的业务容器组标识，以及业务容器组标签和业务容器组标识的对应关系，确定业务容器组131的业务容器组标签。在确定业务容器组131的业务容器组标签后，可以根据优先级列表，确定业务容器组131的业务容器组标签对应的sidecar类型。然后，协议栈111将该连接请求发送至确定出的sidecar类型的数据代理。例如，可以设定业务容器组131的业务容器组标签对应的sidecar类型为高性能sidecar，则协议栈111可以将来自业务容器组131的连接请求发送至高性能sidecar。使得高性能sidecar为来自业务容器组131的数据包提供流量管理。
在一个说明性示例中，优先级列表记录了服务类型和sidecar类型的对应关系。当协议栈111接收到业务容器组，例如业务容器组132，发送的连接请求时，协议栈111可以确定该连接请求的目标服务。然后，可以根据该目标服务的服务类型以及优先级列表，确定该目标服务对应的sidecar类型。然后，协议栈111可以将该连接请求发送至确定出的sidecar类型的数据代理。例如，可以设定业务容器组132发送的连接请求的目标服务对应的sidecar类型为普通能sidecar，则协议栈111可以将来自业务容器组132的连接请求发送至普通性能sidecar。使得普通性能sidecar为来自业务容器组132的数据包提供流量管理。
接下来，结合图4B，对实施例2提供的方案进行更具体地说明。
参阅图4B，节点100中的业务容器组监听模块(例如，pod监听模块(monitor))可以通过步骤501对业务容器组创建模块进行监听，以监听业务容器组创建。示例性的，业务容器组创建模块具体可以为kubelet。业务容器组创建模块在步骤502中，可以创建业务容器组。如此，业务容器组监听模块通过步骤503，可以检测到业务容器组创建事件。通过业务容器组创建事件，业务容器组监听模块可以获取创建的业务容器组的业务容器组标识和业务容器组标签。之后，业务容器组监听模块可以执行步骤504，建立业务容器组标签和业务容器组标识的对应关系。业务容器组监听模块可以通过步骤505，将业务容器组标签和业务容器组标识的对应关系发送至协议栈111。示例性的，业务容器组监控模块可以通过Ebpf map将业务容器组标签和业务容器组标识的对应关系发送至协议栈111。
运维人员可以在控制台300配置优先级列表。优先级列表具体可以参考上文所述，在此不再赘述。控制台300可以通过步骤507，向控制面代理113发送优先级列表。控制面代理113可以通过步骤508，向协议栈111发送优先级列表。协议栈111可以通过步骤509，保存优先级列表。
协议栈111可以通过步骤510，接收业务容器组发送的连接请求。为方便表述，可以将发送连接请求的业务容器组称为待连接业务容器组。协议栈111可以解析连接请求的源地址，并在步骤511中根据连接请求的源地址，确定业务容器组标识。接着，协议栈111可以在步骤512中，根据确定的业务容器组标识，确定业务容器组标签。进而，协议栈111可以在步骤513中，根据优先级列表，确定业务容器组标签对应的sidecar类型，从而可以确定待连接业务容器组对应的sidecar类型。
协议栈111可以在步骤514中，为待连接业务容器组选择sidecar。
若优先级列表为空，即没有配置连接请求和sidecar类型的对应关系。则从sidecar列表中，选择当前连接数最少的sidecar。
若高性能sidecar的当前连接数为零，且待连接业务容器组不对应高性能sidecar，则从sidecar列表中，选择当前连接数最少的sidecar。也就是说，在高性能sidecar没有连接业务容器组的情况系，即使待连接业务容器组对不对应高性能sidecar，也可以允许高性 能sidecar为待连接业务容器组提供流量管理，从而可以在高性能sidecar空闲时，提升sidecar整体资源利用率。
若待业务容器组对应高性能sidecar，则选择高性能sidecar。
之后，协议栈111可以执行步骤515，将连接请求发送至选择的sidecar。以便在该sidecar和待连接业务容器组之间创建链路，使得该sidecar可以对业务容器组通过该链路发送的数据包进行流量管理。
实施例2提供的方案在实施例1支持多活高可用的情况下，支持使用不同性能的sidecar对不同类型的业务容器组或目标服务，进行流量管理，从而保证某些对服务质量要求较高的连接请求使用独立或较多硬件资源，并可以在高性能sidecar空闲时，提升sidecar的整体资源利用率。
实施例3
在服务网格系统中，sidecar需要感知系统中的服务变更，并获取因服务变更而发生更新的服务列表。其中，服务变更可以包括服务上线、服务下线等。
在一种方案中，参阅图5A，服务网格系统中的节点M、节点N等各节点的sidecar和服务网格系统中的控制面之间建立套接字(socket)长连接。在后端容器组发生服务变更时，控制面可以产生服务变更消息。该服务变更消息可用于描述后端容器组发生的服务变更，或者服务变更消息包括因后端容器组发生的服务变更而发生更新的服务列表。控制面通过其和各个sidecar之间的长连接，将服务变更消息发送至各sidecar。其中，后端容器组可以是指能够响应节点M或节点N的服务调用而提供相应服务的容器组。
在该方案中，若服务网格系统中节点数量的增加，控制面的连接数将大大增加。服务网格系统中的每个服务的下线或下线均可引起服务变更，进而使得控制面产生服务变更消息。其中，控制面需要轮询其连接的所有sidecar，并进行服务变更消息的发送。当控制面的连接数非常大时，轮询sidecar以及发送服务变更消息将消耗控制面大量的计算资源并对网络形成较大压力。另外，控制面是按照sidecar的启动顺序或某个随机顺序，进行轮询和服务变更消息发送的。这可能导致同一节点内不同sidecar接收服务变更消息的时间相差较大。进而导致同一节点内sidecar感知到的服务实例不一致的时间窗口变大，如此，可能引发业务异常。
并且控制面的套接字文件描述符数量也有限。
因此，在该方案中，上述限制导致了服务网格系统难以扩大规模。
本实施例提供了一种方案，参阅图5B，控制台300可与各节点的控制面代理建立连接(例如套接字长连接)。如此，控制台300可以将服务变更消息发送至各节点的控制面代理。例如，控制面将服务变更消息发送至节点100的控制面代理113和节点200的控制面代理213。然后，控制面代理113可以将服务变更消息发送至节点100中的sidecar121、sidecar122等各个sidecar。控制面代理213可以将服务变更消息发送至节点200中的sidecar221、sidecar222等各个sidecar。
如此，控制面的连接数将大大减少。并且当服务网格中发生服务变更时，控制面在节点层级上进行轮询和服务变更消息的发送，降低了控制面计算资源的消耗以及对网络的压力。并且，节点的控制面代理接收到服务变更消息后，通过节点内部通信机制，将服务变更消息发送至该节点中各sidecar，如此，大大降低了同一节点内sidecar所感知到的服务实例不一致的时间窗口。
另外，在本实施例中，当控制面和控制代理之间的网络发生故障时，控制面代理可以作 为离线服务中心，为所在节点内的sidecar提供的读取功能，实现sidecar和控制侧的通信。其中，控制侧包括控制面代理和控制面。
接下来，结合图5C，对实施例3提供的方案进行更具体地说明。
参阅图5C，控制台300可以通过步骤601，对后端容器组进行服务变更监听。并且在步骤602中，建立控制台300和节点100内的控制面代理113之间的连接。该连接可以为套接字长连接。在步骤603中，控制面代理113可以和其所在节点内的sidecar建立连接。
当控制台300在感知到后端容器组发生服务变更时，可以在步骤604中产生服务变更消息，并通过步骤605，将服务变更消息发送至控制面代理。
控制面代理在接收到服务变更消息后，可以通过步骤606，向sidecar发送服务变更消息。sidecar在步骤607中，可以根据服务变更消息更新路由信息。
实施例4
客户端第一次向服务端请求会话(session)时，服务端会为客户端创建一个会话，并通过特殊算法算出的一个会话标识(session identity，sessionID)，用于标识该会话。服务端可以向客户端返回该会话标识。客户端可以将会话标识保存在本地cookie中。当客户端再次访问对服务端时，可将会话标识发送至服务端。服务端会再次使用该会话标识对应的会话为客户端提供相应服务。
其中，在服务网格系统中，客户端为可调用服务的业务容器组。服务端也可以称为后端容器组，其可以为业务容器组提供的服务。服务端可以包括多个服务实例，并使用其中的一个或多个服务实例为客户端提供服务。
参考图6A，基于实施例1提供的方案，业务容器组C可以通过链路C1和sidecar M1连接，通过链路C2和sidecar M2连接。业务容器组C可以通过链路C1发送对后端容器组的连接请求，该连接请求可以携带会话标识E。sidecar M1接收到该连接请求后，可以提取其中的会话标识E，然后根据会话标识E，使用哈希(hash)算法，确定路由策略，以便按照该路由策略，将连接请求发送至服务实例。例如，可以设定该路由策略将该连接请求发送至后端容器组中的服务实例D1。
同理，业务容器组C也可以通过链路C2发送对后端容器组的连接请求，该连接请求可以携带会话标识E。sidecar M2接收到该连接请求后，可以提取其中的会话标识E，然后根据会话标识E，使用哈希(hash)算法，确定路由策略，以便按照该路由策略，将连接请求发送至服务实例。例如，可以设定该路由策略将该连接请求发送至后端容器组中的服务实例D2。
可以理解，在后端容器组发送服务变更时，不同sidecar接收服务变更消息的时间不一致，可能导致不同sidecar在进行哈希计算时，采用的哈希环大小不一致。例如，sidecar M1在接收到服务变更消息之前，进行哈希计算；sidecar M2在接收到服务变更消息之后，进行哈希计算；那么sidecar M1和sidecar M2分别进行哈希计算时，所采用的哈希环大小不一致，从而产生不同的路由策略。也就是说，业务容器组C1发起的不同连接请求，可能会被发送至不同的服务实例，进而可能引发业务异常。
实施例4提供了一种方案，在不同sidecar为同一业务容器组的不同链路提供流量管理的情况下，可以将该业务容器组通过不同链路发生的连接请求，发送至同一服务实例。接下来，结合图4B和图4C，对该方案进行示例说明。
该方案可以应用于图6B所示的容器组管理系统，包括节点100、后端容器组210。其中，节点100可以包括sidecar121和sidecar122。示例性的，节点100还可以包括连接控制模块110。业务容器组131可以通过链路1311和sidecar121连接，通过链路1313和sidecar122 连接。后端容器组可以配置有服务实例G1、服务实例G2等多个服务实例。
参阅图6B，节点100可以配置有会话标识F和服务实例的映射关系。其中，当一sidecar，例如sidecar121，根据会话标识F，进行哈希计算，得到的路由策略所对应的服务实例时，sidecar121可以建立会话标识F和该服务实例的映射关系。sidecar121可以向sidecar122分享该映射关系。示例性的，sidecar121可向连接控制模块110发送该映射关系。sidecar122可以读取连接控制模块110中的会话标识F和服务实例的映射关系，也可以将该映射关系保存在本地缓存中。
当sidecar121接收到业务容器组131通过链路1311发送的包含会话标识F的连接请求时，可以按照会话标识F和服务实例的对应关系，确定服务实例。进而将该连接请求路由到该服务实例。当sidecar122接收到业务容器组131通过链路1313发送的包含会话标识F的连接请求时，可以按照会话标识F和服务实例的对应关系，确定服务实例。进而将该连接请求路由到该服务实例。如此，在不同sidecar为同一业务容器组的不同链路提供流量管理的情况下，可以将该业务容器组通过不同链路发生的连接请求，发送至同一服务实例。
接下来，结合图6C，对实施例4提供的方案进行更具体地说明。
如图6C所示，业务容器组131可以通过步骤701向后端容器组210发送会话请求。后端容器组210可以在步骤702中，响应该会话请求为业务容器组131分配会话标识F，并通过步骤703，将会话标识F发送至业务容器组131。业务容器组131在步骤704中，可以保存会话标识F。
业务容器组131可以通过步骤705，向sidecar121发送包含会话标识F的连接请求。
在一个说明性示例中，sidecar121在接收到包含会话标识F的连接请求时，可以通过步骤706a，在连接控制模块110中查询会话标识F对应的服务实例。
在一个说明性示例中，sidecar121在接收到包含会话标识F的连接请求时，可以通过步骤706b，在sidecar121的本地缓存中查询会话标识F对应的服务实例。也就是说，在该示例中，会话标识F和服务实例的对应关系可以被保存在sidecar121的本地缓存中。
业务容器组131可以通过步骤707，向sidecar122发送包含会话标识F的连接请求。
在一个说明性示例中，sidecar122在接收到包含会话标识F的连接请求时，可以通过步骤708a，在连接控制模块110中查询会话标识F对应的服务实例。
在一个说明性示例中，sidecar122在接收到包含会话标识F的连接请求时，可以通过步骤708b，在sidecar122的本地缓存中查询会话标识F对应的服务实例。也就是说，在该示例中，会话标识F和服务实例的对应关系可以被保存在sidecar122的本地缓存中。
继续参阅图6C，sidecar121可以执行步骤709，若未查询到服务实例，则根据会话标识F进行哈希运算，得到对应于会话标识F的服务实例。也就是说，通过上述步骤706a和步骤706b，均未查询到对应于会话标识F的服务实例。这说明在sidecar122执行步骤706a时，连接控制模块110中没有保存会话标识F和服务实例的对应关系；以及sidecar121在执行步骤706b时，sidecar121本地缓存中没有保存会话标识F和服务实例的对应关系。如此，sidecar121根据会话标识进行哈希运算，以得到对应于会话标识F的服务实例。
接着，sidecar121可以执行步骤710，创建会话标识F和服务实例的映射关系，并向连接控制模块110发送会话标识F和服务实例的映射关系。连接控制模块110在步骤711中，可以保存会话标识F和服务实例的映射关系。示例性的，sidecar121可以执行步骤712，更新本地缓存，以将会话标识F和服务实例的映射关系保存在本地缓存中。
并且，sidecar121可以执行步骤713，根据服务实例，创建连接。即将连接请求发送至 对应于会话标识F的服务实例，以建立业务容器组131和该服务实例之间的连接。
继续参阅图6C，sidecar122可以执行步骤714，若未查询到服务实例，则根据会话标识F进行哈希运算，得到对应于会话标识F的服务实例。也就是说，通过上述步骤708a和步骤708b，均未查询到对应于会话标识F的服务实例。这说明在sidecar122执行步骤708a时，连接控制模块110中没有保存会话标识E和服务实例的对应关系；以及sidecar122在执行步骤708b时，sidecar122本地缓存中没有保存会话标识F和服务实例的对应关系。如此，sidecar122根据会话标识进行哈希运算，以得到对应于会话标识F的服务实例。
接着，sidecar122可以执行步骤715，创建会话标识F和服务实例的映射关系，并向连接控制模块110发送会话标识F和服务实例的映射关系。
连接控制模块110可以执行步骤716，确定连接控制模块110中已存在会话标识F和服务实例的映射关系(在上述步骤711中保存了会话标识F和服务实例的映射关系)，不再根据sidecar122发送的会话标识E和服务实例的映射关系进行更新。连接控制模块110还可以执行步骤717，将在步骤711中保存的会话标识F和服务实例的映射关系发送至sidecar122。sidecar122可以执行步骤718，更新本地缓存，以将从连接控制模块110接收的会话标识F和服务实例的映射关系保存在本地缓存中。
sidecar122可以根据从连接控制模块110接收的会话标识F和服务实例的映射关系，确定对应于会话标识F的服务实例。然后，sidecar122可以执行步骤719，根据服务实例，创建连接。即将连接请求发送至对应于会话标识F的服务实例，以建立业务容器组131和该服务实例之间的连接。
另外，在一个说明性示例中，可以配置会话标识的生命周期。连接控制模块110可以在会话标识的生命周期结束时，确定该会话标识过期，并且执行步骤720，清理过期的会话标识。其中，在步骤720中，具体清理过期的会话标识和服务实例的映射关系。连接控制模块110可以执行步骤721a，向sidecar121发送会话标识清理命令。sidecar121可以响应于会话标识清理命令，执行步骤722a，清理缓存，以将过期的会话标识和服务实例的映射关系从本地缓存中清理掉。连接控制模块110可以执行步骤721b，向sidecar122发送会话标识清理命令。sidecar122可以响应于会话标识清理命令，执行步骤722b，清理缓存，以将过期的会话标识和服务实例的映射关系从本地缓存中清理掉。
由此，通过实施例4提供的方案，在不同sidecar为同一业务容器组的不同链路提供流量管理的情况下，可以将该业务容器组通过不同链路发生的连接请求，发送至同一服务实例，进而使得该服务实例可以通过不同的链路为该业务容器组提供服务。
接下来，基于上文所描述的容器组管理方案，对本申请实施例提供的一种节点中的容器组的管理方法进行介绍。可以理解的是，该方法是上文所描述的容器组管理方案的另一种表达方式，两者是相结合的。该方法是基于上文所描述的容器组管理方案提出，该方法中的部分或全部内容可以参见上文对容器组管理的描述。
该节点运行有连接控制模块、边车集群、和第一业务容器组，边车集群包括至少两个边车。参阅图7，该方法包括如下步骤。
步骤7001，连接控制模块接收与节点相连的控制台发送的边车分配策略，根据边车分配策略在边车集群中选择第一边车，并将第一业务容器组发出的数据包转发至第一边车。
步骤7002，第一边车对第一业务容器组发出的数据包进行流量管理。
在一些实施例中，该节点还运行有第二业务容器组，该方法还包括：连接控制模块根据所述边车分配策略在所述边车集群中选择第二边车，并将所述第二业务容器组发出的数据包 转发至所述第二边车；所述第二边车对所述第二业务容器组发出的数据包进行流量管理。
在这些实施例的一个示例中，所述第一边车分配的硬件资源规格高于所述第二边车分配的硬件资源，所述边车分配策略包括第一策略，所述第一策略用于指示所述第一业务容器组优先使用所述第一边车；所述根据所述边车分配策略在边车集群中选择所述第一边车包括：根据所述第一策略在边车集群中选择所述第一边车。
在一些实施例中，所述节点还运行有第二业务容器组，所述边车分配策略还包括第二策略，所述第二策略用于指示所述第一边车的服务对象数量不超过上限值；所述方法还包括：所述连接控制模块确定所述第一边车的服务对象的数量，在所述数量不超过所述上限值的情况下将所述第二业务容器组发出的数据包转发至所述第一边车；所述第一边车对所述第一业务容器组发出的数据包和所述第二业务容器组发出的数据包同时进行流量管理。
在一些实施例中，所述方法还包括：所述连接控制模块，在所述第一边车失效后，从所述边车集群中选择第三边车或通知所述控制台在所述节点创建所述第三边车，将所述第一业务容器组发送的另一数据包转发至所述第三边车；所述第三边车对所述第一业务容器组发出的另一数据包进行流量管理。
在这些实施例的一个示例中，所述第三边车是基于所述第一边车进行功能升级的新版本，或者所述第三边车是所述第一边车的复制版本。
在这些实施例的另一个示例中，所述方法还包括：所述第一边车在对所述第一业务容器组发出的数据包进行流量管理之后，将所述数据包发送至后端容器组。
在该示例的一个例子中，所述方法还包括：所述第一边车产生会话标识并发送所述会话标识至所述第一业务容器组和所述连接控制模块；所述连接控制模块记录所述会话标识与所述后端容器组的对应关系；所述第三边车从所述第一业务容器组获取所述会话标识并根据所述会话标识在所述连接控制模块记录的所述对应关系中确定所述后端容器组，在对所述第一业务容器组发出的另一数据包进行流量管理之后，将所述另一数据包发送至后端容器组。
在一些实施例中，所述边车分配策略包括第三策略，所述第三策略用于指示所述边车集群中的边车的服务对象数量为0时被优先使用；所述根据所述边车分配策略在边车集群中选择所述第一边车，并将所述第一业务容器组发出的数据包转发至所述第一边车包括：所述连接控制模块确定所述第一边车的服务对象的数量，在所述第一边车的服务对象的数量为0的情况下将所述第一业务容器组发出的数据包转发至所述第一边车。
在一些实施例中，所述方法还包括：所述连接控制模块监控所述边车集群中每个边车的工作状态，在发现存在下线的边车时，发送下线的所述边车的信息至所述控制台。
在一些实施例中，所述流量管理包括：流量控制、流量安全以及流量观测。
本申请实施例提供的容器组管理方法，可以按照控制台发送的边车分配策略，从至少两个边车中为业务容器选择边车，并使用选择出的边车对该业务容器组发出的数据包进行流量管理，从而可以实现对该业务容器组的灵活管理，可以为对该业务容器组进行更优的流量管理，保障该业务容器组业务的高可用能力。
值得注意的是，在本申请实施例中，边车也可以对从节点外或节点内其他业务容器组发送至与边车绑定的业务容器组的数据包进行流量处理，流量处理的策略也可以参照以上方式进行设置，本申请实施例对此不作限定。
参阅图8，本申请实施例还提供了一种节点800，包括处理器810和存储器820，处理器810用于执行存储器820中存储的指令，使得节点800可以执行如图7所示的方法。
本申请实施例还提供了一种计算机可读存储介质，包括计算机程序指令，当所述计算机 程序指令由计算设备集群执行时，所述计算设备集群执行如图7所示的方法。
本申请实施例还提供了一种包含指令的计算机程序产品，当所述指令被计算机设备集群运行时，使得所述计算机设备集群执行如图7所示的方法。
可以理解的是，本申请的实施例中的处理器可以是中央处理单元(central processing unit，CPU)，还可以是其他通用处理器、数字信号处理器(digital signal processor，DSP)、专用集成电路(application specific integrated circuit，ASIC)、现场可编程门阵列(field programmable gate array，FPGA)或者其他可编程逻辑器件、晶体管逻辑器件，硬件部件或者其任意组合。通用处理器可以是微处理器，也可以是任何常规的处理器。可以理解的是，在本申请的实施例中涉及的各种数字编号仅为描述方便进行的区分，并不用来限制本申请的实施例的范围。

Claims (27)

  1. 一种运行容器组的节点，其特征在于，所述节点运行有连接控制模块、边车集群、和第一业务容器组，所述边车集群包括至少两个边车，其中，
    所述连接控制模块，用于接收与所述节点相连的控制台发送的边车分配策略，根据所述边车分配策略在边车集群中选择第一边车，并将所述第一业务容器组发出的数据包转发至所述第一边车；
    所述第一边车，用于对所述第一业务容器组发出的数据包进行流量管理。
  2. 根据权利要求1所述的节点，其特征在于，还包括第二业务容器组；
    所述连接控制模块，还用于根据所述边车分配策略在所述边车集群中选择第二边车，并将所述第二业务容器组发出的数据包转发至所述第二边车；
    所述第二边车，用于对所述第二业务容器组发出的数据包进行流量管理。
  3. 根据权利要求2所述的节点，其特征在于，所述第一边车分配的硬件资源规格高于所述第二边车分配的硬件资源，所述边车分配策略包括第一策略，所述第一策略用于指示所述第一业务容器组优先使用所述第一边车；
    所述连接控制模块，具体用于根据所述第一策略在所述边车集群中选择所述第一边车。
  4. 根据权利要求1至3任一项所述的节点，其特征在于，还包括第二业务容器组，所述边车分配策略还包括第二策略，所述第二策略用于指示所述第一边车的服务对象数量不超过上限值；
    所述连接控制模块，还用于确定所述第一边车的服务对象的数量，在所述第一边车的服务对象的数量不超过所述上限值的情况下将所述第二业务容器组发出的数据包转发至所述第一边车；
    所述第一边车，还用于对所述第一业务容器组发出的数据包和所述第二业务容器组发出的数据包同时进行流量管理。
  5. 根据权利要求1至4任一项所述的节点，其特征在于，
    所述连接控制模块，用于在所述第一边车失效后，从所述边车集群中选择第三边车或通知所述控制台在所述节点创建所述第三边车，将所述第一业务容器组发送的另一数据包转发至所述第三边车；
    所述第三边车，用于对所述第一业务容器组发出的另一数据包进行流量管理。
  6. 根据权利要求5所述的节点，其特征在于，所述第三边车是基于所述第一边车进行功能升级的新版本，或者所述第三边车是所述第一边车的复制版本。
  7. 根据权利要求5或6所述的节点，其特征在于，
    所述第一边车，用于在对所述第一业务容器组发出的数据包进行流量管理之后，将所述数据包发送至后端容器组。
  8. 根据权利要求7所述的节点，其特征在于，
    所述第一边车，还用于产生会话标识并发送所述会话标识至所述第一业务容器组和所述连接控制模块；
    所述连接控制模块，用于记录所述会话标识与所述后端容器组的对应关系；
    所述第三边车，用于从所述第一业务容器组获取所述会话标识并根据所述会话标识在所述连接控制模块记录的所述对应关系中确定所述后端容器组，在对所述第一业务容器组发出的另一数据包进行流量管理之后，将所述另一数据包发送至后端容器组。
  9. 根据权利要求1或2所述的节点，其特征在于，所述边车分配策略包括第三策略，所述第三策略用于指示所述边车集群中的边车的服务对象数量为0时被优先使用；
    所述连接控制模块，还用于确定所述第一边车的服务对象的数量，在所述第一边车的服务对象的数量为0的情况下将所述第一业务容器组发出的数据包转发至所述第一边车。
  10. 根据权利要求1至9任一项所述的节点，其特征在于，
    所述连接控制模块，还用于监控所述边车集群中每个边车的工作状态，在发现存在下线的边车时，发送下线的所述边车的信息至所述控制台。
  11. 根据权利要求1至10任一项所述的节点，其特征在于，所述流量管理包括：流量控制、流量安全以及流量观测。
  12. 根据权利要求1至11任一项所述的节点，其特征在于，所述节点为虚拟机、计算机或裸金属服务器。
  13. 一种容器组的管理系统，其特征在于，包括控制台和权利要求1至11所述的节点。
  14. 一种节点中的容器组的管理方法，其特征在于，所述节点运行有连接控制模块、边车集群、和第一业务容器组，所述边车集群包括至少两个边车，所述方法包括：
    所述连接控制模块接收与所述节点相连的控制台发送的边车分配策略，根据所述边车分配策略在边车集群中选择第一边车，并将所述第一业务容器组发出的数据包转发至所述第一边车；
    所述第一边车对所述第一业务容器组发出的数据包进行流量管理。
  15. 根据权利要求14所述的方法，其特征在于，所述节点还运行有第二业务容器组，所述方法还包括：
    所述连接控制模块根据所述边车分配策略在所述边车集群中选择第二边车，并将所述第二业务容器组发出的数据包转发至所述第二边车；
    所述第二边车对所述第二业务容器组发出的数据包进行流量管理。
  16. 根据权利要求15所述的方法，其特征在于，所述第一边车分配的硬件资源规格高于所述第二边车分配的硬件资源，所述边车分配策略包括第一策略，所述第一策略用于指示所述第一业务容器组优先使用所述第一边车；
    所述根据所述边车分配策略在边车集群中选择所述第一边车包括：根据所述第一策略在边车集群中选择所述第一边车。
  17. 根据权利要求14至16任一项所述的方法，其特征在于，所述节点还运行有第二业务 容器组，所述边车分配策略还包括第二策略，所述第二策略用于指示所述第一边车的服务对象数量不超过上限值；所述方法还包括：
    所述连接控制模块确定所述第一边车的服务对象的数量，在所述数量不超过所述上限值的情况下将所述第二业务容器组发出的数据包转发至所述第一边车；
    所述第一边车对所述第一业务容器组发出的数据包和所述第二业务容器组发出的数据包同时进行流量管理。
  18. 根据权利要求14至17任一项所述的方法，其特征在于，所述方法还包括：
    所述连接控制模块，在所述第一边车失效后，从所述边车集群中选择第三边车或通知所述控制台在所述节点创建所述第三边车，将所述第一业务容器组发送的另一数据包转发至所述第三边车；
    所述第三边车对所述第一业务容器组发出的另一数据包进行流量管理。
  19. 根据权利要求18所述的方法，其特征在于，所述第三边车是基于所述第一边车进行功能升级的新版本，或者所述第三边车是所述第一边车的复制版本。
  20. 根据权利要求18或19所述的方法，其特征在于，所述方法还包括：
    所述第一边车在对所述第一业务容器组发出的数据包进行流量管理之后，将所述数据包发送至后端容器组。
  21. 根据权利要求20所述的方法，其特征在于，所述方法还包括：
    所述第一边车产生会话标识并发送所述会话标识至所述第一业务容器组和所述连接控制模块；
    所述连接控制模块记录所述会话标识与所述后端容器组的对应关系；
    所述第三边车从所述第一业务容器组获取所述会话标识并根据所述会话标识在所述连接控制模块记录的所述对应关系中确定所述后端容器组，在对所述第一业务容器组发出的另一数据包进行流量管理之后，将所述另一数据包发送至后端容器组。
  22. 根据权利要求14或15所述的方法，其特征在于，所述边车分配策略包括第三策略，所述第三策略用于指示所述边车集群中的边车的服务对象数量为0时被优先使用；
    所述根据所述边车分配策略在边车集群中选择所述第一边车，并将所述第一业务容器组发出的数据包转发至所述第一边车包括：
    所述连接控制模块确定所述第一边车的服务对象的数量，在所述第一边车的服务对象的数量为0的情况下将所述第一业务容器组发出的数据包转发至所述第一边车。
  23. 根据权利要求14至22任一项所述的方法，其特征在于，所述方法还包括：
    所述连接控制模块监控所述边车集群中每个边车的工作状态，在发现存在下线的边车时，发送下线的所述边车的信息至所述控制台。
  24. 根据权利要求14至23任一项所述的方法，其特征在于，所述流量管理包括：流量控制、流量安全以及流量观测。
  25. 一种运行容器组的节点，其特征在于，包括处理器和存储器，所述处理器用于执行所述存储器中存储的指令，以执行权利要求14至24任一项所述的方法。
  26. 一种计算机可读存储介质，其特征在于，包括计算机程序指令，当所述计算机程序指令由计算设备集群执行时，所述计算设备集群执行如权利要求14至24中任一项所述的方法。
  27. 一种包含指令的计算机程序产品，其特征在于，当所述指令被计算机设备集群运行时，使得所述计算机设备集群执行如权利要求的14至24中任一项所述的方法。
PCT/CN2022/110895 2021-08-09 2022-08-08 运行容器组的节点，容器组的管理系统和方法 Ceased WO2023016415A1 (zh)

Priority Applications (3)

Application Number Priority Date Filing Date Title
JP2024508076A JP7755045B2 (ja) 2021-08-09 2022-08-08 コンテナグループを稼働させるためのノード、ならびにコンテナグループ管理システムおよび方法
EP22855396.2A EP4369181A4 (en) 2021-08-09 2022-08-08 Node for running container group, and management system and method of container group
US18/435,772 US20240250918A1 (en) 2021-08-09 2024-02-07 Node for running container group, and container group management system and method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110910594.X 2021-08-09
CN202110910594.XA CN115705198A (zh) 2021-08-09 2021-08-09 运行容器组的节点，容器组的管理系统和方法

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/435,772 Continuation US20240250918A1 (en) 2021-08-09 2024-02-07 Node for running container group, and container group management system and method

Publications (1)

Publication Number Publication Date
WO2023016415A1 true WO2023016415A1 (zh) 2023-02-16

Family

ID=85179336

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/110895 Ceased WO2023016415A1 (zh) 2021-08-09 2022-08-08 运行容器组的节点，容器组的管理系统和方法

Country Status (5)

Country Link
US (1) US20240250918A1 (zh)
EP (1) EP4369181A4 (zh)
JP (1) JP7755045B2 (zh)
CN (1) CN115705198A (zh)
WO (1) WO2023016415A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117149264A (zh) * 2023-10-31 2023-12-01 山东浪潮科学研究院有限公司 一种多泳道研发环境构建方法、装置、设备及存储介质

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116032806B (zh) * 2023-03-27 2023-06-09 杭州谐云科技有限公司 一种流量染色方法和系统
CN116319793B (zh) * 2023-04-03 2025-11-18 中国工商银行股份有限公司 基于服务网格的数据请求方法、装置及电子设备
CN116880996A (zh) * 2023-06-02 2023-10-13 阿里巴巴（中国）有限公司 虚拟计算资源的调度方法及控制面组件
US20250061009A1 (en) * 2023-08-17 2025-02-20 International Business Machines Corporation Automatic State Migration of Stateful Container During Secondary Application Container Hot Upgrade

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109067597A (zh) * 2018-09-21 2018-12-21 杭州安恒信息技术股份有限公司 一种分布式系统动态智能服务治理方法
CN110262899A (zh) * 2019-06-20 2019-09-20 无锡华云数据技术服务有限公司 基于Kubernetes集群的监控组件弹性伸缩方法、装置及受控终端
US10623390B1 (en) * 2017-08-24 2020-04-14 Pivotal Software, Inc. Sidecar-backed services for cloud computing platform
CN111552496A (zh) * 2020-05-07 2020-08-18 上海道客网络科技有限公司 一种基于添加临时容器实现无缝升级边车的系统与方法
CN112929180A (zh) * 2021-02-05 2021-06-08 中国—东盟信息港股份有限公司 一种Kubernetes零信任网络安全系统及其实现方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3811209A1 (en) 2018-06-25 2021-04-28 Amazon Technologies Inc. Execution of auxiliary functions in an on-demand network code execution system
US11570271B2 (en) 2019-04-10 2023-01-31 Cisco Technology, Inc. Differentiated smart sidecars in a service mesh
US11635995B2 (en) 2019-07-16 2023-04-25 Cisco Technology, Inc. Systems and methods for orchestrating microservice containers interconnected via a service mesh in a multi-cloud environment based on a reinforcement learning policy
US11700150B2 (en) 2019-07-31 2023-07-11 Vmware, Inc. Allocation of tokens for network packets based on application type

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10623390B1 (en) * 2017-08-24 2020-04-14 Pivotal Software, Inc. Sidecar-backed services for cloud computing platform
CN109067597A (zh) * 2018-09-21 2018-12-21 杭州安恒信息技术股份有限公司 一种分布式系统动态智能服务治理方法
CN110262899A (zh) * 2019-06-20 2019-09-20 无锡华云数据技术服务有限公司 基于Kubernetes集群的监控组件弹性伸缩方法、装置及受控终端
CN111552496A (zh) * 2020-05-07 2020-08-18 上海道客网络科技有限公司 一种基于添加临时容器实现无缝升级边车的系统与方法
CN112929180A (zh) * 2021-02-05 2021-06-08 中国—东盟信息港股份有限公司 一种Kubernetes零信任网络安全系统及其实现方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4369181A4

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117149264A (zh) * 2023-10-31 2023-12-01 山东浪潮科学研究院有限公司 一种多泳道研发环境构建方法、装置、设备及存储介质
CN117149264B (zh) * 2023-10-31 2024-01-30 山东浪潮科学研究院有限公司 一种多泳道研发环境构建方法、装置、设备及存储介质

Also Published As

Publication number Publication date
US20240250918A1 (en) 2024-07-25
CN115705198A (zh) 2023-02-17
JP7755045B2 (ja) 2025-10-15
EP4369181A4 (en) 2024-10-30
JP2024529099A (ja) 2024-08-01
EP4369181A1 (en) 2024-05-15

Similar Documents

Publication Publication Date Title
CN111615066B (zh) 一种基于广播的分布式微服务注册及调用方法
WO2023016415A1 (zh) 运行容器组的节点，容器组的管理系统和方法
US10171567B2 (en) Load balancing computer device, system, and method
CN103207841B (zh) 基于键值对缓存的数据读写方法及装置
CN104811328B (zh) 虚拟网络资源管理方法及装置
US20120239730A1 (en) System including a middleware machine environment
CN110391940B (zh) 服务地址的响应方法、装置、系统、设备和存储介质
CN103703724A (zh) 一种资源发放方法
CN102447624A (zh) 在服务器集群上实现负载均衡的方法、节点服务器及集群
CN107846358A (zh) 一种数据传输方法、装置及网络系统
CN113839862A (zh) Mclag邻居之间同步arp信息的方法、系统、终端及存储介质
CN105704042A (zh) 报文处理方法、bng及bng集群系统
CN103140851B (zh) 包括中间件机环境的系统
CN115941493B (zh) 基于组播的云场景nat网关集群的多活分配方法及装置
KR101956320B1 (ko) 트랜잭셔널 미들웨어 머신 환경에서 단일 포인트 병목을 방지하는 시스템 및 방법
CN113709220A (zh) 虚拟负载均衡器的高可用实现方法、系统及电子设备
WO2023066224A1 (zh) 一种部署容器服务的方法及装置
CN114024971B (zh) 业务数据处理方法、Kubernetes集群及介质
CN109120556B (zh) 一种云主机访问对象存储服务器的方法及系统
CN111355602B (zh) 一种资源对象的管理方法及装置
CN117459499A (zh) Ip地址分配方法、装置及计算机设备
WO2019011180A1 (zh) 一种License的发送方法和装置
CN114553704A (zh) 一种支持多设备同时接入服务器实现扩缩容的方法及系统
CN116743845B (zh) 边缘服务发现方法、装置、节点设备和可读存储介质
CN114945023A (zh) 一种网络连接复用方法、装置、设备及介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22855396

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2022855396

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2022855396

Country of ref document: EP

Effective date: 20240206

WWE Wipo information: entry into national phase

Ref document number: 2024508076

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 11202400851W

Country of ref document: SG