[go: up one dir, main page]

CN116668467B - A resource access method, apparatus, cloud hosting system, and electronic device. - Google Patents

A resource access method, apparatus, cloud hosting system, and electronic device.

Info

Publication number
CN116668467B
CN116668467B CN202210158116.2A CN202210158116A CN116668467B CN 116668467 B CN116668467 B CN 116668467B CN 202210158116 A CN202210158116 A CN 202210158116A CN 116668467 B CN116668467 B CN 116668467B
Authority
CN
China
Prior art keywords
crd
target
control plane
resource
middleware
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210158116.2A
Other languages
Chinese (zh)
Other versions
CN116668467A (en
Inventor
曹迪
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Kingsoft Cloud Network Technology Co Ltd
Original Assignee
Beijing Kingsoft Cloud Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Kingsoft Cloud Network Technology Co Ltd filed Critical Beijing Kingsoft Cloud Network Technology Co Ltd
Priority to CN202210158116.2A priority Critical patent/CN116668467B/en
Publication of CN116668467A publication Critical patent/CN116668467A/en
Application granted granted Critical
Publication of CN116668467B publication Critical patent/CN116668467B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1095Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Stored Programmes (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention relates to the technical field of computers, and particularly discloses a resource access method, a device, a cloud hosting system and electronic equipment, which are applied to target middleware, wherein the target middleware is any one of a plurality of middleware deployed in a container cluster, a control plane and a custom CRD resource of each tenant are deployed in the container cluster, each middleware is pre-associated with a control plane and a custom CRD resource of a unique tenant, the method comprises the steps of monitoring whether the pre-associated CRD resource is updated or not, determining the updated CRD resource as a target CRD resource when the updated CRD resource is monitored, and reporting the target CRD resource to the pre-associated target control plane so that the target CRD resource is acquired by the target control plane. Therefore, by the scheme, the isolated access of CRD resources of multiple tenants can be ensured, and meanwhile, the waste of system resources is reduced.

Description

Resource access method and device, cloud hosting system and electronic equipment
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a resource access method and apparatus, a cloud hosting system, and an electronic device.
Background
The service grid serves as an infrastructure layer for inter-service communication for managing service-to-service communication. The service grid comprises a data plane and a control plane, wherein the data plane in the service grid consists of proxy components responsible for routing, load balancing, service discovery, health checking and authorization/authentication, and the control plane in the service grid is used for configuring the behavior of the data plane.
To simplify administration of services, cloud vendors may cloud host control planes in the tenant's service grid. When cloud hosting is performed, the control plane and the customized CRD (Custom Resource Definition, customized resource) resources of the tenant are usually deployed in the container cluster, so that the control plane can sense the configuration change by modifying the customized resources deployed in the container cluster, and further, the customized resources are converted into the configuration content of the proxy component, thereby achieving the purpose of controlling various configuration contents of the data plane.
For cloud hosting of control planes, control planes and CRD resources of multiple tenants are involved, and therefore it is required to ensure that the control planes of different tenants cannot mutually access the CRD resources of each other, namely, isolated access of the CRD resources of multiple tenants.
In the related technology, the control plane of each tenant and the corresponding CRD resource are deployed in an independent container cluster, and then, in the resource access process, the control plane in any container cluster directly reads the CRD resource of the container cluster when detecting that the CRD resource in the container cluster is updated. In this way, isolated access of multi-tenant CRD resources is achieved.
However, setting an independent container cluster for each tenant may result in waste of system resources of the container cluster. Therefore, how to reduce the system resource waste while ensuring the isolated access of the CRD resources of multiple tenants is a problem to be solved.
Disclosure of Invention
The embodiment of the invention aims to provide a resource access method, a resource access device, a cloud hosting system and electronic equipment, so that the isolated access of CRD resources of multiple tenants is ensured, and meanwhile, the waste of system resources is reduced.
The specific technical scheme is as follows:
In a first aspect, the embodiment of the invention provides a resource access method, which is applied to a target middleware, wherein the target middleware is any one of a plurality of middleware deployed in a container cluster, a control plane and a custom CRD resource of each tenant are also deployed in the container cluster, and each middleware is pre-associated with a control plane and a custom CRD resource of only one tenant;
The method comprises the following steps:
Monitoring whether the CRD resources associated in advance are updated or not;
When the updating is monitored, determining CRD resources with the updating as target CRD resources;
And reporting the target CRD resources to a pre-associated target control plane so that the target control plane acquires the target CRD resources.
Optionally, namespaces of control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
The monitoring whether the CRD resource associated in advance is updated comprises the following steps:
Monitoring whether CRD resources with the same name space are updated or not;
the reporting the target CRD resource to a pre-associated target control plane includes:
and reporting the target CRD resources to a target control plane with the same naming space.
Optionally, the reporting the target CRD resource to a pre-associated target control plane includes:
and if a resource update query request sent by a pre-associated target control plane is received, reporting the target CRD resource to the pre-associated target control plane.
Optionally, each middleware is a grid configuration protocol MCP middleware.
In a second aspect, an embodiment of the present invention provides a container cluster, where multiple middleware is deployed in the container cluster, and a control plane and a custom CRD resource of each tenant are associated with each middleware in advance, where each middleware is associated with a control plane and a custom CRD resource of only one tenant;
The system comprises a target middleware, a target control plane and a target control plane, wherein the target middleware is used for monitoring whether a pre-associated CRD resource is updated;
and the target control plane is used for receiving the target CRD resources reported by the target middleware.
Optionally, namespaces of control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
the target middleware monitors whether the CRD resources associated in advance are updated or not, and the target middleware comprises:
Monitoring whether CRD resources with the same name space are updated or not;
The target middleware reports the target CRD resource to a pre-associated target control plane, which comprises the following steps:
and reporting the target CRD resources to a target control plane with the same naming space.
The embodiment of the invention provides a cloud hosting system, which comprises a management platform and a container cluster, wherein a plurality of middleware, a control plane and a custom CRD resource of each tenant are deployed in the container cluster, and each middleware is pre-associated with the control plane and the custom CRD resource of only one tenant;
the management platform is used for updating CRD resources of any tenant in the container cluster when receiving a resource updating instruction for the tenant;
The target middleware in the container cluster is used for monitoring whether the CRD resources associated in advance are updated or not, determining the CRD resources which are updated as target CRD resources when the CRD resources are monitored to be updated, and reporting the target CRD resources to a target control plane associated in advance so that the target control plane can issue the target CRD resources to a data plane of a tenant side;
the target middleware is any middleware in the plurality of middleware.
Optionally, namespaces of control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
the target middleware monitors whether the CRD resources associated in advance are updated or not, and the target middleware comprises:
Monitoring whether CRD resources with the same name space are updated or not;
The target middleware reporting the target CRD resource to a pre-associated target control plane comprises:
and reporting the target CRD resources to a target control plane with the same naming space.
Optionally, the container cluster management platform is further configured to:
Outputting the namespaces of all control planes in the container cluster;
when a management instruction for a target namespace in each namespace is received, the control plane for the target namespace is responsive to the target management instruction.
In a fourth aspect, the embodiment of the invention provides a resource access device, which is applied to a target middleware, wherein the target middleware is any one of a plurality of middleware deployed in a container cluster, the container cluster is also provided with a control plane and a custom CRD resource of each tenant, and each middleware is pre-associated with the control plane and the custom CRD resource of only one tenant;
the device comprises:
The monitoring module is used for monitoring whether the CRD resources associated in advance are updated or not;
the determining module is used for determining CRD resources updated when the updating is monitored and taking the CRD resources as target CRD resources;
And the reporting module is used for reporting the target CRD resources to a pre-associated target control plane so that the target control plane acquires the target CRD resources.
Optionally, namespaces of control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
the monitoring module is specifically configured to:
Monitoring whether CRD resources with the same name space are updated or not;
the reporting module is specifically configured to:
and reporting the target CRD resources to a target control plane with the same naming space.
Optionally, the reporting module is specifically configured to:
and if a resource update query request sent by a pre-associated target control plane is received, reporting the target CRD resource to the pre-associated target control plane.
Optionally, each middleware is a grid configuration protocol MCP middleware.
In a fifth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus;
A memory for storing a computer program;
and the processor is used for realizing any step of the resource access method when executing the program stored in the memory.
In a sixth aspect, embodiments of the present invention provide a computer readable storage medium having a computer program stored therein, which when executed by a processor, implements the steps of any of the above-described resource access methods.
In a seventh aspect, embodiments of the present invention also provide a computer program product comprising instructions which, when run on a computer, cause the computer to perform the steps of any of the above-described resource access methods.
The embodiment of the invention has the beneficial effects that:
In the scheme provided by the embodiment of the invention, a control plane and CRD resources of multiple tenants are deployed in the same container cluster, a plurality of middleware is deployed in the container cluster, each middleware is pre-associated with a control plane and a custom CRD resource of only one tenant, further, whether the CRD resources pre-associated with the middleware are updated or not is monitored through a target middleware deployed in the container cluster, when the updated CRD resources are monitored, the updated CRD resources are determined to be used as target CRD resources, and then the target CRD resources are reported to a target control plane pre-associated with the target middleware, so that the target CRD resources are acquired by the target control plane. It can be seen that, for multiple tenants, the number of container clusters can be greatly reduced by disposing the control plane and CRD resources of multiple tenants in the same container cluster, so that the waste of system resources of the container cluster is reduced, and in the container cluster, a plurality of middleware are disposed, each middleware is pre-associated with the control plane and CRD resources of a single tenant, so that the control plane can sense the update of CRD resources of the same tenant through the middleware pre-associated with the control plane, and acquire the CRD resources of the same tenant, which are updated. Therefore, by the scheme, the isolated access of CRD resources of multiple tenants can be ensured, and meanwhile, the waste of system resources is reduced. In addition, by deploying the multi-tenant control plane in the same container cluster, operation and maintenance management of operation and maintenance personnel of a cloud manufacturer hosting the control plane can be facilitated.
Of course, it is not necessary for any one product or method of practicing the invention to achieve all of the advantages set forth above at the same time.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the invention, and other embodiments may be obtained according to these drawings to those skilled in the art.
Fig. 1 is a schematic diagram of a cloud hosting scheme in the related art;
FIG. 2 is a flowchart of a method for accessing resources according to an embodiment of the present invention;
FIG. 3 is another flowchart of a method for accessing resources according to an embodiment of the present invention;
fig. 4 is a schematic diagram of a cloud hosting scheme to which the resource access method provided by the embodiment of the present invention is applied;
FIG. 5 is a schematic diagram of resource reading in the resource access method according to the embodiment of the present invention;
fig. 6 is a schematic structural diagram of a cloud hosting system according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a configuration of a resource access device according to an embodiment of the present invention;
fig. 8 is a block diagram of an electronic device implementing a resource access method in an embodiment of the present invention.
Detailed Description
The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present application, but not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by the person skilled in the art based on the present application are included in the scope of protection of the present application.
The following first describes terms related to embodiments of the present invention:
Service grid a service grid is defined as a special infrastructure layer for managing the communication between services, which are functional modules that enable certain data processing functions. The service grid has the following functions:
(1) Flow management, namely, flow among services and API (Application Program Interface ) call can be controlled through simple rule configuration and flow routing, so that the configuration of services such as fuses, overtime, retries and the like is simplified;
(2) Observability, namely enabling staff to better know services through tracking, monitoring and logging, and rapidly discovering and repairing problems;
(3) Security-authentication, authorization, and encryption of communications can be managed at the proxy level, policies can be enforced in individual services through quick configuration changes.
The service grid has two parts, a data plane and a control plane.
Data plane the data plane in the service grid consists of proxy components responsible for routing, load balancing, service discovery, health checking and authorization/authentication. These agents run alongside each instance service, intercepting all incoming and outgoing user traffic.
Control plane the control plane in the service grid is used to provide configuration content for the data plane and control the data plane. It consists of APIs and functional plug-ins for managing data plane behavior. The operation and maintenance personnel of the tenant can configure the behavior of the data plane in the service grid through accessing the control plane.
The cloud manufacturer hosts the control plane in the service grid for simplifying the management of the service, including the flow routing and splitting management between service calls, the authentication security of communication between services and the grid observability capability, thereby greatly reducing the workload of development and operation and maintenance.
CRD resources, namely custom resources, adopting rule contents written in YAML format, and arranging the rule contents with a control plane when the cloud is managed, wherein the control plane is responsible for finding out the rule contents and converting the rule contents into operation configuration of a data plane. Illustratively, the CRD resource may be a resource utilization. Yaml file in which rules for traffic management, load balancing rules, etc. may be recorded.
And the container cluster consists of a storage node, a service node and a control node. All the cluster states are stored in a storage node, a control end of a service node of the cluster is operated on a control node, the service node is a host node which truly operates an application container, a processing end is operated on each service node, and the container, the mirror image, the storage and the like on the node are controlled.
In the related technology, the control plane of each tenant and the corresponding CRD resource are deployed in an independent container cluster, and then, in the resource access process, the control plane in any container cluster directly reads the CRD resource of the container cluster when detecting that the CRD resource in the container cluster is updated. In this way, isolated access of multi-tenant CRD resources is achieved.
For example, as shown in fig. 1, by deploying the control plane and the data plane in the cloud vendor environment and the tenant environment, respectively, the lifecycle management of the service grid instance and the service runtime environment can be decoupled to a certain extent, and the control plane in the service grid is hosted, so that the workload of development and operation and maintenance is reduced. When hosting the service grid control plane, first, a cloud manufacturer needs to deploy a single container cluster for one tenant, as shown in fig. 1, three container clusters are deployed in the cloud manufacturer environment for three tenants, then, the control plane and CRD resources of the service grid of the tenant are deployed in each container cluster, then, the tenant senses the change of configuration content by modifying the CRD resources deployed in the container clusters, and then, the control plane of the same tenant converts rule content written by YAML, namely, updated CRD resources, into configuration content of the data plane, thereby achieving the purpose of controlling various configuration content of the data plane. However, for each tenant to set up an independent container cluster, since each container cluster needs to deploy some necessary system components, such as APISERVER components, schedule components, controller manager components, etc., a large number of container clusters need to deploy many system components, which may result in waste of system resources of the container clusters.
Based on the above, in order to ensure isolated access of CRD resources of multiple tenants and reduce system resource waste, the invention provides a resource access method, a device, a cloud hosting system and electronic equipment.
The following first describes a resource access method provided by the embodiment of the present invention.
The resource access method provided by the embodiment of the invention is applied to the target middleware, wherein the target middleware is any one of a plurality of middleware deployed in a container cluster, the container cluster is also provided with control planes and custom CRD resources of all tenants, and each middleware is pre-associated with the control plane and the custom CRD resources of only one tenant. The custom resource may be a default resource capable of meeting the requirements of most daily system deployment and management in a Kubernetes open source system, and the resource content of the custom resource may be rule content for traffic management, load balancing rule content, or the like, for example, the resource content of the custom resource is set in some special requirement scenarios.
It should be noted that, the control plane is a set of control ends for several services of the tenant, that is, a set of multiple control ends, and the data plane is a set of processing ends for several services of the tenant. Each control end in the control plane can be deployed in a container, each control end can read all CRD resources of the container cluster where the control end is located, then screening is carried out to obtain CRD resources matched with the corresponding service, and the CRD resources matched with the corresponding service are converted into configuration content of the data plane to control the behavior of the corresponding service. In addition, the control plane and the data plane can be considered as functional programs provided by cloud vendors, the control plane is hosted by the cloud vendors, and the data plane is deployed with the corresponding services of the tenant.
In addition, some requisite system components, such as APISERVER components, schedulers components, and controller manager components, are deployed in the container cluster. The control system comprises a APISERVER component, a schedule component and a controller manager component, wherein the APISERVER component is responsible for responding to a management request of a user and conducting command coordination work, CRD resources are deployed in the component, the schedule component binds a container to be scheduled to a proper node according to a certain scheduling algorithm, and the controller manager component is a set of controllers and is responsible for controlling and managing corresponding resources such as copies, node nodes and the like.
The resource access method provided by the embodiment of the invention can comprise the following steps:
Monitoring whether the CRD resources associated in advance are updated or not;
When the updating is monitored, determining CRD resources with the updating as target CRD resources;
And reporting the target CRD resources to a pre-associated target control plane so that the target control plane acquires the target CRD resources.
In the scheme provided by the embodiment of the invention, a control plane and CRD resources of multiple tenants are deployed in the same container cluster, a plurality of middleware are deployed in the container cluster, each middleware is pre-associated with a control plane and a custom CRD resource of only one tenant, further, whether the CRD resources pre-associated with the middleware are updated or not is monitored through a target middleware deployed in the container cluster, when the updated CRD resources are monitored, the updated CRD resources are determined to be used as target CRD resources, and then the target CRD resources are reported to a target control plane pre-associated with the target middleware, so that the target CRD resources are acquired by the target control plane. It can be seen that, for multiple tenants, the number of container clusters can be greatly reduced by disposing the control plane and CRD resources of multiple tenants in the same container cluster, so that the waste of system resources of the container cluster is reduced, and in the container cluster, a plurality of middleware are disposed, each middleware is pre-associated with the control plane and CRD resources of a single tenant, so that the control plane can sense the update of CRD resources of the same tenant through the middleware pre-associated with the control plane, and acquire the CRD resources of the same tenant, which are updated. Therefore, by the scheme, the isolated access of CRD resources of multiple tenants can be ensured, and meanwhile, the waste of system resources is reduced. In addition, by deploying the multi-tenant control plane in the same container cluster, operation and maintenance management of operation and maintenance personnel of a cloud manufacturer hosting the control plane can be facilitated.
The resource access method provided by the embodiment of the invention is described below with reference to the accompanying drawings.
As shown in fig. 2, a resource access method provided by an embodiment of the present invention may include the following steps:
s201, monitoring whether the CRD resources associated in advance are updated or not;
In this embodiment, a plurality of middleware, control planes of each tenant, and custom CRD resources are deployed in a container cluster, each middleware is associated with a control plane and custom CRD resource of only one tenant in advance, and each middleware can monitor whether the CRD resource associated with the middleware in advance is updated.
It can be understood that, since each middleware is pre-associated with a control plane and a custom CRD resource of only one tenant, by monitoring whether the pre-associated CRD resource is updated by the target middleware, the target control plane pre-associated with the target middleware can timely acquire the pre-associated CRD resource when the pre-associated CRD resource is updated, and the pre-associated CRD resource and the target control plane belong to the same tenant. That is, each middleware acts as a bridge for connecting the control plane and CRD resources of only one tenant associated with the middleware, thereby facilitating the isolation management of CRD resources of multiple tenants. Among other things, listening to whether an update occurs to a pre-associated CRD resource may specifically refer to listening to whether a new resource is modified, deleted, or added for the pre-associated CRD resource. There may be multiple implementations in which each middleware pre-associates a control plane of only one tenant and custom CRD resources. The method for establishing the association relationship between each middleware and the control plane of only one tenant and the custom CRD resource, namely the implementation manner of pre-associating the control plane of only one tenant and the custom CRD resource by each middleware, comprises the steps of pre-establishing a one-to-one correspondence between the control plane of each tenant and the middleware, namely distributing a control plane for each middleware, then establishing a correspondence between each middleware and the designated CRD resource, wherein the designated CRD resource corresponding to each middleware is the CRD resource belonging to the same tenant as the control plane corresponding to the middleware, and accordingly, the association relationship between each middleware and the control plane of only one tenant and the custom CRD resource is established. In the implementation manner, the naming of the CRD resource of each tenant can be related to the identity of the tenant, so that the tenant to which the CRD resource belongs can be identified based on the identity of the tenant, and the establishment and the update of the corresponding relation between the middleware and the CRD resource are facilitated. With respect to other implementations, the description is provided later in connection with other embodiments.
There may be various ways to update CRD resources. The method for updating the CRD resources can include that an operation and maintenance person submits a resource updating instruction for any tenant through an operation interface of the management platform, and then the management platform updates the CRD resources of the tenant in the container cluster when receiving the resource updating instruction.
There may be multiple implementations in which the target middleware listens to whether the pre-associated CRD resource is updated. The method that the target middleware monitors whether the pre-associated CRD resource is updated or not may be monitoring whether the CRD resource pre-associated with the target middleware, which is set in the management platform, is updated or not through a port communicated with the management platform, or the target middleware may monitor whether the pre-associated CRD resource is updated or not based on pre-recorded historical resource information of the pre-associated CRD resource, where the historical resource information is resource information of CRD resources that have been issued to a corresponding control plane before, and the historical resource information may include file names, version numbers, data sizes of the CRD resources, and the like.
Additionally, optionally, in a specific implementation, each middleware described above may be MCP (Mesh Configuration Protocol, grid configuration protocol) middleware through which configured interactions with other components in the container cluster may be performed. It should be emphasized that the embodiments of the present invention are not limited to specific types of middleware on the premise of being able to implement the solutions provided by the embodiments of the present invention.
S202, when monitoring the update, determining CRD resources with the update as target CRD resources;
It may be appreciated that, since there may be a plurality of CRD resources that the target middleware is responsible for listening to, that is, there may be a plurality of CRD resources pre-associated with the target middleware, when an update in the pre-associated CRD resources is detected, it is necessary to determine, as the target CRD resource, the CRD resource where the update occurs. Thus, the target CRD resource may be read later and reported to the pre-associated target control plane.
S203, reporting the target CRD resource to a pre-associated target control plane so that the target control plane acquires the target CRD resource.
After the target middleware monitors the updated target CRD resources, the target CRD resources are reported to a target control plane pre-associated with the target middleware, so that the target control plane can acquire the target CRD resources. Furthermore, after the target control plane obtains the target CRD resource, the target CRD resource may be converted into configuration content of the data plane to control the behavior of the data plane.
Optionally, in a specific implementation manner, when the target middleware determines the target CRD resource, the determined target CRD resource may be directly reported to the target control plane.
Optionally, in another specific implementation, reporting the target CRD resource to a pre-associated target control plane includes:
And if a resource update query request sent by the pre-associated target control plane is received, reporting the target CRD resource to the pre-associated target control plane.
The target control plane may periodically send a resource update query request to the target middleware, so that the target middleware, after receiving the resource query request, reads the target CRD resource and reports the target CRD resource to the target control plane associated in advance, so that the target control plane may change the target CRD resource into the configuration content of the data plane after reading the target CRD resource by the target middleware, so as to control the behavior of the data plane.
In the scheme provided by the embodiment of the invention, a control plane and CRD resources of multiple tenants are deployed in the same container cluster, a plurality of middleware are deployed in the container cluster, each middleware is pre-associated with a control plane and a custom CRD resource of only one tenant, further, whether the CRD resources pre-associated with the middleware are updated or not is monitored through a target middleware deployed in the container cluster, when the updated CRD resources are monitored, the updated CRD resources are determined to be used as target CRD resources, and then the target CRD resources are reported to a target control plane pre-associated with the target middleware, so that the target CRD resources are acquired by the target control plane. It can be seen that, for multiple tenants, the number of container clusters can be greatly reduced by disposing the control plane and CRD resources of multiple tenants in the same container cluster, so that the waste of system resources of the container cluster is reduced, and in the container cluster, a plurality of middleware are disposed, each middleware is pre-associated with the control plane and CRD resources of a single tenant, so that the control plane can sense the update of CRD resources of the same tenant through the middleware pre-associated with the control plane, and acquire the CRD resources of the same tenant, which are updated. Therefore, by the scheme, the isolated access of CRD resources of multiple tenants can be ensured, and meanwhile, the waste of system resources is reduced. In addition, by deploying the multi-tenant control plane in the same container cluster, operation and maintenance management of operation and maintenance personnel of a cloud manufacturer hosting the control plane can be facilitated.
Optionally, in another embodiment of the present invention, namespaces of control planes of different tenants are different, and the control plane and CRD resource of each tenant have the same namespaces;
The way that each middleware is pre-associated with the control plane of only one tenant and the customized CRD resource comprises that each middleware is uniquely identical with the naming space of the control plane of one tenant and the CRD resource.
In this embodiment, a namespace is used to distinguish control planes of different tenants from a naming dimension. Wherein a namespace, also called a namespace, is a uniquely identified set of names. In a specific application, the namespace of the control plane may be set by a namespace attribute of the control plane, the control plane of one tenant corresponds to one namespace, and the control plane of the tenant and the CRD resource are set with the same namespace. And may be deployed in a control plane of one tenant to multiple containers that share one namespace.
It can be appreciated that, since the control planes of different tenants are provided with different namespaces, and the CRD resource of each tenant and each middleware are also provided with namespaces, it can be determined whether each middleware has an association with each control plane and the custom CRD resource by determining whether the namespaces set by the control planes, the middleware and the CRD resource are the same.
In addition, it should be noted that, for the purpose of tenant isolation, one control plane may correspond to one middleware, that is, each middleware is uniquely the same as the namespace of the control plane of one tenant. Thus, the control plane may acquire the same CRD resources as the control plane namespace through the middleware corresponding thereto.
In this embodiment, as shown in fig. 3, the method includes steps S301 to S303:
S301, monitoring whether CRD resources with the same name space are updated or not;
It can be understood that, since each middleware is uniquely identical to the namespace of the control plane of one tenant, the CRD resource having the same namespace as the target middleware is the CRD resource having the same namespace as the target control plane, and at this time, by monitoring whether the CRD resource having the same namespace is updated, the target control plane can timely read the CRD resource when the CRD resource is updated.
S302, when monitoring the update, determining CRD resources with the update as target CRD resources;
The content of step 302 is similar to that of step S202, and will not be described here again.
S303, reporting the target CRD resource to a target control plane with the same name space, so that the target control plane acquires the target CRD resource.
It can be appreciated that, since the target CRD resource and the target control plane have the same namespace, the target CRD resource and the target control plane are CRD resources and control planes of a container cluster for the same tenant, reporting the target CRD resource to the target control plane having the same namespace can enable the target control plane to access to CRD resources belonging to the tenant without reading CRD resources of other tenants. Therefore, the isolated access of the CRD resources of multiple tenants is realized.
In the embodiment, for multiple tenants, the control plane and CRD resources of the multiple tenants are deployed in the same container cluster, so that the number of the container clusters can be greatly reduced, the waste of system resources of the container clusters is reduced, the same naming space is set for the control plane and CRD resources of each tenant, and a middleware is associated with the control plane of each tenant, so that the control plane can access the CRD resources of the tenants through the middleware, and the isolated access of the CRD resources of the multiple tenants is achieved. Therefore, the scheme can ensure the isolated access of CRD resources of multiple tenants and reduce the waste of system resources. In addition, by deploying the multi-tenant control plane in the same container cluster, operation and maintenance management of operation and maintenance personnel of a cloud manufacturer hosting the control plane is facilitated.
For a better understanding of the present solution, a specific example of an embodiment of the present invention is set forth below in connection with fig. 4 and 5.
Fig. 4 illustrates a schematic diagram of a cloud hosting scheme to which the resource access method provided by the embodiment of the present invention is applied. As shown in fig. 4, a data plane of each tenant is deployed in the tenant environment, and a control plane corresponding to each tenant data plane is deployed in the cloud vendor environment. By deploying the control plane and the data plane in different operating environments, lifecycle management of service grid instances and service runtime environments can be decoupled to some extent.
In the cloud vendor environment, the control planes of the tenants are arranged in one container cluster, and the control planes of the tenants are respectively provided with different namespaces, for example, the "namespace-1" in fig. 4 can be a namespace set for the control plane of the tenant a, and the "namespace-2" can be a namespace set for the control plane of the tenant B. In addition, the MCP middleware and the CRD resource of each tenant are also deployed under the namespaces set by the control plane of each tenant, that is, the MCP middleware and the CRD resource corresponding to the control plane of the tenant are set with the same namespaces. In the deployment mode, only one system component of the container cluster is required to be deployed, so that the resource occupation of the system component is reduced, meanwhile, the control planes of all tenants are positioned in one container cluster, when the service grid control planes under different tenants are managed, the control planes of different tenants do not need to enter into different container clusters for management, and the control planes of different tenants are only required to be switched for a namespace in the same container cluster, so that the workload of operation and maintenance can be reduced.
In the tenant environment, the data plane is a set of processing ends of a plurality of services for the tenant, and each processing end in the data plane is located in a service node in the container cluster, that is, in a container in the container cluster, and each processing end corresponds to one service. The service can call the processing end corresponding to the service to communicate with the processing end corresponding to other services so as to realize the call among different services.
Fig. 5 illustrates a schematic diagram of a control plane reading CRD resources under its own namespace. As shown in fig. 5, since the MCP middleware is deployed in each namespace, the manner of reading the CRD resources by the control plane is set to read from the MCP middleware, and then the MCP middleware is responsible for monitoring and reading all CRD resources under the namespace, so that multi-tenant CRD isolation is performed, and CRD resources of the other party cannot be read by the control planes hosted by different tenants.
Correspondingly, the embodiment of the invention also provides a container cluster, wherein a plurality of middleware, a control plane and a custom CRD resource of each tenant are deployed in the container cluster, and each middleware is pre-associated with the control plane and the custom CRD resource of only one tenant;
The system comprises a target middleware, a target control plane and a target control plane, wherein the target middleware is used for monitoring whether a pre-associated CRD resource is updated;
and the target control plane is used for receiving the target CRD resources reported by the target middleware.
Optionally, namespaces of control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
the target middleware monitors whether the CRD resources associated in advance are updated or not, and the target middleware comprises:
Monitoring whether CRD resources with the same name space are updated or not;
The target middleware reports the target CRD resource to a pre-associated target control plane, which comprises the following steps:
and reporting the target CRD resources to a target control plane with the same naming space.
The specific functional implementation of each component included in the container cluster may be referred to the corresponding content in the above method embodiment, which is not described herein.
Correspondingly, the embodiment of the invention also provides a cloud hosting system, as shown in fig. 6, which comprises a management platform 610 and a container cluster 620, wherein a plurality of middleware, control planes of all tenants and custom CRD resources are deployed in the container cluster 620, and each middleware is pre-associated with the control plane of only one tenant and the custom CRD resources;
the management platform 610 is configured to update CRD resources of any tenant in the container cluster when receiving a resource update instruction for the tenant;
The target middleware in the container cluster 620 is configured to monitor whether a pre-associated CRD resource is updated, determine, when the update is monitored, the CRD resource that is updated is used as a target CRD resource, and report the target CRD resource to a pre-associated target control plane, so that the target control plane issues the target CRD resource to a data plane on the tenant side;
the target middleware is any middleware in the plurality of middleware.
Optionally, namespaces of control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
the target middleware monitors whether the CRD resources associated in advance are updated or not, and the target middleware comprises:
Monitoring whether CRD resources with the same name space are updated or not;
The target middleware reporting the target CRD resource to a pre-associated target control plane comprises:
and reporting the target CRD resources to a target control plane with the same naming space.
Optionally, the container cluster management platform is further configured to:
Outputting the namespaces of all control planes in the container cluster;
when a management instruction for a target namespace in each namespace is received, the control plane for the target namespace is responsive to the target management instruction.
The specific functional implementation of each component included in the system may refer to the corresponding content in the above method embodiment, which is not described herein.
Correspondingly, the embodiment of the method also provides a resource access device, which is applied to the target middleware, as shown in fig. 7, and comprises:
A monitoring module 710, configured to monitor whether the CRD resource associated in advance is updated;
a determining module 720, configured to determine, when an update is monitored, a CRD resource where the update occurs as a target CRD resource;
And a reporting module 730, configured to report the target CRD resource to a target control plane associated in advance, so that the target control plane obtains the target CRD resource.
Optionally, namespaces of control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
the monitoring module is specifically configured to:
Monitoring whether CRD resources with the same name space are updated or not;
the reporting module is specifically configured to:
and reporting the target CRD resources to a target control plane with the same naming space.
Optionally, the reporting module is specifically configured to:
and if a resource update query request sent by a pre-associated target control plane is received, reporting the target CRD resource to the pre-associated target control plane.
Optionally, each middleware is a grid configuration protocol MCP middleware.
The embodiment of the present invention further provides an electronic device, as shown in fig. 8, including a processor 801, a communication interface 802, a memory 803, and a communication bus 804, where the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804,
A memory 803 for storing a computer program;
the processor 801 is configured to implement the steps of any of the resource access methods provided in the embodiments of the present invention when executing the program stored in the memory 803.
The communication bus mentioned above for the electronic device may be a peripheral component interconnect standard (PERIPHERAL COMPONENT INTERCONNECT, PCI) bus or an extended industry standard architecture (Extended Industry Standard Architecture, EISA) bus, etc. The communication bus may be classified as an address bus, a data bus, a control bus, or the like. For ease of illustration, the figures are shown with only one bold line, but not with only one bus or one type of bus.
The communication interface is used for communication between the electronic device and other devices.
The Memory may include random access Memory (Random Access Memory, RAM) or may include Non-Volatile Memory (NVM), such as at least one disk Memory. Optionally, the memory may also be at least one memory device located remotely from the aforementioned processor.
The Processor may be a general-purpose Processor including a central processing unit (Central Processing Unit, CPU), a network Processor (Network Processor, NP), etc., or may be a digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), field-Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components.
In yet another embodiment of the present invention, there is also provided a computer readable storage medium having stored therein a computer program which, when executed by a processor, implements the steps of any of the above-described resource access methods.
In yet another embodiment of the present invention, there is also provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the steps of any of the resource access methods of the above embodiments.
In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, produces a flow or function in accordance with embodiments of the present invention, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another, for example, by wired (e.g., coaxial cable, optical fiber, digital Subscriber Line (DSL)), or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises an element.
In this specification, each embodiment is described in a related manner, and identical and similar parts of each embodiment are all referred to each other, and each embodiment mainly describes differences from other embodiments. In particular, for system and apparatus embodiments, the description is relatively simple, as it is substantially similar to method embodiments, with reference to the section of the method embodiments being relevant.
The foregoing description is only of the preferred embodiments of the present invention and is not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims (8)

1. The resource access method is characterized by being applied to a target middleware, wherein the target middleware is any one of a plurality of middleware deployed in a container cluster, a control plane and a custom CRD resource of each tenant are also deployed in the container cluster, and each middleware is pre-associated with the control plane and the custom CRD resource of only one tenant;
The method comprises the following steps:
Monitoring whether the CRD resources associated in advance are updated or not;
When the updating is monitored, determining CRD resources with the updating as target CRD resources;
reporting the target CRD resource to a pre-associated target control plane so that the target control plane acquires the target CRD resource;
The control planes of different tenants have different namespaces, and the control plane of each tenant and the CRD resource have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
The monitoring whether the CRD resource associated in advance is updated comprises the following steps:
Monitoring whether CRD resources with the same name space are updated or not;
the reporting the target CRD resource to a pre-associated target control plane includes:
and reporting the target CRD resources to a target control plane with the same naming space.
2. The method of claim 1, wherein reporting the target CRD resource to a pre-associated target control plane comprises:
and if a resource update query request sent by a pre-associated target control plane is received, reporting the target CRD resource to the pre-associated target control plane.
3. The method of any of claims 1-2, wherein each middleware is a grid configuration protocol MCP middleware.
4. A container cluster, wherein a plurality of middleware, a control plane and a custom CRD resource of each tenant are deployed in the container cluster, and each middleware is pre-associated with the control plane and the custom CRD resource of only one tenant;
The system comprises a target middleware, a target control plane and a target control plane, wherein the target middleware is used for monitoring whether a pre-associated CRD resource is updated;
the target control plane is used for receiving the target CRD resource reported by the target middleware;
The control planes of different tenants have different namespaces, and the control plane of each tenant and the CRD resource have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
the target middleware monitors whether the CRD resources associated in advance are updated or not, and the target middleware comprises:
Monitoring whether CRD resources with the same name space are updated or not;
The target middleware reports the target CRD resource to a pre-associated target control plane, which comprises the following steps:
and reporting the target CRD resources to a target control plane with the same naming space.
5. The cloud hosting system is characterized by comprising a management platform and a container cluster, wherein a plurality of middleware, a control plane and a custom CRD resource of each tenant are deployed in the container cluster, and each middleware is pre-associated with the control plane and the custom CRD resource of only one tenant;
the management platform is used for updating CRD resources of any tenant in the container cluster when receiving a resource updating instruction for the tenant;
The target middleware in the container cluster is used for monitoring whether the CRD resources associated in advance are updated or not, determining the CRD resources which are updated as target CRD resources when the CRD resources are monitored to be updated, and reporting the target CRD resources to a target control plane associated in advance so that the target control plane can issue the target CRD resources to a data plane of a tenant side;
The target middleware is any middleware in the plurality of middleware;
The namespaces of the control planes of different tenants are different, and the control plane and CRD resources of each tenant have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
The monitoring whether the CRD resource associated in advance is updated comprises the following steps:
Monitoring whether CRD resources with the same name space are updated or not;
the reporting the target CRD resource to a pre-associated target control plane includes:
and reporting the target CRD resources to a target control plane with the same naming space.
6. The resource access device is characterized by being applied to a target middleware, wherein the target middleware is any one of a plurality of middleware deployed in a container cluster, a control plane and a custom CRD resource of each tenant are also deployed in the container cluster, and each middleware is pre-associated with the control plane and the custom CRD resource of only one tenant;
the device comprises:
The monitoring module is used for monitoring whether the CRD resources associated in advance are updated or not;
the determining module is used for determining CRD resources updated when the updating is monitored and taking the CRD resources as target CRD resources;
the reporting module is used for reporting the target CRD resources to a pre-associated target control plane so that the target control plane acquires the target CRD resources;
The control planes of different tenants have different namespaces, and the control plane of each tenant and the CRD resource have the same namespaces;
the method for pre-associating the control plane of only one tenant and the customized CRD resource with each middleware comprises that each middleware is only identical with the naming space of the control plane of one tenant and the CRD resource;
The monitoring whether the CRD resource associated in advance is updated comprises the following steps:
Monitoring whether CRD resources with the same name space are updated or not;
the reporting the target CRD resource to a pre-associated target control plane includes:
and reporting the target CRD resources to a target control plane with the same naming space.
7. The electronic equipment is characterized by comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory are communicated with each other through the communication bus;
A memory for storing a computer program;
a processor for carrying out the method steps of any one of claims 1-3 when executing a program stored on a memory.
8. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored therein a computer program which, when executed by a processor, implements the method steps of any of claims 1-3.
CN202210158116.2A 2022-02-21 2022-02-21 A resource access method, apparatus, cloud hosting system, and electronic device. Active CN116668467B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210158116.2A CN116668467B (en) 2022-02-21 2022-02-21 A resource access method, apparatus, cloud hosting system, and electronic device.

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210158116.2A CN116668467B (en) 2022-02-21 2022-02-21 A resource access method, apparatus, cloud hosting system, and electronic device.

Publications (2)

Publication Number Publication Date
CN116668467A CN116668467A (en) 2023-08-29
CN116668467B true CN116668467B (en) 2025-11-04

Family

ID=87724803

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210158116.2A Active CN116668467B (en) 2022-02-21 2022-02-21 A resource access method, apparatus, cloud hosting system, and electronic device.

Country Status (1)

Country Link
CN (1) CN116668467B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119743530B (en) * 2024-12-16 2025-12-16 苏州元脑智能科技有限公司 Service access methods, devices, communication equipment and storage media

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108989091A (en) * 2018-06-22 2018-12-11 杭州才云科技有限公司 Based on the tenant network partition method of Kubernetes network, storage medium, electronic equipment
CN113709810A (en) * 2021-08-30 2021-11-26 河南星环众志信息科技有限公司 Method, device and medium for configuring network service quality

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11301276B2 (en) * 2020-06-22 2022-04-12 Hewlett Packard Enterprise Development Lp Container-as-a-service (CaaS) controller for monitoring clusters and implemeting autoscaling policies

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108989091A (en) * 2018-06-22 2018-12-11 杭州才云科技有限公司 Based on the tenant network partition method of Kubernetes network, storage medium, electronic equipment
CN113709810A (en) * 2021-08-30 2021-11-26 河南星环众志信息科技有限公司 Method, device and medium for configuring network service quality

Also Published As

Publication number Publication date
CN116668467A (en) 2023-08-29

Similar Documents

Publication Publication Date Title
US11936724B2 (en) Configuring service mesh networking resources for dynamically discovered peers or network functions
JP7724320B2 (en) Cloud Services for Cross-Cloud Operations
US20210132974A1 (en) System for peering container clusters running on different container orchestration systems
US9253265B2 (en) Hot pluggable extensions for access management system
JP7674487B2 (en) Management method and apparatus for a container cluster - Patents.com
EP3059900B1 (en) Network service template management method and device
CN115378944B (en) Network system, service grid configuration method, storage medium and electronic equipment
US11418395B2 (en) Systems and methods for an enhanced framework for a distributed computing system
US11716251B2 (en) Communication system, provider node, communication node, and method for providing a virtual network function to a customer node
WO2022143653A1 (en) Multi-cloud interface adaptation method and system based on micro-service, and storage medium
CN113794652B (en) Data processing method, device, electronic equipment and storage medium
CN110740057B (en) A business deployment method and blockchain platform
CN113810230A (en) Method, device and system for carrying out network configuration on containers in container cluster
CN109743192A (en) A container cluster configuration management method and device
CN114489761B (en) Service integration and application integration method based on container cluster
CN114938394B (en) Cross-cluster network control method, device, equipment and storage medium
CN116668467B (en) A resource access method, apparatus, cloud hosting system, and electronic device.
WO2025010985A1 (en) Service governance method and apparatus, and computer-readable storage medium
CN117319204A (en) A log management method and system
CN111970162B (en) Heterogeneous GIS platform service central control system under super-integration framework
Stackowiak Azure iot hub
CN102868594B (en) Method and device for message processing
CN105827567B (en) Service management and control method and capability open platform
CN115150811B (en) Multi-tenant operation and maintenance management method, device and system
US11768704B2 (en) Increase assignment effectiveness of kubernetes pods by reducing repetitive pod mis-scheduling

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant