RU2851031C1 - Method and system for automated management of application releases - Google Patents

Abstract

FIELD: information technology.

SUBSTANCE: invention relates to the field of hardware and software design, and in particular to a method and system for automated application release management. The following steps are carried out: a request is received to make changes to the application software code stored in at least one repository; a new version of the application is created in accordance with the changes made; the value of the application code base quality metric (Solidarity) is determined; the Solidarity metric value is compared with the acceptable metric values, and if the Solidarity metric value does not correspond to the acceptable values, a command is sent to reject the request for changes, and if the Solidarity metric value corresponds to the acceptable values, a new version of the application with the changes made is compiled into the release; deploy the new version of the application to the application service; during the operation of the application deployed to the application service, log events related to the operation of the application; determine the number of processed exceptions or unprocessed exceptions of a certain type per unit of time; determine the ratio of the number of processed exceptions or unprocessed exceptions to the number of events of a specified type for a specified time; compare the aforementioned ratio with acceptable values; determine that the aforementioned ratio exceeds acceptable values; send a command to cancel the application changes and release the previous version of the application.

EFFECT: automated management of application releases to prevent the deployment of poor-quality releases.

11 cl, 3 dwg

Description

AREA OF TECHNOLOGY

[001] The presented technical solution generally relates to the field of designing software and hardware solutions, and in particular to a method and system for automated management of application releases with the aim of preventing the deployment of low-quality releases.

LEVEL OF TECHNOLOGY

[002] Various solutions aimed at diagnosing program code are known from the prior art.

[003] For example, a method and system for checking the architecture of a software and hardware solution are known, disclosed in patent RU 2783724 C1, published on 16.11.2022. The principle underlying this method and system is the identification by an analyzer from certain artifacts (for example, binary files of distributions, source code and its lexical, syntactic, semantic analysis, data and metrics analysis, expert opinions, configuration files, AI analysis results) describing the architecture - properties, followed by the formation of a reverse architecture and a digital standard of architectural control requirements and then their comparison with the receipt of data on the compliance of the proposed (reverse) architecture with the requirements. However, the disadvantage of this method and system is that the final result of the method is compliance verification information that does not affect either the subsequent operation of the system itself in order to change (improve) the quality of its operation, or other systems controlled by it, i.e. it is a simple conditional finite state machine in its essence, the algorithm also works with a different type of data and the final result has a different function (informative).

[004] Also known are a method and system for identifying exploitable vulnerabilities in software code based on machine learning, described in patent RU 2790005 C1, published 02/14/2023. The method and system are based on the step-by-step acquisition of data on vulnerabilities in the code, the subsequent construction of an attack vector based on this data, the typification of the vulnerability in the source code with subsequent transformation into an AST tree, then the analyzer searches for an element of the attack vector by coordinates in the AST tree, records the paths between the found elements, then a vector representation is formed from the path elements and machine learning (ML) is performed on this path (vector representation) in order to identify the signature of exploited vulnerabilities. The goals achieved by the solution are a significant reduction in manual labor costs, an increase in the accuracy of detection due to the automation of analysis using the ML method and a reduction in manual errors (human factor).

[005] However, the existing solution is unable to identify systemic issues such as performance degradation/interruptions for end users, stylistic deviations in code that lead to loss of compliance with standards, and loss of flexibility in modification/scaling due to deviations from standards—i.e., quality management is not integrated. The method also lacks an approach for implementing accelerated diagnostics of code security issues comprehensively, as part of metrics-based integration/release (it only analyzes the code for vulnerabilities); it uses a different analysis method (AM). Furthermore, the method does not identify potential issues and failures with cloud billing on the platform, which could result in reputational risks and financial losses for the Company itself and for a large number of its Clients simultaneously.

[006] A method for supporting organizational system processes is known, disclosed in patent RU 2725779 C1, published 06.07.2020. The method described in this document is based on the automatic generation of data on the permissible and actual indicators of counteracting processes and the automatic management of the containment of their negative impact on organizational system processes, by comparing the indicators and responding according to predetermined scenarios, by notification and by transmitting data to the responsible organizational units (notification and instructions for implementation). A significant disadvantage of this method is that if the data on the actual indicators of the organizational system processes are lower than the data on the corresponding permissible indicators, and there are no pre-defined scenarios for the implementation of work to contain the negative impact of counteracting processes, then the method transmits the data on the actual and permissible indicators to the organizational system units for the development of new containment scenarios. This is a drawback of the automatic method, a transition to the manual level of developing a new scenario, which can take a significant amount of time.

[007] Known solutions, in general, do not provide a comprehensive and automated approach to resolving the problems of managing the level of quality, security, and reliability of releases, offering automation/analysis of individual stages, for example, only a set of scenarios, benchmark metrics, and properties of input artifacts, without considering the impact of properties on an already running system and monitoring the properties (quality, security, and reliability of operation) of the system at any given time. Known solutions for analysis use only a separate aspect of the analyzed object(s) - the code base or attack vector, without providing an aggregate result in the form of measurements and metrics of integration testing (performance, security, reliability, compliance with benchmarks and standards of the code base) of the release, vital indicators of the platform's behavior before and after the release, or as a whole. The final results also have either an informational or instructive function, without a description of any technical solution that would result in either the reaction of the system itself in the real world, or a comprehensive set of obvious advantages compared to previous levels of technology.

[008] The main problem is the lack of a single (comprehensive) technical method and system that provides automated control and management of all aspects (security, quality, performance, compliance with standards, deployment and rollback processes, automated test coverage, financial risks) of releases, both at the development stage and at the stage of industrial operation in production, with a full functional set of reactions (information, automated deployment, rollback, mechanism for manual approval of stages)

ESSENCE OF THE TECHNICAL SOLUTION

[009] The technical problem or task posed in this technical solution is the creation of a simple and reliable method and system for automated management of application releases.

[0010] The technical result achieved by solving the above-mentioned technical problem is an increase in the reliability and security of the operation of the devices of the application users by assessing the quality of the application code base and rejecting requests for changes to the application program code that lead to a decrease in the quality of the code base.

[0011] The specified technical result is achieved by implementing a method for automated management of application releases, performed by at least one computing device, containing the steps of:

- receive a request to make changes to the program code of an application stored in at least one repository;

- create a new version of the application in accordance with the changes made;

- determine the value of the quality metric of the application code base (Solidarity);

- compare the Solidarity metric value with the acceptable metric values, and if the Solidarity metric value does not correspond to the acceptable values, then send a command to reject the request for changes, and if the Solidarity metric value corresponds to the acceptable values, then build a new version of the application with the changes made to the release;

- place a new version of the application in the application service.

[0012] In one of the particular examples of the implementation of the method, the stage of determining the Solidarity metric contains stages in which:

- retrieve settings from at least one repository;

- compare repository settings with acceptable values of repository parameters and, based on the comparison results, determine the submetric value characterizing the correctness of the repository settings (GitFlow);

where the Solidarity metric is determined taking into account the value of the GitFlow submetric.

[0013] In another particular example of the implementation of the method, the step of determining the Solidarity metric comprises the steps of:

- retrieve a list of application package dependencies stored in at least one repository;

- determine the form of dependency “shouldBeStrict”: strict form of dependency or non-strict form of dependency;

- based on the form of the “shouldBeStrict” dependency, the value of the shouldBeStrict submetric is determined;

- extract dependency data from packages stored in external storage;

- determine whether dependency data with packages stored in external storage are located in the corresponding sections;

- based on the information about the location of the mentioned dependency data, the value of the “shouldBeDevDeps” submetric is determined;

- extract dependency data from packages stored in internal repositories;

- determine whether dependency data with packages stored in internal repositories is located in the appropriate sections;

- based on the information about the location of the mentioned dependency data, the value of the “shouldBeDeps” submetric is determined;

- extract data characterizing the version of a dependency and compare it with the list of current versions of dependencies;

- based on the comparison results, the value of the “shouldBeUpdated” submetric is determined;

- extract dependency data and compare it with a list of prohibited dependencies;

- based on the comparison results, the value of the “shouldNotBeUsed” submetric is determined;

- determine whether the dependency is described as a preview and, based on the results of the determination, assign a value to the “previewPackages” submetric;

- search the set of application files for a file named "renovate" and, based on the presence or absence of the file, assign a value to the file correctness submetric;

- based on the values of the submetrics “shouldBeStrict”, “shouldBeDevDeps”, “shouldBeDeps”, “shouldBeUpdated”, “shouldNotBeUsed” and the correctness of the files, the Dependencies submetric is determined;

where the Solidarity metric is determined taking into account the value of the Dependencies submetric.

[0014] In another particular example of implementing the method, the step of determining the Solidarity metric comprises the steps of:

- extract the pipeline configuration;

- compare the pipeline configuration with a given acceptable configuration to determine the value of the pipeline configuration correctness submetric;

- extract a list of environment variables for each repository;

- compare the list of environment variables with the acceptable variable values to determine the value of the repository variable quality submetric;

- determining an application status indicating whether the application has been deployed in at least one computing environment and, based on said status, assigning a deployment submetric to the application;

- based on the pipeline configuration correctness submetric, the environment variable correctness submetric, and the application deployment submetric, the CI/CD submetric value is determined;

Moreover, the Solidarity metric is determined taking into account the value of the CI/CD submetric.

[0015] In another particular example of the implementation of the method, the step of determining the Solidarity metric comprises the steps of:

- retrieve packet routing parameters from at least one repository;

- compare packet routing parameters with acceptable packet parameter values;

- based on the comparison results, the value of the Routing submetric is assigned;

- moreover, the Solidarity metric is determined taking into account the value of the Routing submetric.

[0016] In another particular example of implementing the method, the step of determining the Solidarity metric comprises the steps of:

- determine the versions of the specified packages;

- compare the versions of the specified packages with the permissible values of package versions;

- based on the comparison results, the value of the Environments submetric is assigned;

where the Solidarity metric is determined taking into account the value of the Environments submetric.

[0017] In another particular example of the implementation of the method, the step of determining the Solidarity metric comprises the steps of:

- check the presence of the package.json and .gitlab/CODEOWNERS files;

- check the presence of information in the files about the code owner, product owner, technical leader;

- based on the presence or absence of the mentioned information, the value of the CODEOWNERS submetric is assigned;

where the Solidarity metric is determined taking into account the value of the CODEOWNERS submetric.

[0018] In another particular example of implementing the method, the step of determining the Solidarity metric comprises the steps of:

- analyze the code using the ESLint tool;

- determine the number of “warning” values in the analysis results;

- based on the number of “warning” values, the value of the CodeStyle submetric is determined;

where the Solidarity metric is determined taking into account the value of the CodeStyle submetric.

[0019] In another particular example of implementing the method, the step of determining the Solidarity metric comprises the steps of:

- extract helm settings and the project name, as well as the application name in the repository;

- compare the application name in the extracted data and, based on the comparison results, assign a value to the Gitlab Structure submetric;

Moreover, the Solidarity metric is determined taking into account the value of the Gitlab Structure submetric.

[0020] In another particular example of implementing the method, the Solidarity metric is determined based on the submetrics GitFlow, Dependencies, CI/CD; Routing, Environments, CODEOWNERS, CodeStyle, Gitlab Structure and the significance coefficients of each submetric

[0021] In another particular example of implementing the method, additional steps are performed in which:

- extract information about the number of unit tests and end-to-end testing performed;

- based on the number of tests, a QA metric is determined for the application;

- the value of the QA metric is compared with the permissible values of the metric, and if the value of the QA metric does not correspond to the permissible values, then a command is sent to reject the request for making changes, and if the value of the QA metric corresponds to the permissible values, then a new version of the application is assembled with the changes made to the release. [0022] In another particular example of implementing the method, additional steps are performed in which:

- during the operation of an application hosted in the application service, events related to the operation of the application are recorded;

- determine the number of handled exceptions or unhandled exceptions of a certain type per unit of time;

- determines the ratio of the number of handled exceptions or unhandled exceptions to the number of events of the type specified for them in a specified time;

- compare the mentioned ratio with the permissible values;

- determine that the said ratio exceeds the permissible values;

- send a command to cancel changes to the application and release the previous version of the application.

[0023] In another preferred embodiment of the claimed solution, a system for automated management of application releases is provided, comprising at least one computing device and at least one memory containing machine-readable instructions that, when executed by at least one computing device, perform the above-mentioned method.

BRIEF DESCRIPTION OF DRAWINGS

[0024] The features and advantages of the present invention will become apparent from the following detailed description of the invention and the accompanying drawings, in which:

Fig. 1 shows an example of the implementation of an automated application release management system.

Fig. 2 shows an example diagram of a method for automated management of application releases.

Fig. 3 shows an example of the general appearance of a computing device. DETAILED DESCRIPTION OF THE INVENTION

[0025] Below, the concepts and terms necessary for understanding this technical solution will be described.

[0026] In this technical solution, the term “system” means, among other things, a computer system, a computer (electronic computer), a numerical control (CNC), a PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, clearly defined sequence of operations (actions, instructions).

[0027] A command processing unit is an electronic unit or integrated circuit (microprocessor) that executes machine instructions (programs).

[0028] The command processing unit reads and executes machine instructions (programs) from one or more data storage devices. The data storage devices may include, but are not limited to, hard disk drives (HDD), flash memory, ROM (read-only memory), solid-state drives (SSD), and optical drives.

[0029] In accordance with the diagram shown in Fig. 1, the automated application release management system, in a particular embodiment of its implementation, contains at least one developer device 10, an application service 20, an application user device 30, a metrics analysis service 100, at least three repositories 101, 102, 103, and a dispatch device 200 and a service 300 for processing live application metrics, connected by communication channels.

[0030] Each mentioned component of the system can be implemented on the basis of a computing device equipped with logic elements on transistors to perform the functions assigned to them.

[0031] In accordance with a specified schedule, the metrics analysis unit 1011 collects metrics of data about service repositories. In particular, the unit 1011 collects settings of repositories 101, 102, 103, as well as data about at least one application through which at least one service is provided. The mentioned application may be a set of files containing the program code of the application, located in repositories 101, 102, 103.

[0032] For example, an application may be designed to provide a key management service, wherein:

- repository 101 may contain a first set of files with the client program code of the application;

- repository 102 may contain a second set of files with a library of visual components, for example, with the parameters of a button, according to which it will be displayed in the interface to the user of the service,

- repository 103 may contain a third set of files with program code describing the method of delivering program code from the container to the user.

[0033] Once the said data has been collected, block 1011 proceeds to determining the application codebase quality metric (Solidarity). The Solidarity metric may be determined based on submetrics that characterize:

- correctness of repository settings (GitFlow);

- the level of stability of package dependencies (Dependencies) (see, for example, the article “Stable Dependencies”), located on the Internet at: https://deviq.com/principles/stable-dependencies);

correctness of the deployment parameters of application environment variables (CI/CD);

- correctness of routing;

- correctness of environment configuration files (Environments);

- correctness of the file for indicating code owners (CODEOWNERS);

- correctness of the program code style (CodeStyle);

- correctness of the program code structure (Gitlab Structure).

[0034] For example, to determine a GitFlow submetric, block 1011 may compare the settings of the repositories in which the application is stored with the permissible values of the repository parameters. To obtain the settings, block 1011 may send a request, for example, in HTTP format, to the data storage block of each repository in which the repository settings are stored, and which, in response to said request, searches for a file with the repository settings, retrieves information about the repository settings from memory, for example, in JSON format, and sends them to block 1011.

[0035] The received repository settings are compared by block 1011 with the allowed setting values and, based on the comparison results, the Gitflow submetrics are determined. For example, from the received settings, block 1011 may extract the values specified in the fields "default_branch", "merge_method", "squash_option" and compare them with the allowed setting values "master", "fast_forward" and "never", respectively, specified in the data storage block 1012. If the repository settings values match the allowed setting values, then block 1011 assigns the maximum value of the GitFlow submetric, for example, 100%. If the repository settings values do not match the allowed setting values, then block 1011 assigns the minimum value of the GitFlow submetric, for example, 0%. If only a portion of the repository settings match the allowed values, then block 1011 can assign a value as a GitFlow submetric that characterizes the ratio between the number of repository parameter values that match the allowed values to the number of all possible repository parameters.

[0036] Next, block 1011 proceeds to the stage of determining the Dependencies submetric. To determine said submetric, block 1011 may be equipped with a program code parser and a repository file API, through which block 1011 searches for the package.json file in the data storage block of each repository and analyzes the program code of said file, for example, semantic analysis to search for the dependencies and devDependencies fields, which contain a list of application package dependencies.

[0037] For example, the list of dependencies may contain information indicating the presence of a dependency of a package from the first set of files stored in the repository 101 with at least one package from the second set of files stored in the repository 102, containing the parameters of the button, in accordance with which it will be displayed in the interface to the user of the service.

[0038] The list of dependencies stored in repository 101 may contain the following data:

where "button" is the package ID (e.g. package name), the identifier "102" indicates that the package "button" comes from repository 102.

[0039] After the list of dependencies is retrieved, block 1011 determines the form of the "shouldBeStrict" dependency, for example, by matching the characters of the specified dependency. The "shouldBeStrict" dependency is represented in the above example as "modal-102: ^∧ 0.0.1". Accordingly, if block 1011 determines that the "shouldBeStrict" dependency is specified in a non-strict form, for example, the specified dependency contains a check mark ( ^∧ ), then block 1011 assigns a minimum value of the shouldBeStrict submetric, for example, 0%. If block 1011 determines that the "shouldBeStrict" dependency is specified in a strict form, i.e., the check mark ( ^∧ ) is absent, then block 1011 assigns a maximum value of the shouldBeStrict submetric, for example, 100%.

[0040] Additionally, block 1011 defines a dependency submetric "shouldBeDevDeps" that characterizes the correct placement of dependency data with packages stored in external repositories in the corresponding sections. Since the "shouldBeStrict" dependency is represented in the above example as "eslint 0.0.1" and since said dependency is not associated with repositories 101, 102, or 103, it must be located in the devDependencies section. Accordingly, if the "shouldBeDevDeps" dependency (in particular, "eslint 0.0.1") is not located in the devDependencies section, block 1011 assigns a minimum value for the "shouldBeDevDeps" submetric, for example, 0%. If the "shouldBeDevDeps" dependency is in the devDependencies section, then block 1011 assigns the maximum value to the "shouldBeDevDeps" submetric, for example, 100%.

[0041] Additionally, block 1011 defines a dependency submetric shouldBeDeps that characterizes the correct placement of dependency data with packages stored in internal repositories (in particular, in repositories 101, 102, 103) in the corresponding sections. The shouldBeDeps dependency is represented in the above example as "link-102:0.0.1" and must be located in the dependencies section, since it has a link with our internal repository 102. Accordingly, if the "shouldBeDeps" dependency (in particular, "link-102:0.0.1") is not located in the dependencies section, then block 1011 assigns the minimum value of the "shouldBeDeps" submetric, for example, 0%. If the "shouldBeDeps" dependency is located in the dependencies section, then block 1011 assigns the maximum value of the "shouldBeDeps" submetric, for example, 100%.

[0042] Additionally, block 1011 determines the shouldBeUpdated dependency submetric, which characterizes the correctness of the dependency version. To determine said submetric, a list of current dependency versions may be pre-loaded into block 1012, containing the dependency ID and its current version number. Accordingly, block 1011 extracts the data characterizing the dependency version, for example, "0.0.1" from the button-102 field, and compares it with the list of current dependency versions. If the data characterizing the dependency version matches the current version, then block 1011 assigns the maximum value of the "shouldBeUpdated" submetric, for example, 100%. If the data characterizing the dependency version does not match the current version (for example, the current version is 0.0.2), then block 1011 assigns the minimum value of the "shouldBeUpdated" submetric, for example, 0%.

[0043] Additionally, block 1011 determines a shouldNotBeUsed dependency submetric that characterizes the safety of the used dependencies. To determine said submetric, a list of prohibited dependencies may be preloaded into block 1012, which may indicate, for example, the "buttonBlackl_ist-102" dependency and its version. Accordingly, block 1011 extracts the dependency data from the package and compares it with the list of prohibited dependencies. If the package dependency data contains a prohibited dependency, in particular "buttonBlackl_ist-102", then block 1011 assigns a minimum value of the "shouldNotBeUsed" submetric, for example, 0%. If the package dependency data does not contain a prohibited dependency, then block 1011 assigns a maximum value of the "shouldNotBeUsed" submetric, for example, 100%.

[0044] Additionally, block 1011 may determine that a dependency is described as a preview and, based on this information, assign a value to the "previewPackages" submetric. For example, block 1011 may retrieve the "form-102:0.0.1-assbfabd" dependency and determine that the specified dependency contains a letter postfix in its version and decide that the dependency is described as a preview, after which it may assign a minimum value to the "previewPackages" submetric, for example, 0%. Accordingly, if the dependency does not contain a letter postfix in its version, then block 1011 may assign a maximum value to the "previewPackages" submetric, for example, 100%.

[0045] Block 1011 also searches the set of files stored in the repository for a file named "renovate." If the file is not present in the set of files, block 1011 assigns a minimum value for the file correctness submetric, such as 0%. If the file is present in the set of files, block 1011 assigns a maximum value for the file correctness submetric, such as 100%.

[0046] After the "shouldBeStrict," "shouldBeDevDeps," "shouldBeDeps," "shouldBeUpdated," "shouldNotBeUsed," and file correctness submetrics are determined, block 1011 determines a metric significance factor (e.g., from 0 to 10) that determines the submetric's contribution to the total value of the Dependencies submetric. The total Dependencies submetric is calculated as the sum of the aforementioned submetrics multiplied by the corresponding submetric significance factor.

[0047]Next, block 1011 proceeds to defining the CI/CD submetric. To determine said submetric, block 1011 requests the pipeline configuration from the data storage block of each repository, for example, in the form of the gitlab-ci.yaml file from repository 101. For example, the pipeline configuration in said file may be represented as the program code: include:

- project: cp/infrastructure/devops

ref: master

file: /gitlab-ci/entries/front.yaml

[0048] The mentioned gitlab-ci.yaml file stores links to files from repository 103, which contain the method for delivering the program code to the user.

[0049] The received pipeline configuration is compared with a given valid configuration to determine a value of the pipeline configuration correctness submetric. For example, the valid configuration data may specify that the pipeline configuration must contain the fields "project", "ref", "file" with the values "cp/infrastructure/devops", "master", "/gitlab-ci/entries/front.yaml", respectively. If at least one field or value is missing from the pipeline configuration or the field value does not match the valid ones, then block 1011 assigns the lowest value of the pipeline configuration correctness submetric, for example, 0%. If the pipeline configurations match the valid configuration, then block 1011 assigns the highest value of the pipeline configuration correctness submetric, for example, 100%.

[0050] Additionally, block 1011 may be configured to determine the correctness of the use of environment variables for each repository. Environment variables are a set of values that determine the behavior and settings of the operating system, as well as other programs running in this environment (see, for example, the article "What are environment variables and how to use them" posted on the Internet at: https://sky.pro/media/chto4akoe-pere To determine the mentioned submetric, block 1011 requests a list of variables from the data storage block of each repository by means of a corresponding request, in response to which a list of variables for the repository is sent. Then, the received list of repository variables is compared by block 1011 with the specified acceptable values of variables. For example, block 1011 can analyze the mentioned list and determine the presence in the mentioned list of information on at least one variable and assign the lowest value of the repository variables quality submetric, for example, 0%. At the same time, if the variable list is empty, i.e. there is no information on the variables, then block 1011 can assign a high value of the repository variables correctness use submetric, for example, 100%.

[0051] Additionally, block 1011 may be configured to determine a submetric of application deployment. To determine said submetric, block 1011 determines the status of the application, indicating whether the application has been deployed in at least one computing environment, such as a "development" or "production" environment. When an application is deployed in a computing environment, corresponding information is generated in the program code of the application, describing the identifier of the computing environment, the time of deployment, information about the successful deployment of the application in the computing environment, etc. For example, said information may be represented in the following form:

[0052] Accordingly, block 1011 queries the repository data storage block 1012 for the above information, retrieves information about the status of the application, and if said status indicates that the application has been deployed in the computing environment, i.e., the "status" field indicates "success", then block 1011 can assign a high value of the application deployment submetric, for example, 100%. If the status indicates that the application has not been deployed in the computing environment, then block 1011 can assign the lowest value of the application deployment submetric, for example, 0%.

[0053]Next, based on the pipeline configuration correctness submetric, the environment variable correctness submetric, and at least one application deployment submetric, block 1011 determines the CI/CD submetric. For example, block 1011 may assign a value that characterizes the percentage of submetrics that characterize a high submetric value relative to the total number of submetrics as the CI/CD submetric. Additionally, block 1011 may determine the Routing submetric.

[0054] To determine said submetric, block 1011 queries the data storage block of each repository for package routing parameters, which may be represented in the package.json file, for example, in the form of package names. The received package routing parameters are compared by block 1011 with the allowed values of the package parameters specified in block 1012, for example, with the allowed names of the packages, and if the package routing parameters correspond to said allowed values, then block 1011 assigns a high value of the metric, for example, 100%. If the package routing parameters do not correspond to said allowed values, then block 1011 assigns a minimum value of the metric, for example, 0%.

[0055] Also, in addition, a list of required packet routing parameters and a list of invalid routing parameters may be stored in block 1012. For example, the package name "@sbercloud/ft-router" may be specified as required packet routing parameters, and the names "react-router-dom, found, @reach/router" may be specified as invalid routing parameters. Accordingly, if block 1011 determines that the packet routing parameters do not contain the required parameters or contain invalid parameters, block 1011 assigns a minimum metric value, for example, 0%. Otherwise, block 1011 assigns a maximum metric value, for example, 100%.

[0056] Additionally, block 1011 may be configured to define the Environments submetric. To determine this submetric, block 1011 queries the data storage unit of each repository for versions of the specified packages, similar to the Dependencies metric, which in turn is intended to check the versions of the package.json and .nmprc files, and compares them with the permissible package version values. For example, package versions may be specified in the npm and node-js fields for compliance with the specified permissible package version values. The list of packages that are checked within the Environments submetric (it is verified that they are used in the project and their versions are up-to-date, not less than some predefined ones): @sbercloud/ft-config-docker @sbercloud/ft-all-linters-pack @sbercloud/ft-config-tsconfig @sbercloud/ft-config-webpack-spa @sbercloud/ft-config-spa-entry

[0057] If the package version does not match the allowed version values or the received response contains an .nmprc file, then block 1011 assigns the minimum value of the Environments submetric, for example, 0%. If the package version matches the allowed version values and the received response does not contain an .nmprc file, then block 1011 assigns the maximum value of the Environments submetric, for example, 100%.

[0058] Additionally, block 1011 may be configured to determine the CODEOWNERS submetric. To determine said submetric, block 1011 accesses the previously obtained package.json file, as well as the .gitlab/CODEOWNERS file, checks for their presence, checks for the presence of information about the code owner, product owner, and technical lead, which may be specified, for example, in the "codeOwners", "productOwner", and "techl_ead" fields. The "pid" field may also be additionally checked for the presence of a project ID. If the information about the code owner or the project ID is absent, then block 1011 assigns a minimum value to the CODEOWNERS submetric, for example, 0%. If the information about the code owner and the project ID is present in the package.json file, then block 1011 assigns a maximum value to the CODEOWNERS submetric, for example, 100%.

[0059] Additionally, block 1011 may be configured to determine a CodeStyle submetric. To determine said submetric, block 1011 launches a code analyzer implemented on the basis of the ESLint tool, which extracts the program code of the application from the data storage blocks of the repositories and analyzes it using known methods. Based on the analysis result, the code analyzer generates a file with the analysis results, after which block 1011 analyzes said file with the results to determine the number of messages in the field of which the value "warning" is contained. Then, based on the determined number of messages, block 1011 determines the CodeStyle submetric. For example, a minimum value of the CodeStyle submetric, such as 0%, may be assigned to 30 messages, and a maximum value of the CodeStyle submetric, such as 100%, may be assigned to 0 messages.

[0060] Additionally, block 1011 may be configured to determine Gitlab Structure submetrics. To determine said submetric, block 1011 requests helm settings and the project name, as well as the name of the application in the repository, from data storage block 1012. If the name in the helm settings matches the name of the application in the repository, and they also contain information about the project name, then block 1011 assigns the maximum value of the Gitlab Structure submetric, for example, 100%. If the name in the helm settings does not match the name of the application in the repository, or the application name or the project name is missing, then block 1011 assigns the minimum value of the Gitlab Structure submetric, for example, 0%.

[0061] Next, block 1011, based on at least one submetric or all submetrics of GitFlow, Dependencies, CI/CD; Routing, Environments, CODEOWNERS, CodeStyle, Gitlab Structure, determines the value of the Solidarity metric. To determine the value of the Solidarity metric, block 1011 determines the importance coefficient of each submetric, previously specified in block 1012 depending on its importance, after which, based on at least one submetric and the importance coefficient, the value of the Solidarity metric is determined, for example, by multiplying at least one submetric and the importance coefficient.

[0062] To obtain a more precise value of the Solidarity metric, it can be determined on the basis of two or more of the mentioned submetrics (including all of the mentioned submetrics) as a weighted average value determined on the basis of the values of the submetrics' significance coefficients and the values of the mentioned submetrics, which is then assigned as the value of the Solidarity metric.

[0063] In an alternative embodiment of the presented solution, the value of the Solidarity metric may be defined as the smallest value from the submetrics GitFlow, Dependencies, CI/CD; Routing, Environments, CODEOWNERS, CodeStyle, Gitlab Structure.

[0064] Next, block 1011 sends the value of the Solidarity metric and submetrics to the data processing block 201 of the dispatch device 200, which stores them in the storage 203 and compares them with the permissible values of the Solidarity metrics. If the value of the Solidarity metric corresponds to the permissible values, then the data block 201 completes its work until new data arrives from block 1011. If the value of the Solidarity metric does not correspond to the permissible values, then block 201 generates a notification to the developer's device to notify about the need to make changes to the repository settings or the program code of the application. Also, after a specified time, block 201 generates a command to each repository in which the application is stored to block them.

[0065] Additionally, block 1011 may be configured to determine a QA metric—a measure that provides a numerical value characterizing the performance quality of the application's software code. To determine said metric, block 1011 may access the memory area of the data storage unit of each repository, which stores reports on running unit tests and end-to-end testing (e2e tests). Unit tests and e2e tests may be executed by a code analyzer implemented using the vitest and test-cafe tools, for example, after uploading the application to the repositories, after making changes to the application's software code, or according to a schedule set by the developer.

[0066] From said reports, block 1011 extracts information about the number of said tests performed, and then, based on the number of tests, determines a QA metric for the application.

[0067] For example, block 1011 may determine that one unit test and two e2e tests have been executed. Block 1011 then accesses a memory in which a minimum value for the number of tests can be specified, for example, 20 unit tests and 20 e2e tests, after which block 1011, based on the number of texts executed and the minimum value of tests, determines a unit test submetric and an e2e test submetric, based on the values of which it determines a QA metric, for example, a QA metric can be defined as the average value of the mentioned submetrics. [0068] For example, block 1011 may determine that 1 unit test and 2 e2e tests have been executed with respect to the application. Accordingly, for a minimum value of the number of tests of 20, the unit test submetric will be equal to 5%, and the e2e test submetric will be equal to 10%. Thus, the application QA metric will be defined as 7.5%. The repository QA metric can also be defined as the average value of all application QA metrics.

[0069]Next, block 1011 sends the value of the QA metric of the application and/or repository to the data processing block 201 of the dispatching device 200, which stores them in the storage 203 and also compares them with the permissible values of the QA metrics. If the value of the QA metric of the application and/or repository corresponds to the permissible values, then the data block 201 completes its work until new data arrives from block 1011. If the value of the QA metric of the application or repository does not correspond to the permissible values, then block 201 generates a request to the developer device 10 to make changes to the repository settings or the program code of the application.

[0070] During the operation of the information processing system, a request (see Fig. 2, position 400) to make changes to the program code of the application, for example, stored in the repository 101, may be sent via the developer device 10. In particular, the request to make changes may contain at least one of:

- corrected program code of the application;

- adjusted list of package dependencies;

- refined application deployment parameters and environment variables;

- refined routing parameters;

- adjusted network environment files;

- corrected files to indicate code owners.

[0071] The said request is received by the request processing block 1013, which generates (401) in block 1012 a new version of the application in accordance with the data of the said request. After the new version of the application has been generated, the metrics analysis block 1011 is launched, which, in the manner described earlier, determines (402) the submetrics GitFlow, Dependencies, CI/CD, Routing, Environments, CODEOWNERS, CodeStyle, Gitlab Structure determines the overall value of the Solidarity metric, as well as the QA metric.

[0072] Next, block 1011 sends the value of the Solidarity metric of the new version of the application, the submetrics and the QA metric to the data processing block 201 of the dispatching device 200, which stores them in the storage 203 and also compares (403) with the permissible values of the said metrics. If the value of the Solidarity metric or the QA metric does not correspond to the permissible values, then block 201 generates a command to block 1013 to reject the request for making changes and send a corresponding notification to the developer device 10.

[0073] If the values of the Solidarity metric and the QA metric correspond to the permissible values, then block 201 generates a command to block 1013 for making changes to the program code of the application, after which block 1013 sends a command to block 1014 for assembling the application new version into a release (404). In particular, block 1014 extracts data about the application stored in the repositories and places them in the application service 20, after which the user can connect to the service 20 via device 30 and download the application. [0074] Also, block 201 can additionally compare the Solidarity metrics and QA metrics of the new version of the application with the specified metrics of the previous/current version of the application and in the event that the said metrics of the new application are less than the metrics of the previous/current version of the application, block 201 generates a command to block 1013 for rejecting the request for making changes in the manner described earlier.

[0075] This improves the reliability and security of the application's user devices by assessing the quality of the application's code base and rejecting requests for changes to the application's program code that would reduce the quality of the code base. Furthermore, the reliability and security of the application's user devices is improved by assessing the quality of the application's program code and rejecting requests for changes to the application's program code that would reduce the quality of the program code.

[0076] During the operation of the application, which is in the production environment after the build and release, the service 20 registers various types of events related to the operation of the application using known methods. Methods for registering the mentioned events are disclosed, for example, in the article "Catch Me If You Can: A Guide to Exception Handling in Python", published on the Internet at https://habr.com/ru/companies/wunderfund/articles/736526/, and the events can be: data logging, displaying a message, an attempt to restore the operation of the program after an error has occurred, etc. The event data is sent by the service 20 to the service 300 for processing and storing live metrics of the application, in particular, to the processing block 301, which for each type of event determines a live metric that characterizes the number of events (related to the operation of the application) of a certain type per unit of time. [0077] For example, a live metric can characterize the number of requests for an application interface page per unit of time, where a request for an interface page is a specific type of event.

[0078] A live metric can also characterize the number of errors (handled exceptions) of a certain type per unit of time. For example, a live metric can characterize the number of incorrectly specified dates by the user in the application interface launched using service 20, which can be registered by said service under the name "DataErrorDate."

[0079] Additionally, the live metric can characterize the number of unidentified errors per unit of time, i.e. unhandled exceptions registered by known methods (see, for example, the article “Unhandled C++ Exceptions” posted on the Internet at: https://learn.microsoft.com/ru-ru/cpp/cpp/unhandled-cpp-exceptions?view=msvc-170).

[0080] Additionally, a live metric can characterize the number of API calls per unit of time, for example, the number of calls to the "api/address" API method. Additionally, a live metric can characterize the number of a certain type of user actions per unit of time related to the operation of the application, for example, successfully sent messages through the application interface.

[0081] Next, block 301 determines the ratio of the number of processed exceptions and/or unhandled exceptions to the number of events of a specified type (for example, in storage 302) for them over a specified time. For example, for the number of an incorrectly specified date by the user, the event type can be set to the number of successfully processed events, for example, requests for an interface page in 10 minutes, and for unhandled exceptions, the event type can be set to successfully sent messages in 10 minutes. The obtained value of the ratio of the number of processed exceptions or unhandled exceptions to the number of events of a specified type for them over a specified time is compared with the permissible values specified in storage 302, and if the said value of the ratios exceeds the permissible value, then block 301 generates a command in the dispatcher 200 to cancel the changes to the application.

[0082] Processing of such metrics can be performed by a specialized tool, for example, Prometheus. Its functionality includes calling a pre-specified third-party API in the event of reaching certain values for any metrics or their derivatives (block 301). For example, in the storage 302 of the live metrics processing service 300, it can be established that it is necessary to call the application release rollback API in the dispatcher 200 if the number of errors, in particular processed exceptions, over the last, for example, 10 minutes, exceeds 1% (the acceptable value) of the total number of interface page requests.

[0083] The dispatcher unit 201 of the 200, after receiving a command to cancel changes to the application, generates a change request to at least one repository in which the application is stored, for example, in the form of a reverse commit, which contains a cancellation of the latest changes, i.e., the release of the previous version of the application. The dispatcher also sends a notification to the developer device 10 that the live metrics of the application's operation contain deviations.

[0084] In this way, the reliability and security of the operation of the devices of the application users is further increased by assessing the number of errors registered as a result of the operation of the application and rolling back the application to the previous version in the event that the number of errors exceeds the permissible values.

[0085] In general (see Fig. 3), the computing device contains one or more processors (501), memory means such as RAM (502) and ROM (503), input/output interfaces (504), input/output devices (505), and a device for network interaction (506), united by a common information exchange bus.

[0086] The processor (501) (or several processors, a multi-core processor, etc.) can be selected from a range of devices that are widely used at present, for example, from manufacturers such as: Intel™, AMD™, Apple™, Samsung Exynos™, MediaTEK™, Qualcomm Snapdragon™, etc. Under the processor or one of the processors used in the device (500), it is also necessary to take into account a graphic processor, for example, an NVIDIA or Graphcore GPU, the type of which is also suitable for the full or partial implementation of the method, and can also be used for training and applying machine learning models in various information systems.

[0087] RAM (502) is a random access memory and is intended for storing machine-readable instructions executable by the processor (501) for performing the necessary operations for logical data processing. RAM (502), as a rule, contains executable instructions of the operating system and the corresponding software components (applications, software modules, etc.). In this case, the available memory capacity of a graphics card or graphics processor may serve as RAM (502).

[0088] ROM (503) represents one or more permanent data storage devices, such as a hard disk drive (HDD), a solid state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media (CD-R/RW, DVD-R/RW, BlueRay Disc, MD), etc.

[0089] To organize the operation of the components of the device (500) and to organize the operation of external connected devices, various types of I/O interfaces (504) are used. The selection of the corresponding interfaces depends on the specific design of the computing device, which may be, but are not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[0090] To ensure user interaction with the device (500), various I/O information means (505) are used, for example, a keyboard, a display (monitor), a touch display, a touchpad, a joystick, a mouse, a light pen, a stylus, a touch panel, a trackball, speakers, a microphone, augmented reality means, optical sensors, a tablet, light indicators, a projector, a camera, biometric identification means (a retinal scanner, a fingerprint scanner, a voice recognition module), etc.

[0091] The network interaction means (506) provides for the transmission of data via an internal or external computer network, for example, an Intranet, the Internet, a LAN, etc. One or more means (506) may be, but are not limited to: an Ethernet card, a GSM modem, a GPRS modem, an LTE modem, a 5G modem, a satellite communication module, an NFC module, a Bluetooth and/or BLE module, a Wi-Fi module, etc.

[0092] Additionally, satellite navigation tools may also be used within the device (500), for example, GPS, GLONASS, BeiDou, Galileo. The specific selection of elements of the device (500) for implementing various software and hardware architectural solutions may vary while maintaining the required functionality.

[0093] Modifications and improvements to the above-described embodiments of the present technical solution will be apparent to those skilled in the art. The preceding description is provided only as an example and does not carry any limitations. Therefore, the scope of the present technical solution is limited only by the scope of the appended claims.

Claims (70)

1. A method for automated management of application releases, performed by at least one computing device, comprising the steps of: - receive a request to make changes to the program code of an application stored in at least one repository; - create a new version of the application in accordance with the changes made; - extract a list of package dependencies of the new version of the application; - determine the form of dependency “shouldBeStrict”: strict form of dependency or non-strict form of dependency; - based on the form of the “shouldBeStrict” dependency, the value of the shouldBeStrict submetric is determined; - extract dependency data from packages stored in external storage; - determine whether dependency data with packages stored in external repositories is located in the corresponding sections; - based on the information about the location of the mentioned dependency data, the value of the “shouldBeDevDeps” submetric is determined; - extract dependency data from packages stored in internal repositories; - determine whether dependency data for packages stored in internal repositories is located in the appropriate sections; - based on the information about the location of the mentioned dependency data, the value of the “shouldBeDeps” submetric is determined; - extract data characterizing the version of a dependency and compare it with the list of current versions of dependencies; - based on the comparison results, the value of the “shouldBeUpdated” submetric is determined; - extract dependency data and compare it with a list of prohibited dependencies; - based on the comparison results, the value of the “shouldNotBeUsed” submetric is determined; - determine whether the dependency is described as a preview, and based on the results of the determination, assign a value to the “previewPackages” submetric; - search the set of application files for a file named "renovate" and, based on the presence or absence of the file, assign a value to the file correctness submetric; - based on the values of the submetrics “shouldBeStrict”, “shouldBeDevDeps”, “shouldBeDeps”, “shouldBeUpdated”, “shouldNotBeUsed” and the correctness of the files, the Dependencies submetric is determined; - determine the value of the application codebase quality metric (Solidarity) taking into account the value of the Dependencies submetric; - compare the Solidarity metric value with the acceptable metric values, and if the Solidarity metric value does not correspond to the acceptable values, send a command to reject the change request, and if the Solidarity metric value corresponds to the acceptable values, then build a new version of the application with the changes made to the release; - place a new version of the application in the application service; - during the operation of an application hosted in the application service, events related to the operation of the application are recorded; - determine the number of handled exceptions or unhandled exceptions of a certain type per unit of time; - determines the ratio of the number of handled exceptions or unhandled exceptions to the number of events of the type specified for them in a given time; - compare the mentioned ratio with the permissible values; - determine that the said ratio exceeds the permissible values; - send a command to cancel changes to the application and release the previous version of the application. 2. The method according to paragraph 1, characterized in that the stage of determining the Solidarity metric contains stages in which: - retrieve settings from at least one repository; - compare repository settings with acceptable values of repository parameters and, based on the comparison results, determine the submetric value characterizing the correctness of the repository settings (GitFlow); where the Solidarity metric is determined taking into account the value of the GitFlow submetric. 3. The method according to paragraph 1, characterized in that the stage of determining the Solidarity metric contains stages in which: - extract the pipeline configuration; - compare the pipeline configuration with a given acceptable configuration to determine the value of the pipeline configuration correctness submetric; - extract a list of environment variables for each repository; - compare the list of environment variables with the acceptable variable values to determine the value of the repository variable quality submetric; - determining an application status indicating whether the application has been deployed in at least one computing environment and, based on said status, assigning a deployment submetric to the application; - based on the pipeline configuration correctness submetric, the environment variable correctness submetric, and the application deployment submetric, the CI/CD submetric value is determined; Moreover, the Solidarity metric is determined taking into account the value of the CI/CD submetric. 4. The method according to paragraph 1, characterized in that the stage of determining the Solidarity metric contains stages in which: - retrieve packet routing parameters from at least one repository; - compare packet routing parameters with acceptable packet parameter values; - based on the comparison results, the value of the Routing submetric is assigned; - moreover, the Solidarity metric is determined taking into account the value of the Routing submetric. 5. The method according to paragraph 1, characterized in that the stage of determining the Solidarity metric contains stages in which: - determine the versions of the specified packages; - compare the versions of the specified packages with the permissible values of package versions; - based on the comparison results, the value of the Environments submetric is assigned; where the Solidarity metric is determined taking into account the value of the Environments submetric. 6. The method according to paragraph 1, characterized in that the stage of determining the Solidarity metric contains stages in which: - check the presence of package.json and .gitlab/CODEOWNERS files; - check the presence of information about the code owner, product owner, and technical leader in the files; - based on the presence or absence of the mentioned information, the value of the CODEOWNERS submetric is assigned; where the Solidarity metric is determined taking into account the value of the CODEOWNERS submetric. 7. The method according to paragraph 1, characterized in that the stage of determining the Solidarity metric contains stages in which: - analyze the code using the ESLint tool; - determine the number of “warning” values in the analysis results; - based on the number of “warning” values, the value of the CodeStyle submetric is determined; where the Solidarity metric is determined taking into account the value of the CodeStyle submetric. 8. The method according to paragraph 1, characterized in that the stage of determining the Solidarity metric contains stages in which: - extract helm settings and the project name, as well as the application name in the repository; - compare the application name in the extracted data and, based on the comparison results, assign a value to the Gitlab Structure submetric; Moreover, the Solidarity metric is determined taking into account the value of the Gitlab Structure submetric. 9. The method according to paragraph 1, characterized in that the Solidarity metric is determined on the basis of the submetrics GitFlow, Dependencies, CI/CD; Routing, Environments, CODEOWNERS, CodeStyle, Gitlab Structure and the significance coefficients of each submetric. 10. The method according to paragraph 1, characterized in that the following steps are additionally performed: - extract information about the number of unit tests and end-to-end testing performed; - based on the number of tests, a QA metric is determined for the application; - compare the value of the QA metric with the acceptable values of the metric, and if the value of the QA metric does not correspond to the acceptable values, send a command to reject the request for making changes, and if the value of the QA metric corresponds to the acceptable values, then build a new version of the application with the changes made to the release. 11. An automated application release management system comprising at least one computing device and at least one memory containing machine-readable instructions that, when executed by at least one computing device, perform the method according to any one of paragraphs 1-10.