RU2633929C1 - System and method of evaluation of medical research according to international standards - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: they use a set of "evaluation" criteria with quantitative parameters. The basic criteria used are: the dynamics of the publication and / or citations activity, the availability of the research trend in the number of 1% of the most cited publications from the total number of cited publications, the availability of the research trend among 0.1% of the most cited publications from the total number of cited publications, the availability of the research trend in the world research front lines, formed by the content of scientific publications reflecting mutual citation for at least the last 5 years, the dynamics of the patent (inventive) activity. Also, the ratio of the number of research organizations and manufacturing companies among the first 10-20 rights holders selected by the maximum number of patent documents for the research trend, possessed by them. Comparison of the dynamics of the filing of patent applications and the dynamics of patent granting, the dynamics of grant financing in the research trend.

EFFECT: automatization of evaluating process for research projects or research areas in the subject area of "biomedicine" for their compliance with the international standard of research through the use of a set of evaluation criteria with quantitative parameters.

25 cl, 17 dwg, 2 tbl

Description

FIELD OF THE INVENTION

The present invention relates to healthcare, scientometric methods and systems for analyzing data on medical research projects in order to identify a promising project or research direction according to the maximum number of points scored by the evaluated criteria and to build a rating for several evaluated research areas in increasing or decreasing points. The invention is an additional toolkit that provides support in making expert decisions on financing a particular project in the field of medicine. The invention allows the use of Big Data tools for data analysis in the subject field of “medicine”, which account for more than a third of publications and patents of world science. The invention can be implemented using computer and network technologies.

State of the art

The prior art technical solutions that provide an assessment of the prospects of projects (research areas), including in the field of medicine, from the point of view of their compliance with the world level according to certain criteria. For example, in modern practice, scientometric methods based on quantitative estimates of indicators related to the citation of scientific works are widely used to assess the prospects of research projects and the effectiveness of the activities of scientists. The selection of indicators (criteria) and their assessment can be carried out using scientometric resources (databases), for example, the companies ThomsonReuters, Elsevier and Questel and others, providing search, systematization and analysis of arrays of scientific, technical and market information on search query with visualization of research results. These resources allow conducting analytical studies of patent activity and publication activity with a choice of different time intervals, geographical regions, research topics, copyright holders, inventors, etc. However, the search and analytical apparatus of these resources does not allow a comprehensive assessment of medical projects from the point of view of their prospects and relevance to national and world levels of research on the whole set of scientometric indicators necessary when making various management decisions.

In this regard, the urgent problem is the development of an algorithm for the process of conducting the mentioned project appraisal with obtaining an objective result of the appraisal and ensuring its visual representation, as well as the possibility of ranking projects, which is solved in the claimed invention.

The prior art solution is presented in the patent for the invention US 9177249 - "Scientometric Methods for Identifying Emerging Technologies", aimed at identifying emerging technologies and assessing their readiness for use. The method is based on the use of a scientometric model that tracks the emergence of new technology (using scientific and technical literature and patents) and its transition to commercial use through various levels of readiness. Evaluation is carried out by the presence of scientific papers, patents, web news on the subject area of the study, as well as by identifying trends in publications, citation, indicators of collaboration, online news templates. The objects of research are individual network sources of information, from which the time flow of activity (the stage of the life cycle of technology development) is revealed using the example of a subject area. The level of technology readiness for use is assessed on the TRLS (Technology readiness levels) scale, which includes 9 levels - from the first, when the technology goes from fundamental to applied research, to the ninth, when the actual application of the technology is present in the final form and in the conditions of the tasks that are encountered in performance testing and evaluation. The task of determining the level of technology readiness is solved by collecting raw data associated with the emergence of technology from sources and storing raw data over a time interval into its own data set for each source; normalization of raw data in each of its own data sets and the creation of normalized data sets; determining initial events and peaks in normalized data sets; assignment of a technological indicator of the level of readiness based on initial events and peaks in at least one of the normalized data sets. The result for the technology being evaluated is a specific indicator selected from nine known indicators (criteria) TRL1-TRL9, in relation to the specific year for which this indicator is defined. The result of the technology research is represented by the specific TRL value and the year in which this indicator is identified. In a particular embodiment, the algorithm includes a characteristic of the estimated search criterion, as well as a characteristic of the source from which the data is obtained, as well as a quantitative indicator of the threshold value of the information units found for a particular TRL. In particular, according to paragraph 6 of the formula, TRL 1 is assigned to technology in the year when the first publication describing the technology appears in the normalized data set obtained from the source of scientific data. According to paragraph 7 of the claims, TRL 2 is assigned in the year when up to 20 articles are published and in the first year when the publication cited in the publication of another author appears in the normalized data set obtained from the source of scientific data. Thus, it is fundamental and essential in this invention to use for each TRL specific characteristics (data source, search criteria, number of data found (information units)), which conclude that the technology is assigned to a particular TRL level on a specific date. Thus, this well-known invention answers the following questions - at what time was the technology at a certain level of availability (out of 9 known TRL levels), what was the highest level of technology availability at the time of the study? Which technology for potential implementation has the highest TRL? Is a competitive product at a higher TRL level? The solution to this problem is important for business to understand the degree of maturity of a technology for its commercial use.

The claimed invention, although it is associated with the study of scientific projects (technologies) using scientometric (bibliometric) data, has a different purpose - the method and computer-implemented system are designed to assess the conformity of research projects or research directions to the world level and contain a different sequence of actions and a completely different functional set of interconnected blocks.

In particular, the claimed invention implements a different algorithm for generating a search query, processing and interpreting the results for certain parameters proposed by the inventors, which differ from the known solution. In the claimed solution, projects are evaluated and the rating is built from projects that are processed for their compliance with the world level of research, and not their readiness for technological use. Thus, the claimed solution has a different composition of blocks, their different relationship (which characterizes a different sequence of material actions on a material object), and a different set of evaluation parameters.

In particular, in the claimed solution, there is a block characterizing the criteria database K1-K4 for evaluating research projects with specified intervals of values in points, the data processing unit has a completely different constructive solution expressed in its function, namely, the data processing unit is endowed with a processing function search query results obtained from external databases according to relevant criteria, and evaluating the processing result, in which the processing result for each criterion is assigned a certain value in points and a predetermined range of values, the values obtained by summing all the criteria and obtaining values of the total score on all criteria of more than 50% of the maximum possible value of the total points according conclude scientific - research project world level studies. In addition, the function of sorting projects or research areas in ascending or descending order of the mentioned scores, which is also endowed with the processing unit, is not known from a known source. The sequence of actions implemented by the claimed system also differs from the known solution. In particular, in US 9177249, after data collection, there is a stage of their normalization in each data set and the creation of normalized data sets, which is carried out by assigning a value of 100 to the year indicator, when there are the largest number of information units, for example, quotes, and other years are only fractional percentage of the highest year, after which initial events and peaks are determined, according to which this or that technological indicator (TRL) is assigned. In the claimed solution, after data collection, all data sets are not normalized, which implies bringing the obtained values (the number of information units of the data stream) to a single form, which is in US 9177249. In the claimed solution, after receiving the data, they are processed according to each of the criteria K1-K4 , which has its own sequence of actions. An essential feature of the claimed system is that it allows you to measure a specific parameter - a value that characterizes the degree of importance of scientific research in terms of world novelty. At the same time, the improvements made make it possible to increase the accuracy of the degree of significance of scientific research from the point of view of world novelty due to the presented set of blocks with endowed functionality and the connections between them. In addition, the result of the assessment of the conformity of the project to the technological level of readiness (TRL) in US 9177249 is a qualitative parameter, in the claimed solution, the interim result of assessing the degree of significance of the project from the point of view of world novelty is a quantitative parameter, on the basis of which the rating of projects is built. It is not possible to assess the degree of significance of the project from the point of view of world novelty and build a rating using the algorithm described in US 9177249. Even if you can use the individual parameters of some TRL criteria, in order to make such an assessment, you will need more system starts with a large number of requests, while for the analysis of the result, in addition to increased machine resources (RAM and time), an additional expert assessment is required. In our case, the system allows you to get the desired result in one pass of machine time.

Disclosure of invention

The objective of the present invention is to develop a new solution relating to the rapid assessment of medical research national projects or research areas for their compliance with the world level of research, providing, including increasing the reliability and objectivity of expert assessment in deciding whether to provide financial (grant) support to a particular research project.

The technical result of the invention is manifested in the possibility of processing heterogeneous data relating to medical research, which allows you to evaluate the research projects and to rank the results. In addition, the claimed invention allows to increase the system performance when solving a suppressed task (i.e., allows processing to produce a result (product) in less time). In addition, the measurement for each of the evaluated projects of a value characterizing the degree of importance of scientific research from the point of view of world novelty requires fewer requests, therefore, less computer time and resources for processing and obtaining the result, including RAM. In addition, the change in the physical state of the processing unit, associated with the addition of new functionality, provides the opportunity to run the system fewer times to obtain a result, therefore, to save the energy consumed by this system. In addition, a decrease in the number of processed requests (due to the use of a functional that provides sampling from a huge array of external databases and processing according to a specific algorithm using target criteria) also reduces the likelihood of an error affecting obtaining a more accurate and reliable result. Thus, the claimed invention provides an objective and reliable quantitative assessment (in points) of the project's compliance with the world level of research through the use of tools that allow you to take into account a certain set of criteria that are significant for this task and their gradation.

The problem is solved in that the method of assessing the conformity of research projects, in particular, in the subject field “medicine”, to the world level of research using a computer-implemented system includes the formation of a search query for the system, which is a combination of parameters (keywords and / or symbols) characterizing the estimated direction of the study; inputting the generated request into a computer-implemented system, made with the possibility of access via wired or wireless communication channels to external databases of scientific, technical and / or patent and / or market information; search for information on the entered query in the appropriate database to obtain an array of data; processing the data array according to relevant criteria with the subsequent evaluation of the processing result for each criterion with assignment of a value from a given range of values in points and their subsequent summation; and when receiving the value of the total points by all criteria more than 50% of the maximum possible value of the total points make a conclusion about the conformity of the research project to the world level of research; at the same time, as the basic criteria for searching, processing a data array and subsequent evaluation of the processing results, use:

- the dynamics (data on the dynamics) of publication and / or quotation activity, in particular, in the subject field “medicine”, (K1) representing the distribution of the number of identified publications in the direction of the study by year for a period of at least 5 recent years, while processing data find an approximating power function Y = X ^a or construct an approximating curve corresponding to this function, where Y is the value of the approximating function, X is an independent variable (time), and is the sought indicator of the approximating power function, with subsequent analysis of publication and / or citational activity by the value of the indicator a and / or the nature of the approximating curve: if a > 1 and / or the curve is characterized by accelerated growth (the approximating curve increases faster than the linear function), K1 assign the maximum value of the points from the given range of values; if a = 0 and / or the curve is characterized by linear growth, or 0 < a <1 and / or the curve is characterized by slow growth (the curve grows slower than the linear function), or a = 0 and / or the curve is characterized by a lack of growth on average, K1 assign the corresponding intermediate scores, while the value of K1 for the case a = 1 is greater than the value of K1 for the case 0 < a <1, which in turn is greater than the value of K1 for the case a = 0; if a <0 and / or the curve shows a decrease in activity, K1 assign a minimum score;

- the presence of a research direction, in particular, in the subject field “medicine”, among 1% of the most cited publications from the total number of cited publications (K2) for a period of at least 5 recent years, if a research direction is identified among 1% of the most cited publications K2 assign the maximum score; in the absence - K2 assign a minimum score;

- the presence of a research direction, in particular, in the subject field “medicine”, among 0.1% of the most cited publications from the total number of cited publications (K3) for a period of at least 2 recent years, if a research direction is identified in these publications, K3 is assigned maximum score; absence - K3 assign a minimum score;

- the presence of a research direction, in particular, in the subject field “medicine”, in world research fronts (K4), formed by the content of scientific publications that reflect mutual citation for a period of at least 5 recent years, in the event that such a presence is found, K4 is assigned the maximum score; absence - K4 assign a minimum score.

The best result is achieved when using an array of data for processing and subsequent evaluation of the processing result along with the basic criteria of additional criteria selected from the group including:

- the dynamics (data on the dynamics) of patent (inventive) activity, in particular in the subject field “medicine” (K5), which is the distribution of the number of patent documents by the date of their publication in the direction of the study by year for a period of at least 5 recent years, when during data processing, an approximating power function Y = X ^{a is found} or an approximating curve is constructed that corresponds to this function, where Y is the value of the approximating function, X is an independent variable (time), and the desired exponent l approximating power function, followed by analysis of publication and / or citational activity by the value of the indicator a and / or the nature of the approximating curve: if a > 1 and / or the curve is characterized by accelerated growth (the approximating curve increases faster than the linear function), K1 assign the maximum score from a given range of values; if a = 1 and / or the curve is characterized by linear growth, or 0 < a <1 and / or the curve is characterized by slow growth (the curve grows more slowly than the linear function), or a = 0 and / or the curve is characterized by a lack of growth on average, K1 assign the corresponding intermediate scores, while the value of K1 for the case a = 1 is greater than the value of K1 for the case 0 < a <1, which in turn is greater than the value of K1 for the case a = 0; if a <0 and / or the curve shows a decrease in activity, K1 assign a minimum score;

- the ratio of the number of research organizations and manufacturing companies, in particular, in the subject area “medicine” (K6), among the first 10-20 copyright holders selected by the maximum number of patent documents they have in the direction of the study for a period of at least 5 recent years , if 50% or more are rightholders of research organizations, K6 assign the maximum value of points, if rightholders of research organizations are less than 50%, K6 is assigned vayut minimum points;

- comparison of the dynamics of filing patent applications and the dynamics of granting patents, in particular, in the subject field “medicine” (K7) in the direction for the last 5 or more years, while approximating curves characterizing the dynamics of filing patent applications and the dynamics of granting are constructed from the obtained values patents, and evaluate the nature of the curves, if the curves are characterized by an increase (positive dynamics), K7 assign the maximum score; if the dynamics of the filing of patent applications is positive, and the dynamics of the grant of patents is negative or the dynamics of the filing of patents is negative, and the dynamics of the grant of patents is positive, K7 is assigned an intermediate value, if the dynamics of the filing of patent applications and the granting of patents is negative, K7 is assigned a minimum score;

- the dynamics (data on the dynamics) of grant financing in the direction of the study, in particular, in the subject field “medicine” (K8) for a period of at least 5 recent years, in the case of positive dynamics (growth of grant financing), K8 is assigned the maximum score, in the absence of positive dynamics (growth in grant financing), K8 is assigned a minimum score.

In addition, the following criteria can be additionally used to evaluate projects and build a rating:

- the dynamics of acquisitions by large companies of start-ups and medium-sized technology companies, in particular, in the subject field “medicine” (K9) for a period of at least 5 recent years, in the case of positive dynamics (an increase in the number of acquisitions), K9 assign the maximum value to points, in the absence of positive dynamics (increase in the number of acquisitions) K9 assign a minimum score.

- dynamics of licensed sales, in particular, in the subject area “medicine” (K10) for a period of at least 5 recent years, in the case of positive dynamics (growth in the number of acquisitions), K10 assign the maximum score, in the absence of positive dynamics (growth in the number of acquisitions ) K10 assign a minimum score.

When evaluating the result of processing the data array for each criterion, the same interval of values in points can be used using for K1, K5, K7 weighting factors selected from the interval of values from 1.5 to 3, and the project can be evaluated using formula (2); or a different range of values in points, while the intervals of values for K1 are characterized by the largest value of the maximum value (for example, from 0 to 20), for K2, K3, K4, K6, K8, K9, K10 - the smallest value of the maximum value (for example, from 0 to 10), for K5 and K7 - an intermediate (between the two groups of criteria) maximum value (for example, from 0 to 15), and the project can be evaluated using formula (1).

или

,

or

,

where S is the total value of all the Ki criteria for the project, for which i means the number of the criterion, where n = 1 ÷ 10; Ci is the weight coefficient (normalization factor) used in one embodiment of the invention, reflecting the degree of significance of one criterion relative to another, namely, Ki = Ki'Ci, where Ki is selected from an interval of equal magnitude (for example, from 0 to any integer , for example, 10) for any i, a Ci is chosen for K1, K5, K9 = 1.5 ÷ 3. Thus, for each criterion, the value is first determined from the general range of values for all criteria, and then the found value is multiplied by the coefficient corresponding to this criterion.

As a rule, there are information Internet resources that provide search and processing of information arrays according to the above criteria. In particular, according to criterion K1, it is possible to process a request using the international bibliometric database WebofScience over the past 20 years, while the option “CreateCitationReport” is used to visualize the results of processing information on this criterion; by criterion K2 - using the WebofScience database over the past 10 years, while the option “Highly Cited Papers” is used to visualize the results of processing information on this criterion; according to the criterion K3-DB Web of Science for the last 2 years, while the option “Hot Papers” is used to visualize the results of processing information on this criterion; according to criterion K4 - the DB Essential Science Indicators for the last 5 years, while to visualize the results of information processing according to this criterion, use the option ((Research Fronts »; according to criterion K5 - the DB Orbit for the last 20 years, while to visualize the results of processing information on for this criterion use the option ((Distribution of search results by Publication years "; according to criterion K6 - the Orbit database, in order to visualize the results of processing information on this criterion, use the options:" Assignees "/" Assignee "of the first 20 in the number of patent applicants in the direction research over the last 20 years; according to the K7 criterion, the Orbit database for the last 20 years, while to visualize the information processing results for this criterion, a sequence of options is used (((((Legal status »/“ Publication years / Colorby / Legal status ”; according to the K8 criterion - SciVal Funding3a database for the last 20 years, while the option ((search »; by K9 criterion - for example, the StartStartTrack service, by K10 criterion - publicly available databases, for example, the Rospatent database.

The problem is also solved by the fact that a computer-implemented system for assessing the conformity of research projects in the subject field “medicine” to the world level includes a user terminal device configured to connect via wired or wireless communication to external databases of scientific, technical and / or patent and / or market information; wherein the terminal device comprises: a search query input unit, which is a combination of the parameters of keywords and / or symbols) that characterize the evaluated research project (or research direction); a block containing a database of criteria for evaluating research projects (research areas) with specified intervals of values in points; a data processing unit configured to process search query results obtained from external databases of scientific, technical and / or patent and / or market information according to relevant criteria, and evaluate a processing result in which a processing result is assigned a specific value in points from a given range of values, summarize the obtained values according to all criteria, and upon receipt of the value of the sum of points according to all criteria, more than 50% of the maximum possible value of the sum of points They conclude that the research project corresponds to the world level of research; an output unit to the user terminal device of the results of analytical (graphic) processing; at the same time, the above basic (K1-K4) and additional (K5-K8) criteria are used as criteria for evaluating research projects used when searching in external databases, processing search query results and evaluating processing results.

International bibliometric databases based on the ISI Web of Knowledge platform: Web of Science and / or Essential Science Indicators (http://webofknowledge.com) are used as external databases of scientific, technical and / or patent and / or market information; and / or Orbit Questel database (http://www.orbit.com); and / or SciValFunding database (http://www.funding.scival.com).

The problem is also solved by the fact that the method of constructing a rating (ranking) of the list of research projects in the subject field “medicine” according to their degree of compliance with the world level of research includes assessing the conformity of each of the research medical projects included in the list with the world level of research with assignment a certain value of the total score according to the criteria under study, which is carried out according to the method described above, after which the projects are sorted or areas of research in ascending or descending order of the mentioned scores.

The problem is also solved by the fact that a computer-implemented system for ranking (ranking) the list of research projects in the subject field “medicine” according to their degree of compliance with the world level of research, based on the use of the computer-implemented system described above, made with the possibility of evaluation compliance of each of the research projects included in the list or direction of research with the world level of research with assignment to the project or direction of research research certain value sum scores study end points, and further comprising a database or lines of research projects evaluation results, the data processing unit is further projects sorting function or research areas in ascending or descending order of values of said points.

Brief Description of the Drawings

The invention is illustrated by drawings, where in FIG. 1 shows a structural diagram of a computer-implemented system; in FIG. 2 and 3 - presents the algorithms of the proposed method, which can be implemented in software; in FIG. 4-12 presents algorithms for determining the values of the criteria K1-K9, respectively, in FIG. 13 - an algorithm for determining the total (final) value of points for the studied project and its compliance with the world level; in FIG. 14 - algorithm for ranking projects; in FIG. 15 is an example of a graph characterizing the dynamics of publication and / or citation activity in the scientific and technical direction over the past 20 years, obtained using the Web of Science system, to evaluate criterion K1; in FIG. 16 is an example of a graph characterizing the dynamics of patent activity in the scientific and technical direction over the past 20 years, obtained using the Orbit system; in FIG. 17 is an example of a graph characterizing the dynamics of filing patent applications and issuing patents in the scientific and technical field over the past 20 years, obtained using the Orbit system;

The implementation of the invention

The following terminology is used in the present invention:

A medical research project is plans, activities, and other tasks of a scientific nature related to scientific research, conducting research, experiments in order to expand existing and obtain new knowledge, test scientific hypotheses, establish patterns that manifest in nature and in society in the subject area “Medicine”, including “biomedicine”, “clinical medicine”.

Direction of research (scientific direction) - a deep and comprehensive study of the subject in order to elucidate the laws of the phenomena studied, the results of which are supposed to solve practical problems.

Publication activity in the field of research - the total number of publications indexed, for example, in the Web of Sci, in the field of science, written by the world scientific community for a certain time interval.

Cited activity in the direction of research - the total number of links received on publications, indexed, for example, in the Web of Sci, in the scientific direction, written by the world scientific community for a certain time interval.

Citation in the global publication stream is the number of links that articles in a given scientific field received during the billing period of the data received.

Reflection of publications in the direction of research (scientific direction) in the lists of “high” citation (Highly cited papers) of the Essential Science Indicators (ESI) database - 1% of the total number of publications published over a period including the last 10 years and the current interval of the current year, and received stable citations above the world average in a given subject area at a specified time interval, indexed in Web of Science for a specified time interval. The consistently high level of citation of articles from the Highly Cited Papers category, from the point of view of ESI developers, makes it possible to define these publications as a standard of research activity.

Reflection of publications in the direction of research (scientific direction) in the lists of "hot" citation (Hot papers) in the database Essential Science Indicators (ESI) -0.1% of the total number of publications published over the previous two years and the current interval of the current year and received abnormally high citation in this subject area, indexed in Web of Science for the specified time interval. The ultra-high citation level of articles from the Hot papers category, from the point of view of ESI developers, gives grounds to define publications as reflecting the beginning of breakthrough research.

The world level of research (or a research project or research area) is the relevance of the stated topic of a research project to research areas reflected in the lists of highly cited publications ((Highly cited papers, Hot papers) and in the global research fronts.

The prospect of a research project is the correspondence of the research direction to the world level and the goal of creating new technologies.

The research front is a group of highly cited publications, which is determined on the basis of cluster analysis and is combined on a thematic basis using the citation method. The research front includes those articles which themselves received high citation and for which a high level of mutual citation was simultaneously noted. The source of information on world research fronts is the Essential Science Indicators (ESI) database.

Production company - an organization that produces tangible products and provides material services.

Patent document - all types of published applications for inventions and utility models (accepted, laid out, continued, etc.), all types of security documents for inventions and utility models (patents, certificates, etc.).

Patent (or inventive) activity - the total number of published applications for inventions and utility models or security documents in the direction of research in different countries by priority dates (years) (or by dates of filing applications for protection documents) or by dates of publication of applications or patents.

Dynamics of patent (or inventive) activity - reflected in patent documents (or title documents) is a change in inventive activity in the studied field of technology over a certain period of time.

Grant financing - free provision of financing for research projects.

Accelerated growth - an assessment of the obtained data on publication and patent activity from the approximating curve of the exponential function Y = X ^a at a> 1, when the approximating curve grows faster than the linear function.

Linear growth is an assessment of the obtained data on publication and patent activity by the approximating curve of the exponential function Y = X ^a with a = 1.

Slowed down (or unstable) growth or stagnation - an assessment of the obtained data on publication and patent activity from the approximating curve of the exponential function Y = X ^a at a value of 0 <a <1, when the approximating curve grows more slowly than a linear function.

The fall in activity is the assessment of the obtained data on publication and patent activity by the approximating curve of the exponential function Y = X ^a with a value of <0.

A startup is a short-term organization whose main task is to develop and bring to market a new product or service in conditions of high uncertainty. In fact, it is a company created to search for a reproducible and scalable business model.

A medium-sized technology company is a rapidly growing organization that develops and manufactures sophisticated mechanisms, electronics, medicines, or program code, and this activity brings it at least half the revenue. Each year, in such companies, total revenue is kept in the range of 0.1-10 million rubles. with a growth rate of more than 15% with a share of average R&D costs of more than 5% and average costs of technological innovations of more than 19%.

The takeover of startups and medium-sized technology companies by large companies is a transaction made to establish control over a startup or medium-sized technology company that created a successful business model while promoting a new product or service on the market. Absorption is carried out by acquiring more than 30% of the authorized capital (shares, shares, etc.) of the absorbed company while maintaining its legal independence.

Licensed sales are transactions on the production and sale of products and services produced and sold on the basis of a previously concluded license agreement, according to which “one party - the holder of the exclusive right to the result of intellectual activity (licensor) - agrees to provide the other party (licensee) the right to use such result to the extent provided by the license agreement.

Assessment of scientific, technical and technological projects is carried out according to four basic criteria K1-K4 and four additional K5-K10, each of which is assigned the maximum, minimum and intermediate values in points, the sum of which makes a conclusion for each project about its compliance with the world level.

The following criteria are used as basic criteria:

K1 - the dynamics (or data on the dynamics) of publication and citation activity for the project (research direction);

K2 - the presence of a direction of research among 1% of the most cited publications from the total number of cited publications (or the reflection of publications in the direction of research in Highly cited papers);

K3 - the presence of a direction of research among 0.1% of the most cited publications out of the total number of cited publications (or the reflection of publications in the direction of research in the hot citation lists (Hot papers));

K4 - the presence of the direction of research in world research fronts, formed by the content of scientific publications that reflect mutual citation.

As additional criteria, the following are used:

K5 - the dynamics (data on the dynamics) of patent (or inventive) activity in the direction of research;

K6 - the ratio of the number of research organizations and manufacturing companies among the first 10-20 copyright holders selected by the maximum number of patent documents they have in the direction of research;

K7 - comparison of the dynamics of filing patent applications and the grant of patents;

K8 - dynamics (dynamics data) of grant financing in the direction of the study;

K9 - dynamics (dynamics data) of the number of acquisitions by large startup companies and medium-sized technology companies in the direction of the study;

K10 - dynamics (dynamics data) of licensed sales.

Below is a more detailed description of each criterion and an example of its use for generating a search query into external databases, processing results from external databases and quantitative interpretation in relation to the project being evaluated.

K1 - the dynamics (or dynamics data) of publication and / or citation activity in the subject field “medicine”, is the distribution of the number of identified publications in the direction of the study by year for a certain period of at least 5 years, preferably at least 10 years. Most preferred is a study period of 20 years. Moreover, the most preferable is the presentation of the number of identified publications by year in graphical form. As a tool for determining this indicator, for example, the international bibliometric database Web of Science can be used, in which the “Create Citation Report” option is provided for visualizing the dynamics of publication activity (see Fig. 15). After receiving the appropriate schedule, they are evaluated.

As you know, with any type of information retrieval, information loss is inevitable and is random in nature, so any thematic array is a system that obeys the laws of distribution of random variables. To identify the most likely direction of development (the general tendency of the documents found to be distributed over the years), the obtained data are approximated and further interpreted depending on what kind of exponential function, for example, Y = X ^a , the data are closest to. The results of search queries always give digital values greater than unity, and therefore the value of X is always positive, and the degree of “ a ” can take five values ( a > 1, a = 1, 0 < a <1, a = 0, a <0 ), depending on which the function Y = X ^a has the corresponding form: for a > 1 - it is characterized by accelerated growth, for a = 1 - linear growth, for 0 < a <1 the curve is characterized by slow growth (the curve grows more slowly, than a linear function), with a = 0, the curve takes the form of a horizontal line (or is characterized by I have a lack of growth on average), while the value of a <0 - the curve shows a drop in activity.

Thus, depending on the value of indicator a or the nature of the graph (approximating curve), criterion K1 is assigned a certain score: in the presence of "accelerated growth" K1, a maximum score is assigned, for example, 20 points; in the presence of "linear growth" or "slow / unstable growth" / "stagnation" or "lack of growth on average", K1 assign intermediate points, for example, 15, 10 and 5 points, respectively; in the presence of a "drop in activity", K1 assign a minimum score, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 20 points. Evaluation of projects by this criterion can be carried out at least by three values - maximum, minimum, and at least one intermediate.

It should be noted that when processing the graphic results of a search query, in addition to a power dependence (a power approximating function), a linear dependence can be used, the equation of which is of the form: y _i = a + bt _i , where y _i is the value of the points of the approximating lines; a is the zero coefficient, i.e. the ordinate of the point of intersection of the approximating line with the coordinate axis; b is the first coefficient, i.e. straight line angular coefficient; t _i is an independent variable (in this case, time).

K2 - the presence of a direction of research in the subject field “medicine” among 1% of the most cited publications from the total number of cited publications over the past 10 years, or the presence of a direction of research in the lists of “high citation” (Highly Cited Papers) that can be formed , for example, using the Web of Science system. Lists of "high citation" are formed from 1% of the most cited publications from the total number of all cited articles indexed in Web of Science, published over the past 10 years and the current period, and received citation in the same time interval. If publications in the direction are found in the lists of “Highly Cited Papers” K2 assign the maximum value of points, for example, 10 points; if publications in the direction are not in the Highly Cited Papers lists, K2 assign a minimum score, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 10 points.

K3 - the presence of a direction of research in the subject field “medicine” among 0.1% of the most cited publications from the total number of cited publications for a period of at least 2 years, or the presence of a direction of research in the lists of “hot citation” (Hot Papers), which also can be formed, for example, using the Web of Science system. Hot citation lists are compiled from 0.1% of the most cited publications indexed in Web of Science over the past 2 years, which have received an abnormally high citation over the past two months. Thus, the “Hot Papers” includes publications from those areas of knowledge that are characterized by rapid quoting. If publications in the field of research are found in the Hot Papers lists, K3 assigns a maximum score, for example, 10 points; if they are not detected, K3 assigns a minimum score, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 10 points.

K4 - the presence of a direction of research in the subject field “medicine” in world research fronts, formed by the content of scientific publications that reflect mutual citation for a period of at least 5 recent years (preferably, for the last 10 years and the period of the current year) in the direction of research. Research fronts are formed in the corresponding databases, for example, Essential Science Indicators (ESI). This database allows you to identify the main trends in the development of scientific fields and research fronts. The main criterion for the significance and productivity (or quality and efficiency) of scientific research, from the point of view of the ideology and methodology of ESI developers, is citation, that is, the number of links that articles in this scientific field received during the calculation period of the data selection. The database is aggregated from peer-reviewed scientific journals indexed in the Web of Science system and covers the archives of publications over the past 10 years and information for the period of the current year.

In the event that during the study period the direction of the research “falls” into the research fronts formed by the database, K4 assigns a maximum score, for example, 10 points; if not included, i.e. the appearance of new research fronts has stopped; K4 is assigned a minimum score, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 10 points.

K5 - the dynamics (data on the dynamics) of patent (or inventive) activity in the subject field “medicine”, which is the distribution of the number of identified publications of patent documents in the direction of the study over the years for a certain period, for example, the last 20 years. The dynamics of patent activity is understood as reflected in patent (for example, protective) documents the change in the activity of inventive activity in the studied field of technology for a certain period of time. To determine the dynamics of patent activity (or the dynamics of patenting), an array of patent documents is systematized according to the priority dates or publication of patent documents (as a rule, excluding equivalent patents). Evaluation of the dynamics of patent activity can be carried out using various patent databases. Moreover, it is most preferable to present the number of published publications by year in the form of a graph that can be generated using the appropriate options, for example, in the Orbit database (using the ((Distribution of search results by Publication years ») option or the Lexsis-Nexsis database). After that, the graphical presentation is evaluated according to the specific direction of the research, similar to the criterion K1. If the graph is characterized by "accelerated growth", K5 is assigned the maximum score, for example, 15 points; "linear growth" or "unstable and growth ", K5 assign intermediate points, for example, 10 and 5 points, respectively;" falling activity "- K5 assign a minimum value for points, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 15 points.

K6 - the ratio of the number of research organizations and manufacturing companies in the subject area of “medicine” among the first 10-20 copyright holders selected by the maximum number of patent documents they have in the research direction for a period of at least 5 recent years (preferably over the past 10 years , most preferably over the past 20 years). Processing a request by criterion K6 can be carried out using the Orbit international bibliometric database, and to visualize the results of processing information on this criterion, the following options are used: last 20 years.If 50% or more are copyright holders of research organizations (including universities and research centers), K6 assign a maximum score, for example, 10 points, if right holders of research organizations make up less than 50%, K6 assign a minimum value for points, for example, 0 points. Thus, the maximum score a project can receive by this criterion is 10 points.

K7 - comparison of the dynamics of filing patent applications and the dynamics of granting patents in the subject field “medicine” in the direction for the last 5 or more years (preferably for the last 10 years, most preferably for the last 20 years). Request processing by criterion K7 is carried out using the Orbit international bibliometric database, and the options “Legal status” / “Publication years / Colorby / Legal status” are used to visualize the results of processing information on this criterion. Using the obtained values, approximating curves are constructed (similarly to processing data by criterion K1), characterizing the dynamics of filing patent applications and the dynamics of granting patents, and evaluating the nature of the curves, if the curves are characterized by positive dynamics, K7 assign a maximum score, for example, 15 points; if the dynamics of filing patent applications is positive, and the dynamics of filing patents is negative or the dynamics of filing patents is negative, and K7 is assigned an intermediate value, for example, 10 and 5 points, if the dynamics of filing patent applications and issuing patents are negative, K7 is assigned the minimum value of points, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 15 points.

K8 - the dynamics (data on the dynamics) of grant financing in the subject field “medicine” in the direction of research for a period of at least 5 last years (preferably 10 years, most preferably over 20 years), which reflects the availability of funding in this direction of research on a competitive basis for certain period. Request processing by criterion K8 is carried out using the SciVal Funding international bibliometric database, and the option: “search” is used to visualize the information processing results for this criterion. If the diagram shows a positive trend in obtaining research support, K8 assign a maximum score, for example, 10 points, the lack of dynamics of research support, K8 assign a minimum score, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 10 points.

K9 - the dynamics (data on the dynamics) of acquisitions by large start-up companies and medium-sized technology companies in the subject field “medicine” in the direction of the study over a period of at least 5 recent years (preferably 10 years, most preferably over 20 years), which reflects the number of acquisitions per this area of research on a competitive basis for a certain period. Request processing by criterion K9 is carried out using, for example, the StartTrack service, and for this, the option “search” is used to visualize the results of processing information on this criterion. If the positive dynamics of obtaining support for research is noted in the diagram, K9 assign the maximum score, for example, 10 points, the lack of dynamics of support for research, K9 assign the minimum score, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 10 points.

K10 - the dynamics of licensed sales in the subject field "medicine" in the direction of the study for a period of at least 5 last years (preferably 10 years, most preferably 20 years), which reflects the presence and volume of licensed sales. Request processing by criterion K10 is carried out using any public service, for example, the Rospatent database. If the diagram shows a positive trend in obtaining research support, K10 assign a maximum score, for example, 10 points, the lack of dynamics of research support, K10 assign a minimum score, for example, 0 points. Thus, the maximum score that a project can receive by this criterion is 10 points.

When assigning a maximum score value to each criterion, weighted values of the criteria with respect to each other can be used. The criterion K1 is characterized by the maximum weight (for example, the maximum score for this criterion corresponds to 20 points), the criteria K2, K3, K4, K6, K8, K9, K10 are characterized by the minimum weight (for example, the maximum score according to this criterion corresponds to 10 points) and intermediate weight K5, K7 (for example, the maximum score for this criterion corresponds to 15 points). A different order of assigning the maximum value in points to each criterion is also possible. In one of the embodiments, points are presented, which are presented in Table 1. The ratio of the number of points is maintained, the absolute value of the points is coordinated with the project appraisal system used.

According to the results of the project assessment according to all criteria, a quantitative assessment of the project in points is obtained, which is the sum of the values according to all criteria.

When assessing several medical projects using this methodology, it is possible to build a rating (or ranking) according to the degree of conformity of projects (research areas) to the world level of research in increasing or decreasing scores.

This method can be implemented using a hardware-software complex or computer system, which includes at least a processing unit (analytical) data, a database (memory block), input and output blocks (Fig. 1). Algorithms implemented using the inventive system are presented in FIG. 2-14.

The input unit and the output unit interact with the computer-implemented system with the operator, and through wired and / or wireless communication lines with external databases. Through the input unit, the input of the generated search query is carried out. The output unit carries out data output in a format convenient for the operator to perceive (for example, in graphic, tabular and / or text format), as well as other operations for outputting data from a computer-implemented system.

The database of a computer-implemented system contains information on the characteristics and parameters of the assessment criteria K1-K10, information about search queries, information obtained as a result of processing search queries by external databases, etc. The data processing unit interacts with the database of the computer-implemented system, external databases, input and output blocks, performs software processing of information from the database and stores the results of processing (analysis, evaluation) in the database. A processor that implements the claimed algorithm, i.e. implements the described innovations in the form of executable commands, coordinates the actions of the components of the computer system.

The computer-implemented system interacts via wired and / or wireless communication lines with external databases to which it sends search queries, and, after receiving the results of query processing, places it in its database for further software processing.

As a computer-implemented system, a personal computer with the Windows 7 operating system can be used, containing a central processor, random access memory, a hard disk drive (or a memory block, or data storage, or a database), a keyboard, monitor, and communication devices to provide work with Internet resources (network card and / or modem), as well as an output device (data output / processed data display unit, for example, a monitor, or a monitor in conjunction with a printer). It is also known that the processor is an electronic unit or an integrated circuit (microprocessor) that executes machine instructions (program code) - the main part of the hardware. Depending on the machine instructions entered into the processor, the corresponding task is solved. In the claimed invention, machine instructions are set forth in the form of functions prescribed for execution by a data processing unit (processor), namely, providing a search for a specific data array according to a set of criteria, processing and data analysis, allowing to solve the problem of determining the compliance of a particular research project with world level research. Moreover, the processor is capable of processing information (data) obtained by a search query from external databases, according to the claimed method (algorithm) using the evaluation criteria K1-K10. The data processing unit is usually installed in a system unit that executes computer-executable instructions. The inventive system in one embodiment can be built on the von Neumann architecture - the well-known principle of the joint storage of commands (instructions) and data in the computer's memory.

As an example of using external databases in the claimed invention, international bibliometric databases Web of Science, and / or Essential Science Indicators database, and / or Orbit database, and / or SciVal Funding database, as well as any other open and commercial information resources can be proposed .

The algorithm of the computer-implemented system is shown in FIG. 2-14. The operator of the project being evaluated (the direction of scientific research) generates a search query, for example, from keywords in accordance with the rules that are used for the selected external database. At the same time, when forming a search query, keywords and phrases should be used, their synonyms having the greatest weight, i.e. reflecting the essence of the project.

The generated request is sent to an external database that has an array of information for evaluating the project according to the relevant criterion. As a rule, external databases are equipped with tools (analytical, software-implemented) that allow the necessary sorting (processing) of the information array by the search query for specific search fields to obtain the processing result in a graphical or tabular representation. At least, the tools of the databases listed above allow you to process an array of information on a search query, taking into account (or in accordance with) the criteria K1-K10, while the result of processing the request in the form of text, and / or graphic, and / or formatted in which or any other form of information that can be downloaded or stored in the appropriate database (on a hard disk or flash drive or in cloud storage) in a computer-implemented system, where it is processed in a data processing unit and placed in the system database. At the next stage of the method’s work, all information on the project being evaluated is processed, which is available in the system database, as a result, each project is assigned a certain value in points for each of the evaluation criteria (in one embodiment of the invention, taking into account normalization coefficients), after which all points are summed up according to all evaluation criteria, and a final assessment of the total number of points obtained by all criteria is made. If this value of points is 50% or more of the maximum possible number of points, they conclude that the project being evaluated corresponds to the world level, if this value is less than 50%, they conclude that the project under study does not correspond to the world level.

In addition, a computer-implemented system may contain an additional function that ensures the construction of a rating of several evaluated projects according to the number of points scored.

Examples of specific embodiments of the invention.

The inventive method was tested in assessing compliance with the world level of 75 projects (topics) submitted for financial support. Each topic was evaluated according to the proposed basic and additional criteria (see Table 1). The resulting set of quantitative assessments for each project made it possible to build a rating from the presented projects according to their international level of research, and, accordingly, investment attractiveness. The maximum score for each criterion was chosen so that the total maximum number of points that one project can score was 100 points, i.e. exactly as much as an expert can set a project.

It is assumed that if the number of points set by an expert is comparable to the number of points set by the claimed system, the assessment of the project can be considered objective. In the event of a significant (at times) discrepancy in the number of points set by the expert and the claimed system, it is recommended that the project be sent for re-examination with the obtained evaluation results using the proposed system.

Below is an example of an assessment on one of the research projects on the topic “Obtaining antibacterial substances using the peptide inventory method”, which received a total rating on the scale presented in Table 1 of 55 points (out of 100 possible), based on which a conclusion was made on compliance this project world-class research. The dynamics of the number of acquisitions by large startup companies and medium-sized companies in the area of research, as well as the dynamics of licensed sales were not detected, therefore, the criteria K1-K8 are used in the example. The results of the assessment according to the criteria K1-K8 are presented in Table 2.

The aim of the project being evaluated was to develop a set of standard operating procedures (SOPs) for the detection of bacteriocins based on high-performance identification of the composition of the peptide. The search query for external databases was formed from the following keywords that reflect the essence of the project: bacteriocins, peptide, composition, identification, inventory, standard operating procedures (SOP) (for Russian-language databases) / bacteriocin, peptidecomposition, identification, inventory, SOP ( StandardOperatingProcedure) (for English-language databases).

Search in the Orbit database was carried out in three fields: the name of patent documents, abstracts of descriptions of patent documents and claims

The advantages of the claimed invention

The invention is an analytical tool (algorithm) for supporting an expert solution, aimed at improving the effectiveness of financial investments in medical research projects, differentiating costs, and can find answers to questions which areas of basic science and applied science should be supported more and which less volume. The resulting set of quantitative estimates for projects ensures their ranking according to the degree of compliance with the world level of research, investment attractiveness and priority in making managerial decisions.

The claimed invention was tested on medical projects, including related to clinical medicine, biomedicine, and other related areas. Moreover, the invention can also be used in the management of research and technological projects and in other areas of science and technology, will contribute to the innovative development of science and technology, ensure the competitiveness of products and services, increase the efficiency of spending budget funds allocated for the development of national scientific technological sphere.

Claims (45)

1. A computer-implemented system for assessing the conformity of medical research projects or research areas to the world level, including a user terminal device configured to connect via wired or wireless communication to external databases of scientific, technical, and / or patent, and / or market information; wherein the terminal device contains: - input block search query, which is a combination of parameters characterizing the estimated research project, - a memory block including a database of criteria for evaluating research projects in the subject field “medicine” with specified intervals of values in points, search queries, results of processing search queries by external databases, results of project evaluations, while the following dynamics were used as criteria: publication dynamics and / or citation activity (K1), the presence of a research direction among 1% of the most cited publications out of the total number of cited publications (K2), the presence of a research direction among 0.1% of the most cited my publications from the total number of cited publications (K3), the presence of research directions in world research fronts (K4); - block output to the terminal device of the user the results of analytical processing, - data processing unit, configured to: processing the results of a search query obtained from external databases of scientific and technical, and / or patent, and / or market information according to relevant criteria, evaluation of the processing result, in which the processing result for each criterion is assigned a certain value in points from a given range of values, the obtained values are summarized according to all criteria, and upon receipt of the value of the total points for all criteria more than 50% of the maximum possible value of the total points make a conclusion about scientific compliance - research project world-class research, the data processing unit is configured to process the results of the search query according to criterion K1 by determining the distribution of the number of identified publications in the direction of the study by year for a period of at least 5 recent years, while approximating a power function Y = X ^a is found in the data processing or constructing the approximating curve corresponding to the given function, where Y is the value of the approximating function, X is the independent variable (time), and a is the sought indicator of the approximating step function, followed by analysis of publication and / or quotation activity by the value of a and / or the nature of the approximating curve: if a > 1 and / or the curve is characterized by accelerated growth, K1 assign the maximum score from a given range of values; if a = 1 and / or the curve is characterized by linear growth, or 0 < a <1 and / or the curve is characterized by slow growth, or a = 0 and / or the curve is characterized by a lack of growth on average, K1 assign the corresponding intermediate scores, while the value K1 for the case a = 1 is greater than the value of K1 for the case 0 < a <1, which in turn is greater than the value of K1 for the case a = 0; if a <0 and / or the curve characterizes a drop in activity, K1 assign a minimum score; the data processing unit is also capable of processing the results of a search query according to criterion K2 by determining whether there is a research direction among 1% of the most cited publications from the total number of cited publications for a period of at least 5 recent years, if a research direction is identified among 1% of the most cited K2 publications assign a maximum score; in the absence of K2, a minimum score is assigned; the data processing unit is also capable of processing the results of a search query according to the K3 criterion by determining whether there is a research direction among 0.1% of the most cited publications from the total number of cited publications for a period of at least 2 recent years, if the research direction is identified in these K3 publications assign the maximum score; absence - K3 assign a minimum score; the data processing unit is also capable of processing the results of a search query according to the K4 criterion by determining the availability of the research direction in the world research fronts, formed by the content of scientific publications reflecting mutual citation for a period of at least 5 recent years, in case of the presence of such a presence, K4 is assigned the maximum score ; absence - K4 assign a minimum score. 2. The computer-implemented system according to claim 1, characterized in that the data processing unit is further configured to process the results of a search query to determine the dynamics of patent or inventive activity (K5), which is the distribution of the number of patent documents by the date of their publication in the direction of research over years for a period of at least 5 recent years, while processing the data, find an approximating power function Y = X ^a or construct an approximating curve that corresponding to this function, where Y is the value of the approximating function, X is an independent variable, a is the desired indicator of the approximating power function, with subsequent analysis of the publication and / or quotation activity by the value of the parameter a and / or the nature of the approximating curve: if a > 1 and / or the curve is characterized by accelerated growth, K5 assign the maximum value of points from a given range of values; if a = 1 and / or the curve is characterized by linear growth, or 0 < a <1 and / or the curve is characterized by slow growth, or a = 0 and / or the curve is characterized by a lack of growth on average, K5 assign the corresponding intermediate scores, while the value K5 for the case a = 1 is greater than the value of K5 for the case 0 < a <1, which in turn is greater than the value of K5 for the case a = 0; if a <0 and / or the curve shows a decrease in activity, K5 assign a minimum score; 3. The computer-implemented system according to claim 1, characterized in that the data processing unit is further configured to process the results of a search query to determine the ratio of the number of research organizations and manufacturing companies (K6) among the first 10-20 copyright holders selected by the maximum the number of patent documents they have in the field of research for a period of at least 5 recent years, if 50% or more are copyright holders of research organizations th, K6 is assigned the maximum points if rightsholders research organizations make up less than 50%, K6 is assigned to the minimum score. 4. The computer-implemented system according to claim 1, characterized in that the data processing unit is additionally configured to process the results of a search query to compare the dynamics of filing patent applications and the dynamics of granting patents (K7) in the direction for the last 5 or more years, while using the obtained values, approximating curves are constructed that characterize the dynamics of filing patent applications and the dynamics of granting patents, and the nature of the curves is evaluated if the curves are characterized by positive dynamics th, K7 assign the maximum score; if the dynamics of filing patent applications is positive, and the dynamics of filing patents is negative or the dynamics of filing patents is negative, and the dynamics of granting patents is positive, K7 assign an intermediate value, if the dynamics of filing patent applications and granting patents are negative, K7 assign a minimum score. 5. The computer-implemented system according to claim 1, characterized in that the data processing unit is additionally configured to process the results of a search query to determine the dynamics of grant financing in the research direction (K8) for a period of at least 5 recent years, if there is a positive dynamics K8 assign the maximum score, in the absence of positive dynamics K8 assign the minimum score. 6. The computer-implemented system according to claim 1, characterized in that international bibliometric databases based on ISI Web of Knowledge: Web of Science and / or are used as external databases of scientific, technical, and / or patent and / or market information. Essential Science Indicators; and / or Orbit Questel database and / or SciValFunding database. 7. Computer-implemented system according to any one of paragraphs. 1-5, characterized in that the data processing unit when evaluating the result of processing the data array is additionally configured to use weights for K1, K5 and K7 selected from the range of values from 1.5 to 3, while the same criteria are used for all criteria interval of values in points. 8. Computer-implemented system according to any one of paragraphs. 1-5, characterized in that the intervals of values in the database of assessment criteria for K1 are characterized by the largest value of the maximum value, for K2, K3, K4, K6 and K8 - the smallest value of the maximum value, for K5 and K7 - the intermediate value of the maximum value. 9. A method for assessing the conformity of medical research projects or research areas to the world level of research using a computer-implemented system according to claim 1, including: - the formation of a search query for the system, which is a combination of parameters characterizing the estimated direction of research, - entering the generated request into a computer-implemented system, made with the possibility of access via wired or wireless communication channels to external databases of scientific, technical and / or patent and / or market information, - search for information on an entered query in the appropriate databases with an array of data using criteria from the subject field “medicine”: dynamics of publication and / or citation activity (K1), the presence of a research direction among 1% of the most cited publications from the total number of cited publications ( K2), the presence of a research direction among 0.1% of the most cited publications out of the total number of cited publications (K3), the presence of a research direction in world research fronts (K4); - processing the data array according to relevant criteria with the subsequent evaluation of the processing result for each criterion with assignment of a value from a given interval of values in points and their subsequent summation, - and when receiving the value of the sum of points according to all criteria more than 50% of the maximum possible value of the sum of points, they conclude that the research project corresponds to the world level of research, at the same time, during the search, processing of the data array and subsequent evaluation of the processing results according to criterion K1, which characterizes the dynamics of publication and / or citational activity, determine the distribution of the number of identified publications in the direction of the study by year for a period of at least 5 recent years, while processing the data, find an approximating power function Y = X ^a or construct an approximating curve, corresponding to a given function, where Y - the value of the approximating function, X - independent variable, a - figure approximating the desired power function, followed by analysis of publications hydrochloric and / or activity quotational a largest exponent and / or the nature of the approximating curve: if a> 1 and / or curve is characterized by rapid growth, K1 is assigned the maximum score of a predetermined range of values; if a = 1 and / or the curve is characterized by linear growth, or 0 < a <1 and / or the curve is characterized by slow growth, or a = 0 and / or the curve is characterized by a lack of growth on average, K1 assign the corresponding intermediate scores, while the value K1 for the case a = 1 is greater than the value of K1 for the case 0 < a <1, which in turn is greater than the value of K1 for the case a = 0; if a <0 and / or the curve characterizes a drop in activity, K1 assign a minimum score; according to the K2 criterion, the presence of a research direction is determined among 1% of the most cited publications from the total number of cited publications for a period of at least 5 recent years, if the research direction is identified, 1% of the most cited publications of K2 is assigned the maximum score; in the absence of K2, a minimum score is assigned; according to the K3 criterion, the presence of a research direction is determined among 0.1% of the most cited publications from the total number of cited publications for a period of at least 2 recent years, if the research direction is identified in the indicated K3 publications, the maximum score is assigned; absence - K3 assign a minimum score; according to the K4 criterion, the presence of a research direction in world research fronts formed by the content of scientific publications reflecting mutual citation for a period of at least 5 recent years is determined; absence - K4 assign a minimum score. 10. The method according to p. 9, characterized in that as a criterion for processing the data array and subsequent evaluation of the processing result, the dynamics of patent or inventive activity (K5) is additionally used, which is the distribution of the number of patent documents by the date of publication, in the direction of the study, over years for a period of at least 5 last years, while processing the data, find an approximating power function Y = X ^a or construct an approximating curve corresponding to this function ii, where Y is the value of the approximating function, X is an independent variable, a is the desired indicator of the approximating power function, with subsequent analysis of the publication and / or quotation activity by the value of the indicator a and / or the nature of the approximating curve: if a > 1 and / or the curve characterized by accelerated growth, K5 assign the maximum value of points from a given range of values; if a = 1 and / or the curve is characterized by linear growth, or 0 < a <1 and / or the curve is characterized by slow growth, or a = 0 and / or the curve is characterized by a lack of growth on average, K5 assign the corresponding intermediate scores, while the value K5 for the case a = 1 is greater than the value of K5 for the case 0 < a <1, which in turn is greater than the value of K5 for the case a = 0; if a <0 and / or the curve characterizes a drop in activity, K5 assign a minimum score; 11. The method according to p. 9, characterized in that, as a criterion for processing the data array and subsequent evaluation of the processing result, the ratio of the numbers of research organizations and manufacturing companies (K6) is additionally used among the first 10-20 copyright holders selected by the maximum number the patent documents they have in the research area for a period of at least 5 years, if 50% or more are copyright holders of research organizations, K6 assigns max cial value points if rightsholders research organizations make up less than 50%, K6 is assigned to the minimum score. 12. The method according to p. 9, characterized in that as a criterion for processing the data array and subsequent evaluation of the processing result, a comparison of the dynamics of filing patent applications and the dynamics of granting patents (K7) in the direction for the last 5 or more years is additionally used, the obtained values are used to construct approximating curves characterizing the dynamics of filing patent applications and the dynamics of granting patents, and evaluate the nature of the curves, if the curves are characterized by positive dynamics, K7 is assigned r maximum points; if the dynamics of filing patent applications is positive, and the dynamics of filing patents is negative or the dynamics of filing patents is negative, and the dynamics of granting patents is positive, K7 assign an intermediate value, if the dynamics of filing patent applications and granting patents are negative, K7 assign a minimum score. 13. The method according to p. 9, characterized in that, as a criterion for processing the data array and subsequent evaluation of the processing result, the dynamics of grant financing in the research direction (K8) for at least the last 5 years is additionally used, if positive K8 is assigned the maximum value of points, in the absence of positive dynamics K8 assign the minimum value of points. 14. The method according to p. 9, characterized in that the processing of the request by criterion K1 is carried out using the international bibliometric database Web of Science for the last 20 years, and the option “Create Citation Report” is used to visualize the results of processing information on this criterion. 15. The method according to p. 9, characterized in that the processing of the request by criterion K2 is carried out using the international bibliometric database Web of Science for the last 10 years, while the option "Highly Cited Papers" is used to visualize the results of processing information on this criterion. 16. The method according to p. 9, characterized in that the processing of the request by criterion K3 is carried out using the international bibliometric database Web of Science for the last 2 years, while the option "Hot Papers" is used to visualize the results of processing information according to this criterion. 17. The method according to p. 9, characterized in that the processing of the request by criterion K4 is carried out using the international bibliometric database Essential Science Indicators for the last 5 years, and the option “Research Fronts” is used to visualize the results of processing information on this criterion. 18. The method according to p. 10, characterized in that the request is processed according to the K5 criterion using the Orbit international bibliometric database for the last 20 years, and the option "Distribution of search results by Publication years" is used to visualize the information processing results for this criterion . 19. The method according to claim 11, characterized in that the request is processed according to criterion K6 using the Orbit international bibliometric database, and the options: “Assignees” / “Assignee” of the first 20 by the number of patents are used to visualize the results of processing information according to this criterion applicants for research over the past 20 years. 20. The method according to p. 12, characterized in that the processing of the request by criterion K7 is carried out using the Orbit international bibliometric database for the last 20 years, while the sequence of options “Legal status” / “Publication” is used to visualize the results of processing information on this criterion years / Colorby / Legal status. " 21. The method according to p. 13, characterized in that the processing of the request by criterion K8 is carried out using the SciVal Funding international bibliometric database for the last 20 years, and the option “search” is used to visualize the results of processing information on this criterion. 22. The method according to any one of paragraphs. 9-13, characterized in that when evaluating the result of processing the data array for each criterion, the same interval of values in points is used using weight coefficients for K1, K5, K7 selected from the interval of values from 1.5 to 3. 23. The method according to any one of paragraphs. 9-13, characterized in that the ranges of values for K1 are characterized by the largest value of the maximum value, for K2, K3, K4, K6 and K8 - the smallest value of the maximum value, for K5 and K7 - the intermediate, between the two groups of criteria, the value of the maximum value . 24. A computer-implemented system for rating a list of medical research projects or research areas according to the degree of their compliance with the world level of research, based on the use of a computer-implemented system according to claim 1, made with the possibility of assessing the conformity of each of the listed scientific a research project or research direction to a world-class level of research with assigning to a project or research direction a certain value of the total score given criteria and additionally containing a database of the results of the evaluation of projects or research areas, while the data processing unit is designed with the additional function of sorting projects or research areas in ascending or descending order of the mentioned scores. 25. The method of rating the list of medical research projects or areas of research according to the degree of their compliance with the world level of research using a computer-implemented system according to p. 24, including assessing the conformity of each of the listed research projects or research areas with the world level of research with the assignment to the project or direction of the study of a certain value of the sum of points according to the criteria under study, which is carried out in accordance with paragraph 9, last which sorts the projects or lines of research in ascending or descending order of said score values.

Images