RU2844072C2 - System and method for forming training data set for training machine learning algorithm - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to methods of generating recommendations of digital items for users of an online recommendation platform. Method includes generating synthesized user responses for new digital elements without an actual user response; formation includes: at the first stage – training the server of the evaluating machine learning model to determine the predicted indication of the user response for the digital element; and at a second step obtaining an indication of a novel digital element of a supplier of new elements, applying a machine learning evaluation model with respect to the new digital element to determine the indication of its synthesized user response and forming, based on the new digital elements with assigned indications on the synthesized user responses, a training data set for training the MLA algorithm.

EFFECT: high efficiency of the recommendation platform by solving the cold start problem.

20 cl, 10 dwg

Description

Field of technology to which the invention relates

[001] The present technology generally relates to recommender systems and, in particular, to a method for generating a training data set for training a machine learning algorithm to provide recommendations of elements to users of the recommender system.

State of the art

[002] Online recommendation platforms host various digital elements for provision to users. Some online recommendation platforms contain streaming services, such as audio or video streaming services such as the Spotify™ audio streaming service, the Yandex.Music™ audio streaming service, the Netflix™ video streaming service, the Amazon™ Prime Video streaming service, etc., which host various audio tracks and video clips. Other online recommendation platforms may be implemented as online listing platforms, such as the Yandex.Market™ online listing platform, the Amazon™ online listing platform, the Ozon.ru™ online listing platform, etc., through which their users can purchase various goods and services online. Thus, various element providers such as singers, film producers or online stores upload millions or even tens of millions of digital elements to the online recommendation platform to present them to users.

[003] In order to facilitate user navigation among such a number of digital items hosted on the online recommendation platform and to increase the user's satisfaction with the platform from interaction with it, the online recommendation platform may provide the user with user recommendations. For example, a user may rate a digital item on the online recommendation platform, such as a track on a streaming audio service, and the online recommendation platform may provide a recommended digital item for rating after this digital item. In another example, the online recommendation platform may determine and then provide recommended digital items to the user in response to the user launching the online recommendation platform on his electronic device or in response to sending a general request for recommendations. For example, the recommended digital items may be of a similar category or may be provided by the same provider.

[004] In general, in order to generate individual user recommendations for users, such online recommendation platforms may execute a Machine-Learning Algorithm (MLA) specially trained for this purpose. In particular, the MLA algorithm may be trained based on a training dataset containing various features indicating (a) the users of the online recommendation platform and/or (b) the digital items posted thereon and/or (c) the past user actions of users with the digital items. The latter may include user responses left by users for the digital items after reviewing them, such as a “Like” or “Dislike” review, comments, etc.

[005] In this case, new digital items uploaded to the online recommendation platform may lack some features due to the lack of previous user responses from users and may therefore be ignored by the MLA algorithm when generating user recommendations. In the art, this problem is known as the cold start problem, where new digital items are not recommended to users of the online recommendation platform because other users have not yet interacted with or rated them. Consequently, users may not receive relevant user recommendations containing new digital items, and providers of new items may not receive their target audience on the online recommendation platform. As a result, user satisfaction may be negatively affected for both parties.

[006] To solve the technical problem described above, some well-known approaches have been proposed.

[007] In patent application US 2006085818 A1 "Introducing new content items in a community-based recommendation system" (Koninklijke Philips NV, published April 20, 2006) a method is described for working with content items, forming a user preference profile and presenting new content items in a community-based recommendation system. Upon detection of the availability of a new content item, a virtual user terminal determines a new user preference profile. The virtual user terminal comprises an initialization processor that sets a preference value for the new content item in the user preference profile. The virtual user terminal further comprises a relationship processor that determines the corresponding content items and a profile processor that sets the preference values in the user preference profile associated with at least one content item. In this way, a relationship is formed between the first content item and the said content items, as a result of which the new content item is associated with the existing content items and, consequently, the likelihood of recommending the new content item is increased.

[008] Patent application CN 114238752 A "Article recommendation method and device and storage medium" (Hunan University, published March 25, 2022) describes a method for recommending items and equipment that improve the accuracy of a cold start. The method includes the following steps: determining multimodal information for each cold start item in a set of cold start items and multimodal information for each hot start item in a set of hot start items, wherein the multimodal information comprises at least two items from among identification information, characteristics of visual information, characteristics of audio information, and characteristics of text information; performing hierarchical clustering according to the multimodal information of each cold start item and each hot start item to obtain a plurality of clustering results; forming an additional interaction graph between each cold start item and each hot start item according to the plurality of clustering results, wherein the hot start item corresponds to the set of hot start items; determining a first final representation of each cold start object and a second final representation of a hot start object corresponding to each cold start object according to the additional interaction graph; and recommending each cold start object according to the first final representation and the second final representation.

[009] Patent CN 10833492 B "Personalized recommendation method based on combination of content and collaborative filtering" (Nanjing University of Posts and Telecommunications, issued on November 2, 2021) discloses a personalized recommendation method based on a combination of content and collaborative filtering, which includes two stages of training and personalized recommendation: in the training stage, an "existing user" - "new item" preliminary recommendation matrix is formed by collecting data; in the personalized recommendation stage, an item is recommended for a new user and an additional recommendation is made after the user has generated ratings. Compared with the traditional personalized recommendation method, this method and system can solve the cold start problem, perform a prediction for newly added items, recommend the newly added items to the user, and use the recommendation based on collaborative filtering after the preliminary recommendation, so as to improve the diversity and correctness of the recommended items. In addition, by generating virtual users through a clustering algorithm, the matrix size is significantly reduced and the calculation complexity is reduced.

Disclosure of invention

[010] In connection with the above, systems and methods are needed that allow one to avoid, reduce or overcome the limitations of known technologies.

[011] The developers of the present technology have developed methods and systems for solving the cold start problem for new digital items uploaded to the online recommendation platform. In particular, the developers have developed methods and systems for generating synthesized user responses for new digital items based on the actual user response provided for existing old digital items of the online recommendation platform by its users.

[012] In some non-limiting embodiments of the present technology, the present methods and systems are designed to train a Machine Learning (ML) scoring model to determine synthesized user responses to assign to new digital items. In particular, in these embodiments of the invention, the ML scoring model is trained to reproduce the current distribution of responses to indicate actual user responses provided for old digital items, such as positive and negative reviews, for example.

[013] According to at least other non-limiting embodiments of the present technology, the synthesized user responses for new digital items uploaded to the online recommendation platform by a provider of new items may mimic the actual user responses provided for old digital items of a provider of old items similar to the provider of new items. In this regard, in these embodiments of the invention, the present methods and systems are intended to (a) determine a current distribution of responses for indications of actual user responses provided for old digital items of a provider of old items, and (b) to assign indications to the synthesized user responses for new digital items according to the current distribution of responses thus determined.

[014] New digital items with such synthesized user responses are considered by the MLA algorithm of the online recommendation platform and can thereby obtain user recommendations for users. Thus, the methods described in this document are able to provide a better presentation of new digital items to users of the online recommendation platform, allowing new digital items to obtain actual user responses, which (if different) can subsequently replace the synthesized user responses.

[015] Therefore, various non-limiting embodiments of the present technology can improve the satisfaction of users and providers of elements of online recommendation platforms.

[016] In particular, according to the first aspect of the present technology, a method is implemented for generating a training data set for training an MLA algorithm to generate recommendations of digital items for users of an online recommendation platform. The online recommendation platform contains a plurality of digital items. The plurality of digital items contains: (1) an old portion of the digital items, each of which was uploaded to the online recommendation platform by one of a plurality of old item providers, wherein at least some of the old digital items from the old portion of the digital items received a user response from at least one user of the online recommendation platform, and indications of user responses of at least some of the old digital items are distributed in the old portion of the digital items according to a distribution of responses; and (2) a new portion of the digital items, each of which was uploaded to the online recommendation platform by a new item provider from a plurality of new item providers and does not have a user response. Generating the training data set includes generating synthesized user responses for the new digital items. The method is performed by a server hosting the online recommendation platform. The method comprises: at a first stage - training by a server of an ML evaluation model to determine a predicted indication of a user response for a digital item, wherein the training comprises receiving by the server indications of an old part of digital items in order to form a plurality of training digital objects for evaluation, each of which contains (a) an old digital item from the old part of digital items and (b) an indication of a user response received by the old digital item, input by the server of a plurality of digital objects for evaluation into the ML evaluation model and thus training the ML evaluation model to determine a predicted indication of a user response for a digital item using the distribution of responses for indications of user responses in the old part of digital items as actual data; at a second stage, following the first stage - receiving by the server an indication of a supplier of new items from a plurality of suppliers of new items who has uploaded a plurality of new digital items from the new part of digital items to the online recommendation platform; applying by the server of the ML evaluation model in relation to a new digital item from a plurality of new digital items of the supplier of new items in order to determine an indication of a synthesized user response for the new digital item; generating by the server a training data set containing a plurality of training digital objects, wherein the training digital object contains (a) one of a plurality of new digital elements of a supplier of new elements and (b) an indication of a synthesized user response; and inputting by the server the plurality of training digital objects into an MLA algorithm for training the MLA algorithm to generate recommendations of digital elements.

[017] In some embodiments of the method, using the ML scoring model includes determining a first value of the probability of receiving an indication of a positive user response by a new digital item of a supplier of new digital items, wherein the determination is based on an assessment of the similarity of the supplier of new items with a plurality of suppliers of old items.

[018] In some embodiments of the method, the first probability value is determined by the formula:

,

where artistScore is the similarity score of the new item supplier to the set of old item suppliers, a and b are coefficients determined by training the machine learning scoring model.

[019] In some embodiments of the method, it further includes determining, by the server, an estimate of the similarity of the supplier of the new items with the plurality of suppliers of the old items.

[020] In some embodiments of the method, determining a similarity score of a supplier of new elements includes: obtaining, for a supplier of old elements from a plurality of suppliers of old elements, a value of a similarity parameter of the supplier of old elements with a supplier of new elements; generating, by a server, based on the values of the similarity parameter of each supplier from the plurality of suppliers of old elements with a supplier of new elements, a directed graph structure, where a vertex of the directed graph structure represents a supplier of elements from the plurality of suppliers of old elements or from the plurality of suppliers of new elements, and an edge connecting a pair of vertices represents a non-zero value of the similarity parameter of the supplier of new elements with one of the plurality of suppliers of old elements; determining, by the server, for the edge, a weight value indicating the value of the similarity parameter of the supplier of new elements with one of the plurality of suppliers of old elements; and determining, by the server, the similarity score of the supplier of new elements as the maximum total weight value when moving from the vertex of the directed graph structure associated with the supplier of new elements to its initial vertex.

[021] In some embodiments of the method, determining the weight value includes determining the inverse value for the similarity parameter value of the supplier of new items with one of the plurality of suppliers of old items.

[022] In some embodiments of the method, the similarity parameter represents a degree of similarity of the supplier of new elements with each of the plurality of suppliers of old elements.

[023] In some embodiments of the method, it further includes receiving from the human evaluator a degree of similarity of the supplier of the new items to each of the plurality of suppliers of the old items.

[024] In some embodiments of the method, applying the ML evaluation model further includes determining a second value of the probability of receiving an indication of a positive user response for the new digital item of the provider of the new digital items, wherein the determination is based on a value of the popularity of the new digital item in the given area.

[025] In some embodiments of the method, the second probability value is determined by the formula:

,

where Popularity is the popularity value of a new digital element in a given area, maxPopularity is the popularity value of the most popular new digital element in a given area from the corresponding set of digital elements of the supplier of new elements, minPopularity is the popularity value of the least popular new digital element in a given area from the corresponding set of digital elements of the supplier of new elements, c and d are coefficients determined by training the evaluating machine learning model.

[026] In some embodiments of the method, the new digital item is an audio stream, and the popularity value of the new digital item indicates at least one of the following: (a) the number of streams of the audio stream on the media resource, (b) the number of requests to play the audio stream on the media resource, (c) the position of the audio stream in the hit parade, and (d) the number of albums sold containing the audio stream.

[027] In some embodiments of the method, it further includes assigning by the server to the new digital element an indication of a positive synthesized user response if the first and/or second probability value is greater than an upper probability threshold, and assigning by the server to the new digital element an indication of a negative synthesized user response if the first and/or second probability value is less than a lower probability threshold, which is less than the upper probability threshold.

[028] In some embodiments of the method, before applying the ML evaluation model, it further includes the server selecting a new digital element from a plurality of new digital elements.

[029] In some embodiments of the method, this selection includes selecting a new digital item from a plurality of new digital items in one of the following ways: (a) randomly, (b) using uniform selection probabilities, (c) using a selection probability value proportional to a popularity value of the new digital item in the given area, and (d) using a selection probability value proportional to the square root of the popularity value of the new digital item in the given area.

[030] In some embodiments of the method, this selection and application is performed iteratively for a predetermined number of iterations before forming the training data set.

[031] In some embodiments of the method, it further includes updating by the server the training digital object from the plurality of training digital objects by replacing therein the indication of the synthesized user response with an indication of the user response if the server has determined that a new digital element from the plurality of new digital elements has received a user response.

[032] In some embodiments of the method, the ML estimation model is a matrix factorization ML model.

[033] In some embodiments of the method, the MLA algorithm is a decision tree-based MLA algorithm, a transformer-based MLA algorithm, or a deep semantic similarity-based MLA algorithm.

[034] According to a second aspect of the present technology, a method for generating a training data set for training an MLA algorithm to generate recommendations of digital items for users of an online recommendation platform is implemented. The online recommendation platform contains a plurality of digital items. Generating the training data set includes generating synthesized user responses for new digital items. The method is performed by a server on which the online recommendation platform is hosted. The method includes: the server receiving an indication of a supplier of new items from among the suppliers of new items that has a plurality of new digital items to be uploaded to the online recommendation platform, wherein the new digital item from the plurality of new digital items does not have a previously received user response; the server receiving an indication of a plurality of suppliers of old items that have uploaded a plurality of old digital items, at least some of which have received a user response from at least one user of the online recommendation platform; the server analyzing the supplier of new items and the plurality of suppliers of old items in order to determine a reference supplier of old items based on the similarity parameter of the supplier of new items with each of the plurality of suppliers of old items; based on user responses received by a plurality of old digital items of a reference supplier of old items, determining by a server synthesized user responses for a plurality of new digital items of a supplier of new items, including determining by the server a distribution of responses for indications of user responses in a plurality of old digital items associated with the reference supplier of old items, and assigning by the server indications of the synthesized user responses to the plurality of new digital items of the supplier of new items according to the distribution of responses for indications of user responses in a plurality of old digital items of the reference supplier of old items; generating by the server a training data set comprising a plurality of training digital objects, wherein the training digital object comprises (a) one of a plurality of new digital items of the supplier of new items and (b) an indication of the synthesized user response; and inputting by the server the plurality of training digital objects into an MLA algorithm for training the MLA algorithm to generate recommendations of digital items.

[035] According to a third aspect of the present technology, a server is implemented for generating a training data set for the purpose of training an MLA algorithm to generate recommendations of digital items for users of an online recommendation platform hosted on the server. The online recommendation platform contains a plurality of digital items. The plurality of digital items contains (1) an old portion of the digital items, each of which was uploaded to the online recommendation platform by one of a plurality of old item providers, wherein at least some of the old digital items from the old portion of the digital items received a user response from at least one user of the online recommendation platform, and indications of user responses of at least some of the old digital items are distributed in the old portion of the digital items according to a distribution of responses, and (2) a new portion of the digital items, each of which was uploaded to the online recommendation platform by a new item provider from a plurality of new item providers and does not have a user response. The generating includes generating synthesized user responses for the new digital items. The server contains a processor and a machine-readable physical memory storing instructions. When executing commands, the processor is capable of: at the first stage - training the ML evaluation model to determine the predicted indication of the user response for the digital item by receiving indications of the old part of the digital items in order to form a plurality of training digital objects for evaluation, each of which contains (a) the old digital item from the old part of the digital items and (b) an indication of the user response received by the old digital item, and inputting the plurality of digital objects for evaluation into the ML evaluation model and thereby training the ML evaluation model to determine the predicted indication of the user response for the digital item using the distribution of responses for indications of the user responses in the old part of the digital items as actual data; at the second stage following the first - receiving an indication of a supplier of new items from a plurality of suppliers of new items who has uploaded a plurality of new digital items from the new part of the digital items to the online recommendation platform, and applying the ML evaluation model in relation to the new digital item from the plurality of new digital items of the supplier of new items in order to determine the indication of the synthesized user response for the new digital item; generate a training data set comprising a plurality of training digital objects, wherein the training digital object comprises (a) one of a plurality of new digital items of a supplier of new items and (b) an indication of a synthesized user response; and input the plurality of training digital objects into an MLA algorithm for training the MLA algorithm to generate recommendations for digital items.

[036] In the context of the present description, the term "server" means a computer program executed by appropriate hardware and capable of receiving requests (e.g. from electronic devices) via a network and executing or causing these requests to be executed. The hardware may be one physical computer or one computer system, which is not essential for the present technology. In the present context, the expression "server" does not mean that every task (e.g. received command or request) or some specific task is received, executed or started by the same server (i.e. the same software and/or hardware). This expression means that any number of software or hardware can receive, send, execute or cause to be executed any task or request or the results of any tasks or requests. All of these software and hardware may represent one server or several servers, and both of these cases are implied in the expression "at least one server".

[037] In the context of the present description, the term "electronic device" means any computer hardware capable of executing programs suitable for solving a given task. In the context of the present description, the term "electronic device" implies that the device can function as a server for other electronic devices, however, this is not necessary for the present technology. Thus, some (non-limiting) examples of electronic devices include an autonomous driving unit, personal computers (desktops, laptops, netbooks, etc.), smartphones and tablets, as well as network equipment such as routers, switches and gateways. It should be understood that in the present context, the fact that a device functions as an electronic device does not mean that it cannot function as a server for other electronic devices.

[038] In the context of the present description, the term "information" includes information of any kind or type capable of being stored in a database. Thus, information includes, but is not limited to, visual works (e.g., maps), audiovisual works (e.g., images, films, sound recordings, presentations, etc.), data (e.g., location data, weather data, traffic data, numerical data, etc.), text (e.g., opinions, comments, questions, messages, etc.), documents, spreadsheets, etc.

[039] As used herein, the term "database" means any structured collection of data, regardless of its specific structure, database management software, or computer hardware for storing, using, or otherwise enabling the use of that data. A database may be located on the same hardware as the process that enables the storage or use of the information stored in the database, or the database may be located on separate hardware, such as a dedicated server or multiple servers.

[040] In the context of the present description, the numerals "first," "second," "third," etc. are used merely to indicate a distinction between the nouns to which they refer, but not to describe any specific relationship between those nouns. Furthermore, as occurs in another context in the present description, reference to a "first" element and a "second" element does not exclude the possibility that these two elements may in fact be the same element.

[041] Each embodiment of the present technology relates to at least one of the above-mentioned objectives and/or to one of the above-mentioned aspects, but not necessarily to all of them. It should be understood that some aspects of the present technology associated with an attempt to achieve the above-mentioned objective may not correspond to this objective and/or may correspond to other objectives not explicitly mentioned herein.

[042] Additional and/or alternative features, aspects and advantages of embodiments of the present technology are contained in the following description, in the attached drawings and in the claims.

Brief description of the drawings

[043] These and other features, aspects and advantages of the present technology are explained in the following description, in the appended claims and in the following drawings.

[044] Fig. 1 is a diagram of an illustrative computer system that may be used to implement certain non-limiting embodiments of the present technology.

[045] Fig. 2 is a diagram of a network computing environment comprising the computer system of Fig. 1 and suitable for use with certain non-limiting embodiments of the present technology.

[046] Fig. 3 is a schematic diagram of a graphical user interface (GUI) of an online recommendation platform hosted on a server in the network computing environment shown in Fig. 2, according to some non-limiting embodiments of the present technology.

[047] Fig. 4A and 4B are schematic diagrams of the structure of input and output data of the ML evaluation model that is used by the server of the network computing environment shown in Fig. 2 to determine predicted user response indications for a plurality of digital elements of the online recommendation platform, according to some non-limiting embodiments of the present technology.

[048] Fig. 5 is a schematic diagram of a similarity graph of item suppliers for suppliers of a plurality of digital items hosted on an online recommendation platform, according to some non-limiting embodiments of the present technology.

[049] Fig. 6 is a schematic diagram of a step for determining the similarity of a new and old supplier of items from a plurality of digital items hosted on an online recommendation platform, according to some non-limiting embodiments of the present technology.

[050] Fig. 7 is a diagram of the architecture of a machine learning model used to implement at least some non-limiting embodiments of the present technology.

[051] Fig. 8 shows a diagram of a first method for generating by a server from the network computing environment shown in Fig. 2 a training data set for training the MLA algorithm to generate recommendations of digital elements for users of the online recommendation platform, according to some non-limiting embodiments of the present technology.

[052] Fig. 9 shows a diagram of a second method for the server to generate, from the network computing environment shown in Fig. 2, a training data set for training the MLA algorithm to generate recommendations of digital elements for users of the online recommendation platform, according to some non-limiting embodiments of the present technology.

Implementation of the invention

[053] The examples and conventional language provided herein are intended to provide a better understanding of the principles of the present technology, and not to limit its scope to such specifically provided examples and conditions. It is obvious that those skilled in the art are capable of devising various methods and devices that are not explicitly described or shown, but implement the principles of the present technology within its spirit and scope.

[054] Furthermore, in order to facilitate better understanding, the following description may contain simplified embodiments of the present technology. Those skilled in the art will understand that other embodiments of the present technology may be significantly more complex.

[055] In some cases, useful examples of modifications of the present technology are given. They facilitate understanding, but also do not define the scope or boundaries of the present technology. The presented list of modifications is not exhaustive and a person skilled in the art can develop other modifications within the scope of the present technology. In addition, if in some cases modifications are not described, this does not mean that they are impossible and/or that the description contains the only possible embodiment of a particular element of the present technology.

[056] Moreover, the description of the principles, aspects and embodiments of the present technology, as well as specific examples thereof, are intended to cover their structural and functional equivalents, regardless of whether they are currently known or will be developed in the future. For example, those skilled in the art should understand that any block diagrams described herein correspond to conceptual representations of illustrative circuit diagrams implementing the principles of the present technology. It should also be understood that any block diagrams, process diagrams, state transition diagrams, pseudocodes and the like correspond to various processes that can be represented on a machine-readable physical storage medium and can be executed by a computer or processor, regardless of whether such a computer or processor is explicitly shown or not.

[057] The functions of the various elements shown in the drawings, including any functional unit designated as a "processor", may be implemented using dedicated hardware as well as using hardware capable of executing associated software. If a processor is used, these functions may be performed by a single dedicated processor, a single shared processor, or a plurality of separate processors, some of which may be shared. Furthermore, the explicit use of the term "processor" or "controller" shall not be construed as referring exclusively to hardware capable of executing software, and may include, but are not limited to, a digital signal processor (DSP), a network processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a read-only memory (ROM) for storing software, a random access memory (RAM), and/or a non-volatile memory. Other general-purpose and/or custom hardware may also be included.

[058] Software modules or simply modules, the implementation of which is intended to be in the form of software, may be represented here as any combination of flow chart elements or other elements indicating the execution of process steps and/or containing a text description. Such modules may be executed by hardware, shown explicitly or implied.

[059] Next, taking into account the above principles, some non-limiting examples are considered, illustrating various options for implementing aspects of this technology.

Computer system

[060] Fig. 1 shows a computer system 100 suitable for use in some embodiments of the present technology. The computer system 100 includes various hardware elements, including one or more single- or multi-core processors, generally represented by a processor 110, a graphics processor 111, a solid-state drive 120, RAM 130, a display interface 140, and an input/output interface 150.

[061] Communication between the various elements of the computer system 100 may be accomplished via one or more internal and/or external buses 160, such as a PCI bus, a USB bus, an IEEE 1394 FireWire bus, a SCSI bus, a Serial-ATA bus, etc., to which the various hardware elements are electronically connected.

[062] The input/output interface 150 may be connected to a touch screen 190 and/or to one or more internal and/or external buses 160. The touch screen 190 may also be referred to as a screen, such as the screen (not specifically indicated) of the electronic device 210 shown in Fig. 2. In the embodiments shown in Figs. 1 embodiments of the invention, the touch screen 190 includes touch means 194 (for example, pressure-sensitive cells embedded in the display layer and capable of detecting physical interaction between a user and the display) and a touch input/output controller 192 that communicates with the display interface 140 and/or with one or more internal and/or external buses 160. In some non-limiting embodiments of the present technology, the input/output interface 150 may be connected to a keyboard (not shown), a mouse (not shown), or a touch pad (not shown) that enable user interaction with the computer system 100 in addition to or instead of the touch screen 190.

[063] It should be noted that in some non-limiting embodiments of the present technology, some elements of the computer system 100 may be missing. For example, a keyboard and mouse (not shown separately) may be missing, particularly if the computer system 100 is implemented as a small-sized electronic device, such as (but not limited to) a smartphone.

[064] According to embodiments of the present technology, the solid-state drive 120 stores software instructions suitable for loading into the RAM 130 and executing by the processor 110 and/or the GPU processor 111. The software instructions may, for example, be part of a library or an application.

Network computing environment

[065] Fig. 2 is a diagram of a network computing environment 200 suitable for use with some non-limiting embodiments of the present technology. The network computing environment 200 includes an electronic device 210 connected via a communication network 240 to a server 250. In some non-limiting embodiments of the present technology, the electronic device 210 may be associated with a user 220.

[066] In non-limiting embodiments of the present technology, the electronic device 210 may be any computer hardware capable of executing programs suitable for solving the assigned task. Thus, some non-limiting examples of the electronic device 210 may include personal computers (desktops, laptops, netbooks, etc.), smartphones, and tablets. The electronic device 210 may include some or all of the elements of the computer system 100 shown in Fig. 1.

[067] According to some non-limiting embodiments of the present technology, an online recommendation platform 260 may be hosted on the server 250 and may be accessible to the processor 110 of the electronic device 210 via the communication network 240. In general, the online recommendation platform 260 is a web resource that provides the user 220 with access to a plurality 235 of digital items stored in a database 230 connected to the server 250 via a corresponding communication line. The online recommendation platform 260 may determine digital items that may be of interest to the user 220 and then transmit indications of such digital items to the electronic device 210 for evaluation by the user.

[068] There are no limitations on the implementation of the online recommendation platform 260. In some non-limiting embodiments of the present technology, the online recommendation platform 260 can be implemented as a streaming audio platform, such as the Sportify™ streaming audio platform, the Yandex.Music™ streaming audio platform, the YouTube.Music™ streaming audio platform, etc., and the plurality 235 digital items stored in the database 230 can contain various audio tracks, such as songs, podcasts, audiobooks, etc. In other non-limiting embodiments of the present technology, the online recommendation platform can be implemented as a streaming video platform, such as the Netflix™ streaming video platform, the Amazon™ Prime Video streaming platform, etc., and the plurality 235 digital items can, for example, contain video clips, movies, and television programs. In other non-limiting embodiments of the present technology, the online recommendation platform 260 may be implemented as an online listing platform, such as the Yandex.Market™ online listing platform or the Amazon™ online listing platform, and the plurality of 235 digital items may contain categorized lists of various goods and services available for online purchase. Other embodiments of the online recommendation platform 260 are also possible without going beyond the boundaries of the present technology.

[069] In some non-limiting embodiments of the present technology, server 250 may be implemented as a conventional computer server and may include some or all of the elements of computer system 100 shown in Fig. 1. In one non-limiting example, server 250 is implemented as a Dell™ PowerEdge™ server running a Microsoft™ Windows Server™ operating system, but it may also be implemented using any other suitable hardware, application software, and/or firmware, or a combination thereof. In the illustrated non-limiting embodiments of the present technology, server 250 is a single server. In other non-limiting embodiments of the present technology (not shown), the functions of server 250 may be distributed among multiple servers.

[070] In some non-limiting embodiments of the present technology, the server 250 may be operated by an organization that operates the online recommendation platform 260. For example, if the online recommendation platform 260 is the Yandex.Music™ streaming audio platform, then the server 250 may also be operated by Yandex LLC (16 Leo Tolstoy St., Moscow, 119021, Russia). In alternative non-limiting embodiments of the present technology, the server 250 may be operated by an organization other than the one that provides the online recommendation platform 260.

[071] For example, using the electronic device 210, the user 220 can access the online recommendation platform 260 from the electronic device 210. In addition, the user 220 can send a recommendation request 215 to the server 250. In response, the server 250 can (a) determine a set 225 of recommended digital items and (b) send indications of the set 225 of recommended digital items to the electronic device for presentation to the user 220. There are no restrictions on the method for sending the recommendation request 215 by the user 220.

Communication network

[072] In some non-limiting embodiments of the present technology, the communication network 240 is the Internet. In alternative non-limiting embodiments of the present technology, the communication network 240 may be implemented as any suitable local area network (LAN), wide area network (WAN), private communication network, etc. It should be understood that the embodiments of the communication network are given for illustrative purposes only. The implementation of the respective communication links (not separately indicated) between the electronic device 210 and the server 250 on the one hand and the communication network 240 on the other hand depends, among other things, on the implementation of the electronic device 210 and the server 250. Just as a non-limiting example, in those embodiments of the present technology where the electronic device 210 is implemented as a wireless communication device, such as a smartphone, the communication link may be implemented as a wireless communication link. Examples of wireless communication links include a 3G communication network channel, a 4G communication network channel, etc. The communication network 240 may also use a wireless connection to the server 250.

Recommendation platform

[073] Fig. 3 illustrates a graphical user interface (GUI) 300 of an online recommendation platform 260 displayed on an electronic device 210 associated with a user 220, according to some non-limiting embodiments of the present technology.

[074] According to some non-limiting embodiments of the present technology, the server 250 may cause the electronic device 210 to display the GUI interface 300 in response to the user 220 sending a request to access the online recommendation platform 260, such as by entering a universal resource locator (URL) of the server 250 into a browser application (not separately shown) of the electronic device 210 or by launching an application (not separately shown in Fig. 3) associated with the online recommendation platform 260 therein.

[075] It is contemplated that in some non-limiting embodiments of the present technology, the GUI interface 300 may comprise at least the following: (a) indications of at least a portion of the set 225 of recommended digital items, which, for example, depends on the dimensions of the screen of the electronic device 210; (b) an indication of the currently presented recommended digital item 302; (c) controls 304 by which the user 220 can control the presentation of the recommended digital item; and (d) feedback controls 306 by which the user 220 can leave his/her feedback for a recommended digital item from the set 225 of recommended digital items, such as the currently presented recommended digital item 302.

[076] Although in the embodiments of the invention according to Fig. 3 the GUI interface 300 represents an online recommendation platform 260 implemented as a streaming audio platform, it should be understood that in other non-limiting embodiments of the present technology, similar GUI interfaces may be displayed for other types of online recommendation platforms 260, some of which are indicated above.

[077] There is no limitation on the method for sending a recommendation request 215 from the electronic device 210 regarding the set 225 recommended digital items. In some non-limiting embodiments of the present technology, the user 220 can explicitly send a recommendation request 215 to the server 250, for example, using a special control of the GUI interface 300 (such as "Play Radio" (not shown in Fig. 3) in the example where the online recommendation platform 260 is a streaming audio platform). In other non-limiting embodiments of the present technology, the recommendation request 215 can be initiated by the user 220 accessing the online recommendation platform 260 from the electronic device 210 by launching the GUI interface 300. In other non-limiting embodiments of the present technology, the recommendation request 215 can be initiated by the user 220 rating a digital item from the plurality 235 digital items.

[078] In some non-limiting embodiments of the present technology, the server 250 may generate a set 225 of recommended digital items for the user 220 based at least on an item popularity value of each digital item from the plurality 235 digital items, as described in more detail below. According to some non-limiting embodiments of the present technology, the server 250 may determine an item popularity value for a digital item from the plurality 235 digital items posted on the online recommendation platform 260 based on user actions of users of the online recommendation platform 260, such as the user 220, with this digital item. For example, in some non-limiting embodiments of the present technology, such user actions may include posting, for example using the feedback controls 306, a user response under the digital item.

[079] According to some non-limiting embodiments of the present technology, the GUI interface 300 of the online recommendation platform 260 may comprise feedback controls 306 that allow a user response to be indicated using binary values, such as "Like" or "Dislike", respectively, indicating a positive or negative user action of the user 220 with the digital item. In other non-limiting embodiments of the present technology, the feedback controls 306 may comprise another configuration, such as star indicators, emotion indicators, and the like (not shown), that allow a user response to be indicated using one value from a predetermined range of values, such as from 1 to 5 or from 1 to 10, where the value 1 indicates a negative user action of the user 220 with the digital item, and the value 5 or 10, respectively, indicates a positive user action of the user 220 with the digital item.

[080] In some non-limiting embodiments of the present technology, the server 250 may determine the popularity value of the item based on other user actions of the user 220 with the digital item, which, like the user response, may also indicate a positive or negative user action with the digital item. According to some non-limiting embodiments of the present technology, such other user actions of the user 220 indicating a positive user action with the digital item may include: (a) selecting the digital item for rating; (b) searching for the digital item on the online recommendation platform 260; (c) using the "share" function for the digital item through the user 220 social networking accounts; (d) enabling a repeat presentation of the digital item; (e) adding the digital item to the user 220's list of favorite digital items, etc.

[081] User actions of the user 220 indicating a negative user action with a digital element from the plurality 235 of digital elements may include, among other things: (a) skipping a portion of the digital element; (b) skipping the entire digital element and moving on to the next digital element; (c) interrupting the presentation of the digital element and then searching for another digital element for evaluation, etc. Other user actions of the user 220 indicating a positive and negative user action with a digital element are also possible, without going beyond the scope of the present technology.

[082] Thus, according to some non-limiting embodiments of the present technology, in response to receiving a recommendation request 215 from the electronic device 210, the server 250, based on at least indications of user responses, may (a) determine an item popularity value for each item from the plurality 235 digital items and (b), based on the popularity values of the items from the plurality 235 digital items, determine a set 225 recommended digital items for provision to the user 220, as described above. For example, a recommended digital item from the set 225 recommended digital items may be one of the most popular digital items from the plurality 235 digital items. In another example, the server 250 may determine as a recommended digital item a digital item provided by an item provider associated with at least one of the most popular digital items. In another example, the server 250 may determine as a recommended digital item for the user 220 a digital item that is most popular among users similar to the user 220, based on predetermined user features.

[083] In some non-limiting embodiments of the present technology, the server 250 may determine the set 225 of recommended digital items so as to maximize a probability value of a positive user action of the user 220 with each element of the set 225 of recommended digital items, as indicated by, for example, a positive user response. In other non-limiting embodiments of the present technology, the server 250 may determine the set 225 of recommended digital items so as to minimize a probability value of a negative user action of the user 220 with each element of the set 225 of recommended digital items, as indicated by, for example, a negative user response.

[084] In particular, for this purpose, in some non-limiting embodiments of the present technology, the server 250 may determine for the set 225 of recommended digital items recommended digital items with element features similar to the features of those digital items with which the user 220 has performed the most positive user actions, as described above. In another example, the server 250 may determine for inclusion in the set 225 of recommended digital items recommended digital items from among the items with positive user actions by other users of the online recommendation platform 260 with user features similar to the features of the user 220.

[085] According to some non-limiting embodiments of the present technology, the element features associated with the digital element may indicate its own parameters, which may conceivably depend on the nature of the digital element. For example, in those embodiments of the invention where the online recommendation platform 260 is an audio streaming platform and the digital element is a phonogram, the element features may, among other things, include: (a) the genre of the phonogram; (b) a predetermined mood evoked by the phonogram; (c) a popular period associated with the phonogram, such as the sixties, seventies, nineties, etc.; (d) a provider of the phonogram, such as an artist, producer, recording studio, etc.; (e) the length of the phonogram; (f) the release date of the phonogram; (g) the rhythm of the phonogram; (h) the album of the phonogram, etc.

[086] User characteristics of a user of the online recommendation platform 260, such as a user 220, may include, among other things: (a) socio-demographic characteristics of the user 220, including the age of the user 220, the gender of the user 220, the employment status of the user 220, the average income of the user 220, etc.; (b) data indicating the search history of the user 220; (c) data indicating the web browsing history of the user 220, etc.

[087] In some non-limiting embodiments of the present technology, to determine the set 225 of recommended digital items for the user 220, as described above, the server 250 may execute the MLA algorithm 280 trained to determine the recommended digital items based on the above-mentioned features for each user of the online recommendation platform 260. In particular, the executed MLA algorithm 280 may be trained to determine the probability values of a positive action by the user 220 with each item from the plurality 235 of digital items, such as the user 220 generating a positive user response for it. In addition, the server 250 may (a) rank the digital items according to the corresponding probability values and (b) select N digital items with the highest rank, such as 100, 500 or 1000, for inclusion in the set 225 of recommended digital items.

[088] As described in more detail below, according to some non-limiting embodiments of the present technology, the server 250 may train the MLA algorithm 280 based on a training data set comprising a plurality of training digital objects, each of which may comprise, among other things: (a) an indication of a digital item from the plurality 235 digital items; (b) item features for the digital item; (c) indications of past user actions of users of the online recommendation platform 260 with the digital item, such as a user response provided for the current digital item, as described above; (d) user features of users who have interacted with the digital item, such as by leaving their user responses under it.

[089] Thus, by determining and providing recommended digital items to users of the online recommendation platform 260, the server 250 can: (a) present various digital items to the users; (b) receive their user responses for the recommended digital items; (c) improve the accuracy of subsequent recommendations based on the user responses. It is believed that by providing such recommended items, the satisfaction of (a) the users can be improved by providing them with digital items of interest to them and (b) the item providers by more fully presenting their various digital items to users of the online recommendation platform 260.

[090] According to some non-limiting embodiments of the present technology, the plurality 235 of digital items posted on the online recommendation platform 260 may comprise (a) an old portion 255 of digital items uploaded to the online recommendation platform 260 by a plurality 245 of old item providers using their electronic devices (not separately indicated), and (b) a new portion 275 of digital items uploaded to the online recommendation platform 260 by a plurality 265 of new item providers using their electronic devices (not separately indicated).

[091] In the context of the present description, the term "item provider" generally means a person, such as an individual, musician or music producer, or an organization, such as a music recording studio, that has a plurality of digital items uploaded to the online recommendation platform 260 for shared access by users.

[092] In addition, according to some non-limiting embodiments of the present technology, an old digital item from the old portion 255 of digital items means a digital item that has been available on the online recommendation platform 260 for longer than a predetermined threshold of availability period, such as, for example, one week, one month, or one year. In other non-limiting embodiments of the present technology, old digital items may be digital items that have received at least one indication of a user response from at least one user of the online recommendation platform 260. In other non-limiting embodiments of the present technology, an old digital item may be a digital item provided by a provider from the plurality 245 of old item providers that is present on the online recommendation platform 260 for longer than a predetermined threshold of presence time, which may correspond to, for example, a month, a quarter, or a year. Thus, in at least some non-limiting embodiments of the present technology, at least some items from the plurality 257 legacy digital items uploaded to the online recommendation platform 260 by the provider 247 legacy items have received at least one indication of a user response from at least one user.

[093] In the context of the present description, according to some non-limiting embodiments of the present technology, a new digital item from the new portion 275 of digital items is a digital item that has no indication of user actions with it, including a user response, for example, because (a) it was available on the online recommendation platform 260 for less than a predetermined threshold value of the availability period or (b) it was uploaded by a provider from the plurality 265 providers of new items that is unknown to users of the online recommendation platform 260 (unpopular). For example, the plurality 277 of new digital items of the provider 267 of new items may become available on the online recommendation platform 260 as a result of expanding the service area of the online recommendation platform 260 to a new geographic area, the providers of items from which were not previously present on the online recommendation platform 260.

[094] Since each item from the set 277 new digital items has no user response or only a very limited user response, the server 250 may subsequently be unable to determine item popularity values for each item from the set 277 new digital items, and the MLA algorithm 280, trained to determine recommended digital items based on, among other things, the popularity values of items from the set 235 digital items, may ignore the set 277 new digital items of the provider 267 new items. Thus, items from the set 277 new digital items may not be included in the set 225 recommended digital items.

[095] Accordingly, such a “pessimization” of the set 277 digital elements when compiling the set 225 recommended digital elements may result in the user 220 not finding relevant recommended digital elements and, consequently, the new supplier 267 elements will lose its target audience.

[096] One of the simplest approaches to solving this data sparseness problem is to obtain item popularity values for each new digital item from the new portion 275 of digital items based on the correlation between its item features and the item features of each old digital item from the old portion 255 of digital items. However, due to the potentially large number of different item features and the number of digital items themselves, such an analysis can be a very resource-intensive task for the processor 110 of the server 250.

[097] The developers of the present technology have found that an indication of a user response can be determined directly for each new digital item from the new portion 275 of digital items based on indications of user responses distributed in the old portion 255 of digital items. In particular, the present methods and systems are intended to assign indications of a synthesized user response to the digital items from the new portion 275 of digital items, which, unlike the actual user response, was not provided by users of the online recommendation platform 260, but was determined by the server 250. In some non-limiting embodiments of the present technology, the present methods include training an algorithm to determine the values of the probability of providing indications of user responses (both positive and negative) by users of the online recommendation platform 260 for the digital items from the new portion 275 of digital items based on the distribution of indications of user responses in the old portion 255 of digital items.

[098] New digital items from the new portion 275 of digital items that have received indications of the synthesized user responses may be considered by the MLA algorithm 280 in the process of determining recommended digital items for the user 220, which allows for the new digital items to be presented to the users of the online recommendation platform 260. Accordingly, as described above, the result may be a mutual benefit for the users and for the plurality 265 providers of new items of the online recommendation platform 260, since the former may discover new relevant digital items of interest to them, and the latter may expand their target audience.

[099] Below, with reference to Figs. 4-6, a possible analysis of the distribution of user response indications in the old portion 255 of digital elements is described in detail for the purpose of determining synthetic response indications for new digital elements according to some non-limiting embodiments of the present technology.

ML Estimation Model

[0100] According to some non-limiting embodiments of the present technology, in order to determine the indications of the synthesized user responses for each new digital item from the new portion 275 of digital items, as described above, the server 250 may first, based on the indications of the user responses received for some items from the old portion 255 of digital items, train an ML scoring model to determine the predicted indications of the user responses for digital items, such as items from the old portion 255 of digital items that do not have indications of the user responses. There are no limitations on the implementation of the ML scoring model. However, in some non-limiting embodiments of the present technology, the ML scoring model may be implemented as a matrix factorization ML model.

[0101] Figs. 4A and 4B are schematic diagrams of an input matrix 402 and an output matrix 404 of an ML matrix factorization model according to some non-limiting embodiments of the present technology.

[0102] It is assumed that the input matrix 402 has a size of m*n , where m is the number of users of the online recommendation platform 260, n is the number of old digital items in the old part 255 of digital items. Thus, an element of the input matrix 402 contains: (a) an actual indication 406 of a user response, for example, “Like” or “Dislike” as described above, provided by a user, such as user 220, for an old digital item from the old part 255 of digital items; (b) item features for the old digital item; and (c) user features of user 220 who formed the actual indication 406 under the old digital item.

[0103] In general, the matrix factorization model is an ML model of vector representations that can be trained, for example, by the server 250, to determine a matrix P of vector representations of users (for example, users of the online recommendation platform 260) and a matrix Q of vector representations of items (for example, items from the old part 255 of digital items) such that the output matrix 404, defined as PQ ^T , approximates the input matrix 402 with a desired level of confidence. In particular, a row of the matrix Q of vector representations of items contains latent features of an item for an item from the old part 255 of digital items, representing its actual features of the item, and a row of the matrix P of vector representations of users contains latent features of a user, for example, user 220, which indicate the degree of interest of user 220 in an item with such actual features of an item from the old part 255 of digital items.

[0104] In this regard, the server 250 may (a) generate an objective function (also known in the art as a loss function) indicating the differences between the corresponding elements of the input and output matrices 402, 404, and (b) by minimizing the objective function, determine the elements of the matrices P and Q of the vector representations of the users and elements. For example, in some non-limiting embodiments of the present technology, the objective function may be a mean square error objective function, which may be represented by the following equation:

Where - input matrix element 402, - the corresponding element of the output matrix 404, - the corresponding element of the matrix P of vector representations of users, - the corresponding element of the matrix Q of vector representations of the elements.

[0105] Therefore, by minimizing the objective function, for example using the gradient descent method, the server 250 can determine the elements of the matrix P of vector representations of users and the matrix Q of vector representations of items. Based on the thus determined matrices P and Q of vector representations of users and vector representations of items, the server 250 can determine the elements of the output matrix 404, schematically represented in Fig. 4B. As shown in the output matrix 404, based on the similarity of the features of the item and the user associated with the items from the old portion 255 of digital items that have actual indications of user responses, the matrix factorization ML model trained in this way is capable of: (a) determining, for an item from the old portion 255 of digital items that previously did not have an indication of a user response, a predicted indication 408 of a user response; and (b) approximate the existing actual user response indications for the elements from the old portion 255 of digital elements that had them, such as for the old digital element associated with the actual user response indication 406.

[0106] In addition, in some non-limiting embodiments of the present technology, to determine the indications of a synthesized user response for each element of the new portion 275 of digital elements, the server 250 may (a) increase the size of the input matrix 402 by adding columns thereto indicating elements of the new portion 275 of digital elements; (b) re-determine the matrices P and Q of the vector representations of users and the vector representations of elements; and (c) generate an updated version of the output matrix 404, which thus contains indications of synthesized user responses that are predicted indications of user responses for each element of the new portion 275 of digital elements.

[0107] It should be understood that in addition to the matrix factorization model described above, other ML architectures may be used to implement the ML evaluation model, including, among others, a decision tree-based ML model, a neural network-based ML model, etc.

[0108] In this case, as described above, determining the relationships between the user features and the feature of the item for each item from the new portion 275 of digital items by retraining the ML model may be a task for the server 250 that requires significant resources. In addition, the feature of the item for some items from the new portion 275 of digital items may be unavailable.

[0109] In this regard, in some non-limiting embodiments of the present technology, the server 250 may, using the predictions of the ML scoring model, train an algorithm for generating synthesized user responses that is capable of generating an indication of a synthesized user response for each item from the new portion 275 of digital items based on sparse input data, for example, without taking into account the features of an item from the new portion 275 of digital items. In particular, according to some non-limiting embodiments of the present technology, the algorithm for generating synthesized user responses may determine an indication of a synthesized user response for each item from the new portion 275 of digital items such that they are distributed in the new portion 275 of digital items according to a distribution of responses for predicted indications of responses in the old portion 255 of digital items. In other words, the server 250 may train the algorithm for generating synthesized user responses using the distribution of responses of the predicted indications of responses in the old portion 255 of digital items as actual data.

[0110] In general, the distribution of responses represents the distribution of predicted indications of positive and negative user responses in the old portion 255 of digital items. However, it should be understood that in other non-limiting embodiments of the present technology, the distribution of responses may take into account only actual indications of user responses provided for some items from the old portion 255 of digital items by users of the online recommendation platform 260.

[0111] Thus, in some non-limiting embodiments of the present technology, the server 250 may determine the response distribution by determining specific parameters thereof, such as, but not limited to, at least one of the following: mathematical expectation, standard deviation, skewness coefficient, and slope of the response distribution. In other non-limiting embodiments of the present technology, the server 250 may determine the response distribution using a graphical representation thereof.

[0112] As a result, the computational resources required on the server 250 to determine the indications of the synthesized user responses can be reduced and new digital elements can be more efficiently presented to users of the online recommendation platform 260.

[0113] The following describes how the server 250 may train an algorithm for generating a synthesized response given a distribution of responses according to some non-limiting embodiments of the present technology.

Algorithm for generating synthesized user responses. Training stage

[0114] According to some non-limiting embodiments of the present technology, the server 250 may train the synthesized user response generation algorithm to determine the synthesized user response indications for each element of the new portion 275 of digital elements based on the old portion 255 of digital elements.

[0115] In particular, according to some non-limiting embodiments of the present technology, the server 250 may train the algorithm for generating synthesized user responses so that, given a particular approach to selecting old digital items from the old portion 255 of digital items, the algorithm for generating synthesized user responses is capable of: (a) determining, for an old digital item, such as an item from the set 257 of old digital items associated with the provider 247 of old items, a value of the probability of the old digital item receiving, for example, a positive user response; and (b) assigning, based on this probability value, an indication of a synthesized user response for the old digital item so that the indications of the synthesized user responses are distributed in the set 257 of old digital items according to a distribution of responses as described above.

[0116] For example, the server 250 may assign an indication of a synthesized user response to the old digital item based on a probability value associated with it and specific probability thresholds. In particular, if the probability value is greater than a predetermined upper probability threshold (such as 0.75, 0.80, or 0.9), the server 250 may assign an indication of a positive synthesized user response to the old digital item, and if the probability value is less than a predetermined lower probability threshold (such as 0.1, 0.25, or 0.4), the server 250 may assign an indication of a negative synthesized user response to the old digital item. In addition, if the probability value is equal to a predetermined upper probability threshold (such as 0.75, 0.80, or 0.9), the server 250 may assign an indication to the old digital item that the old digital item has been evaluated.

[0117] By training the algorithm for generating synthesized user responses as described above, the server 250 can determine its specific parameters of the usage stage as described below. The server 250 can then apply the algorithm for generating synthesized user responses with its parameters of the usage stage thus determined in relation to the new part 275 of the digital elements.

[0118] According to some non-limiting embodiments of the present technology, since the features of the item for the new portion 275 of digital items may not be available, the server 250 may determine other features for the old portion 255 of digital items in order to train the algorithm for generating synthesized user responses. In particular, some non-limiting embodiments of the present technology are based on the assumption that the similarity of the old digital item from the old portion 255 of digital items and the new digital item from the new portion 275 of digital items can be determined not only based on the features of the item, but also based on the similarity of the suppliers of the old and new items, as well as specific features indicating the popularity of the item, which is associated with the new portion 275 of digital items and can be obtained based on data outside the online recommendation platform 260.

[0119] Thus, in some non-limiting embodiments of the present technology, the server 250 may represent the probability value as comprising (a) a first probability value determined based on a similarity score between the old item provider associated with the old digital item and the plurality 245 old item providers, and/or (b) a second probability value determined based on a popularity parameter value associated with the old digital item.

[0120] The following describes a method for determining by the server 250 the first and second probability values taking into account the distribution of responses for predicted indications of user responses in the old portion 255 of the digital elements according to some non-limiting embodiments of the present technology.

[0121] According to some non-limiting embodiments of the present technology, the server 250 (see FIG. 2) may determine a first value of the probability of receiving an indication of a positive user response by an old digital item based on an assessment of the similarity of the old item provider 247 that uploaded the old digital item to the online recommendation platform 260 and the plurality 245 of old item providers according to the following formula:

, (2)

where artistScore is the similarity score of the supplier of 247 old items with the set of 245 suppliers of old items, a and b are coefficients.

[0122] According to some non-limiting embodiments of the present technology, the coefficients a and b are parameters from among the parameters of the use stage for the algorithm for generating synthesized user responses. As described in more detail below, the server 250 can determine these coefficients using the distribution of responses as actual data. In other words, the server 250 can determine the coefficients a and b so that the indications of the synthesized user responses determined for the items from the old portion 255 of the digital items based on at least their first probability values are distributed in the old portion 255 of the digital items according to the distribution of responses for the predicted indications of the user responses in the old portion 255 of the digital items determined by the ML evaluation model, as described above.

[0123] According to some non-limiting embodiments of the present technology, the server 250 may determine a similarity score for the old item provider 247 based on the values of a similarity parameter of each provider of the plurality of 245 old item providers with the old item provider 247. In some non-limiting embodiments of the present technology, such a similarity parameter may comprise a degree of similarity between the old item provider 247 and another provider of the plurality of 245 old item providers.

[0124] According to some non-limiting embodiments of the present technology, the similarity of the legacy item provider 247 and another provider of the plurality of legacy item providers may be determined based on at least one item provider feature associated therewith. For example, in embodiments where the online recommendation platform 260 is an audio streaming platform and the legacy item provider 247 is a musician, the item provider features of the legacy item provider 247 may include, among other things: (a) the genre in which the musician works; (b) the number of albums released by the musician; (c) the language used by the musician; (d) a period of popularity associated with the musician; (e) the number of streams for the musician's songs on the online recommendation platform 260; (f) the age group of the musician's target audience; (g) the number of users of the online recommendation platform 260 who have left at least one indication of positive or negative user feedback for the musician's songs; (z) at least one user attribute of a user of the online recommendation platform 260 who has left at least one indication of a positive or negative user response under the musician’s songs, etc.

[0125] Furthermore, in some non-limiting embodiments of the present technology, the server 250 may determine the similarity of the old item provider 247 and another provider from the plurality of 245 old item providers by: (a) generating, for each provider from the plurality of 245 old item providers, a feature vector of the item provider; (b) mapping the feature vectors to a multi-dimensional vector space; and (c) determining, in the multi-dimensional vector space, a distance value between the feature vectors of the item providers representing the old item provider 247 and the other provider from the plurality of 245 old item providers. Thus, in the example where the old item provider is a musician, the feature vectors of the item providers of musicians who use the same language or languages of the same language group are located closer to each other in the multi-dimensional vector space than the vectors of musicians who use different languages. In another example, the feature vectors of item suppliers of musicians whose songs received indications of positive user feedback from users of the same age group are located closer to each other in the multidimensional vector space than the vectors of musicians whose songs were rated by users of different age groups.

[0126] Based on the distance values between the old item supplier 247 and each other supplier of the set 245 old item suppliers, the server 250 may rank each supplier of the set 245 old item suppliers relative to the old item supplier 247 and thereby determine the similarity parameter values between it and the old item supplier 247.

[0127] In addition, according to some non-limiting embodiments of the present technology, to determine the similarity score of the old item supplier 247, the server 250 may form a graph structure based on the similarity parameter values. Fig. 5 shows a schematic illustration of a similarity graph 500 of item suppliers according to some non-limiting embodiments of the present technology.

[0128] In some non-limiting embodiments of the present technology, the server 250 may form a similarity graph 500 of item suppliers such that (a) a node thereof, such as a first node 502, represents a supplier of old items from the plurality 245 suppliers of old items, and (b) an edge 506 connecting the first node 502 and the second node 504 indicates a non-zero value of a similarity parameter between them, determined as described above. In some non-limiting embodiments of the present technology, the similarity graph 500 of item suppliers may be a directed graph, the edges of which have an orientation indicating with respect to which supplier of old items the value of the similarity parameter is determined.

[0129] In addition, in some non-limiting embodiments of the present technology, the server 250 may, based on the values of the similarity parameter for the plurality 245 of old item suppliers, determine weight values of transitions between nodes of the similarity graph 500 of item suppliers. In some non-limiting embodiments of the present technology, the server 250 may determine the weight value of a transition for a pair of nodes of the similarity graph 500 of item suppliers as inversely proportional to the values of the similarity parameter between the old item suppliers located on this transition. In cases where a pair of nodes are connected by an edge, for example, the first and second nodes 502, 504 connected by an edge 506, the server 250 can determine the weight value of the transition from the second node 504 to the first node 502 as a value inversely proportional to the value of the similarity parameter between the suppliers of old elements associated with the first and second nodes 502, 504. In another example, where a pair of nodes does not have a direct connection through one edge, for example, the third node 508 and the fourth node 510, but there is a transition through the fifth node 512, the server 250 can determine the weight value of the transition from the third node 508 to the fourth node 510 as a value determined by the formula:

, (3)

Where sim(A,C) - the value of the similarity parameter for the third and fifth vertices is 508, 512,sim(C, B) - the value of the similarity parameter for the fifth and fourth vertices 512, 510,R- a predetermined coefficient, for example, equal to 10.

[0130] In addition, the server 250 may determine the similarity score associated with the old item provider 247 as the maximum total weight value for moving from a node associated with the old item provider 247 to a node associated with a provider from the set of 514 reference old item providers (thickened lines in Fig. 5).

[0131] In some non-limiting embodiments of the present technology, each supplier from the set 514 reference suppliers of old items may be randomly selected from the plurality 245 suppliers of old items. In other non-limiting embodiments of the present technology, the set 514 reference suppliers of old items may be selected from the plurality 245 suppliers of old items as a predetermined number of the most popular suppliers of old items among the users of the online recommendation platform 260. In some non-limiting embodiments of the present technology, the server 250 may determine the number of the most popular suppliers of old items from the plurality 245 suppliers of old items by analyzing the history of user actions of the users of the online recommendation platform 260 with the old portion 255 of digital items. For example, the server 250 may determine the number of the most popular suppliers of old items as the suppliers of items whose old digital items satisfy at least one of the following conditions: (a) they are searched for the most on the online recommendation platform 260; (b) they are rated the most on the online recommendation platform 260; (c) they have received the greatest number of indications of positive user feedback on the online recommendation platform 260, etc. In another example, the server 250 may (a) determine a number of the best suppliers of old items by offering, for example, using a GUI element in the GUI interface 300 of the online recommendation platform 260, its users to indicate several, for example, 3, 4 or 10, suppliers from the set 245 suppliers of old items that are most interesting to users; and (b) based on such indications, determine a number of the most popular suppliers of old items, comprising a set 514 reference suppliers of old items.

[0132] In addition, as described above, based on the thus determined similarity score of the supplier 247 of old items, the server 250 can determine a first value of the probability of receiving an indication of a positive user response by an old digital item from the plurality 257 of old digital items.

[0133] As described above, the server 250 may also determine a second probability value of receiving an indication of a positive user response by an old digital item from the plurality 257 old digital items based on a popularity parameter value associated with the old digital item in a certain area, such as a city, country, region, etc. In some non-limiting embodiments of the present technology, the server 250 may determine the second probability value according to the following equation:

, (4)

where Popularity is the popularity value of an old digital element in a given area, maxPopularity is the popularity value of the most popular old digital element in a given area from the set of 257 old digital elements of a supplier of 247 old elements, minPopularity is the popularity value of the least popular old digital element in a given area from the set of 257 old digital elements of a supplier of 247 old elements, c and d are coefficients.

[0134] Like the coefficients a and b from equation (2), the coefficients c and d are parameters from among the parameters of the use stage for the algorithm for generating synthesized user responses. Thus, as described in more detail below, the server 250 can determine these coefficients so that the indications of the synthesized user responses determined for the elements from the old portion 255 of the digital elements based on at least their second probability values are distributed in the old portion 255 of the digital elements according to the actual distribution of responses for the predicted indications of the user responses in the old portion 255 of the digital elements determined by the ML evaluation model, as described above.

[0135] In the context of the present description, in contrast to the popularity of an item determined based on indications of user responses left on the online recommendation platform 260, as described above, the popularity parameter is a popularity metric associated with an old digital item and determined based on indications of user actions with this old digital item, performed outside the online recommendation platform 260, using other sources available in this area, such as radio stations, television channels, music clubs, live concerts, etc., or other online sources. In particular, in the example described above, where the online recommendation platform 260 is an audio streaming platform and the old digital item is an audio stream, the popularity parameter may, among other things, include at least one of the following: (a) the number of streams of the audio stream on the media resource; (b) the number of requests to play the audio stream on the media resource; (c) the position of the audio stream in the hit parade; (d) the number of albums sold containing the audio stream, etc.

[0136] In addition, as described above, based on the first and/or second probability values, the server 250 may determine a probability value of the old digital element receiving an indication of a positive user response. In some non-limiting embodiments of the present technology, the probability value may be determined based on only the first probability value. In other non-limiting embodiments of the present technology, the server 250 may determine the probability value based on only the second probability value. In other non-limiting embodiments of the present technology, the server 250 may determine the probability value based on a combination of, for example, the sum of the first and second probability values.

[0137] In addition, as described above, the server 250 may train the algorithm for generating synthesized user responses to determine its usage stage parameters, such as coefficients a, b, c and d , for subsequent use in determining the indications of a synthesized user response for each element of the new portion 275 of digital elements.

[0138] To this end, according to some non-limiting embodiments of the present technology, the server 250 may: (a) initialize the parameters of the use stage for the algorithm for generating synthesized user responses, such as randomly; (b) iteratively execute the algorithm for generating synthesized user responses, selecting different elements from the old portion 255 of digital elements; (c) after a predetermined number of iterations of executing the algorithm for generating synthesized user responses, determine a training distribution of synthesized responses for the thus determined indications of synthesized user responses in the old portion 255 of digital elements; and (d) adjust the parameters of the use stage for the algorithm for generating synthesized user responses such that the training distribution of synthesized responses matches with a desired accuracy the distribution of responses for the predicted indications of user responses determined by the ML scoring model.

[0139] In some non-limiting embodiments of the present technology, the server 250 may use all of the elements from the old portion 255 of the digital elements to train the algorithm for generating the synthesized user responses. In other non-limiting embodiments of the present technology, the server 250 may determine a training set of old digital elements that includes a specific portion of the old portion 255 of the digital elements, such as sets of old digital elements of specific suppliers from the set 245 of old element suppliers, which the server 250 may then use to train the algorithm for generating the synthesized user responses. There is no limitation on the method for the server 250 to select a predetermined number of iterations of executing the algorithm for generating the synthesized user responses, which may include selecting this number based on the number of digital elements used to train the algorithm for generating the synthesized user responses. For example, the predetermined number may correspond to, among other things, 100, 1000, or even tens or hundreds of thousands of iterations.

[0140] There is also no limitation on the method by which the server 250 selects an old digital item from the plurality 257 old digital items, which in some non-limiting embodiments of the present technology may include selecting an old digital item based on the value of its popularity parameter in the given area, determined as described above.

[0141] For example, in some non-limiting embodiments of the present technology, the server 250 may select an old digital item from the plurality of 257 old digital items to determine its probability value using, among other things, one of the following approaches: (a) randomly; (b) using uniform selection probabilities, i.e., with an equal chance of selecting an old digital item relative to other items from the plurality of 257 old digital items regardless of the popularity parameter value; (c) using a selection probability value proportional to the popularity parameter value, i.e., with a greater chance of selecting an old digital item with a greater value of its popularity parameter; and (d) using a selection probability value proportional to the square root of the popularity parameter value, i.e., with a greater chance of selecting an old digital item with a greater value of the square root of its popularity parameter value. Other selection approaches are also possible.

[0142] For example, in some non-limiting embodiments of the present technology, the server 250 may adjust the parameters of the use step for the algorithm for generating synthesized user responses so as to minimize differences between the parameters of the training distribution of the synthesized responses and the response distribution. In another example, the server 250 may adjust the parameters of the use step for the algorithm for generating synthesized user responses so that the graphical representations of the training distribution of the synthesized responses and the response distribution match each other. It should be noted that various approaches may be used to determine the parameters of the use step for the algorithm for generating synthesized user responses in order to ensure that at least one of these conditions is met, including, for example, optimization algorithms, which may include, among other things, algorithms used for differentiable objective functions (such as a gradient descent optimization algorithm, a Newton-based optimization algorithm, etc.) and non-differentiable objective functions (such as direct optimization algorithms, stochastic optimization algorithms, population optimization algorithms, etc.). Also, without going beyond the boundaries of the present technology, embodiments of the invention are possible, where the parameters of the use stage for the algorithm for generating synthesized user responses are selected empirically to ensure compliance with at least one of the above-mentioned conditions.

[0143] Thus, in a non-limiting example, server 250 may determine the parameters of the use stage for the algorithm for generating synthesized user responses such that equations (2) and (4) are represented as follows:

(5)

[0144] The thus trained algorithm for generating synthesized user responses, the server 250 can be used to determine indications of synthesized user responses for each element of the new portion 275 of digital elements, as described below.

Algorithm for generating synthesized user responses. Use stage

[0145] According to some non-limiting embodiments of the present technology, in order to use the trained algorithm for generating synthetic user responses for the purpose of determining an indication of a synthetic user response for a new digital item (that does not have an indication of a user response), such as an item from the plurality 277 of new digital items associated with the provider 267 of new items, the server 250 may (a) determine for the new digital item a first and/or second value of a probability of a usage step for the new digital item to receive an indication of a positive user response according to equations (5); (b) based on the first and/or second values of a probability of a usage step, determine a probability value of a usage step for the new digital item to receive an indication of a positive user response; and (c) based on the probability value of the usage step, determine an indication of a negative or positive synthetic user response, as described above in connection with training the algorithm for generating synthetic user responses.

[0146] Similar to what is described above, the server 250 may determine a first value of the probability of the usage stage for the new digital item to receive an indication of a positive user response based on the similarity assessment of the usage stage for the new item provider 267. To this end, the server 250 may (a) determine the values of the similarity parameter for the plurality 265 new item providers and (b) generate a graph of similarity of the item providers of the usage stage similar to the graph 500 of similarity of the item providers as described above. However, since the features of the item provider from the plurality 265 new item providers may also be unavailable, in some non-limiting embodiments of the present technology, the server 250 may obtain the values of the similarity parameter in a different manner.

[0147] In particular, in some non-limiting embodiments of the present technology, the server 250 may obtain for each supplier of the plurality 265 suppliers of new items, such as the new supplier 267 of items, values of the similarity parameter relative to the suppliers of old items. To this end, in some non-limiting embodiments of the present technology, the server 250 may determine values of the similarity parameter associated with the supplier 267 of new items based on ratings provided by human raters.

[0148] Fig. 6 is a schematic diagram of a human evaluator 600 determining similarity values between a set 265 suppliers of new items and suppliers from a set 245 suppliers of old items according to some non-limiting embodiments of the present technology.

[0149] It is obvious that the human evaluator 600 can access the data of the sets 245, 265 of suppliers of old and new items using an electronic device implemented similarly to the electronic device 210 of the user 220 and connected to the server 250 via the communication network 240, as described above. Moreover, in some non-limiting embodiments of the present technology, the human evaluator 600 can include a plurality of different human evaluators.

[0150] According to some non-limiting embodiments of the present technology, the server 250 may request, for example by transmitting an indication of a ranking task, the human evaluator 600 to directly provide a degree of similarity between the new item provider 267 and, for example, a predetermined number (such as 3, 5, 10 or 20) of old item providers from the set 245 of old item providers. In response, the human evaluator 600, based on its perception of the digital elements of the new item provider 267 and the set 245 of old item providers, may determine a set 602 of similar old item providers. Accordingly, a provider from the set 602 of similar old item providers may contain a label 604 generated by the evaluator indicating a degree of similarity related to the new item provider 267. As described above, the server 250 may use the similarity degrees determined in this manner as values of the similarity parameters between each of the plurality 265 new item providers and the old item providers from the plurality 245 old item providers.

[0151] Similar to the above, the server 250 may form a similarity graph of the use stage element suppliers based on (a) the set 265 new element suppliers, (b) old element suppliers that have a non-zero similarity parameter value with a supplier from the set 265 new element suppliers, such as the set 602 similar old element suppliers associated with the new element supplier 267, and (c) the similarity parameter values between suppliers in (a) and (b).

[0152] In addition, the server 250 may determine in the similarity graph of the suppliers of the use stage elements for the plurality 265 suppliers of new elements a set of new reference suppliers of elements in the same way as described above for the similarity graph of the suppliers of elements 500. And finally, as described above, the server 250 may determine the similarity score of the supplier 267 of the new elements as the maximum total weight of the transition in the similarity graph of the suppliers of the use stage elements between the nodes representing the supplier 267 of the new elements with a supplier from the set of reference suppliers of new elements. Accordingly, by applying equations (5) with respect to the similarity score of the use stage determined in this way, the server 250 may determine a first value of the probability of the use stage for the new digital element to receive a positive user response.

[0153] In addition, as described above with respect to the old digital item, the server 250 may (a) determine for the new digital item a usage stage popularity parameter value and (b) by applying equations (5) with respect to the usage stage popularity parameter value, determine a second usage stage probability value for the new digital item to receive an indication of a positive user response.

[0154] In addition, as also described above, based on the first and/or second values of the probability of the stage of use, the server 250 can determine a value of the probability of the stage of use for receiving an indication of a positive user response by the new digital item, based on which the server 250 can determine an indication of a synthesized user response for the new digital item.

[0155] Accordingly, by iteratively executing the algorithm for generating synthesized user responses for a predetermined number of iterations and selecting at each iteration a different new digital element from the new portion 275 of digital elements, as described above in relation to the old portion 255 of digital elements, the server 250 can generate an indication of a synthesized user response for each element from the new portion 275 of digital elements.

[0156] In other non-limiting embodiments of the present technology, the server 250 may determine an indication of a synthesized user response for each item from the new portion 275 of digital items in a different manner. In particular, in these embodiments of the invention, the server 250 may: (a) determine for the provider 267 of new items a reference provider of old items from the plurality 245 providers of old items; (b) determine a local distribution of responses for indications of user responses in the plurality of old digital items associated with the reference provider of old items; and (c) assign for the plurality 277 of new digital items associated with the provider 267 of new items indications of synthesized user responses according to the local distribution of responses.

[0157] There is no limitation on the method for the server 250 to determine the reference supplier of old items for the supplier 267 of new items. In some non-limiting embodiments of the present technology, the server 250 may determine the reference supplier of old items as the most similar to the supplier 267 of new items from the plurality 245 suppliers of old items. For example, in these embodiments of the invention, the server 250 may determine the most similar supplier to the supplier 267 of new items from the plurality 245 suppliers of old items based on the labels of the human raters, as described above with reference to FIG. 6. For example, the server 250 may determine the reference supplier of old items for the supplier 267 of new items as the supplier with the highest rank from the set 602 of similar suppliers of old items.

[0158] In other non-limiting embodiments of the present technology, the server 250 may determine the reference supplier of old items for the supplier of new items 267 as the most popular supplier from the plurality 245 suppliers of old items, as described above with reference to FIG. 5.

[0159] In addition, in some non-limiting embodiments of the present technology, the server 250 may determine a local distribution of responses only for actual user response indications in the plurality of old digital items associated with the reference provider of old items. In other non-limiting embodiments of the present technology, the server 250 may determine a local distribution of responses for predicted user response indications in the plurality of old digital items determined by the ML scoring model, as described above.

[0160] There is no limitation on the method for determining the local distribution of responses by the server 250. It may include any of the approaches described above in relation to the distribution of responses.

[0161] Thus, having determined a local distribution of indications of user responses in the set of old digital elements of the reference supplier of old elements, the server 250 can assign, according to such distribution, an indication of a synthesized user response for each element of the set 277 of new digital elements.

[0162] For example, it can be assumed that the set of old digital elements of the reference supplier of old elements contains five old digital elements with indications of a positive or negative user response distributed therein as follows: [100/25; 108000/580; 9995874/36954; 135001/398; 127/24]. Thus, using this local distribution of responses, the server 250 can assign indications of synthesized user responses to the set 277 of new digital elements so as to minimize the differences between parameters, such as the mathematical expectation and the standard deviation, for the local distribution of responses and the thus formed distribution of indications of a synthesized user response in the set 277 of new digital elements (described above). Thus, if the set 277 of new digital elements also contains five digital elements, then the server 250 can determine for it indications of synthesized user responses that coincide with the local distribution of responses from the example described above. As a result, it can be said that the server 250 simulates the local distribution of responses for indications of user responses in the set of old digital elements in order to generate indications of synthesized user responses for each element of the set 277 of new digital elements.

[0166] Thus, having determined an indication of a synthesized user response for each item from the new portion 275 of digital items, the server 250 can use the new portion 275 of digital items to train the MLA algorithm 280 to determine a set 225 of recommended digital items, as described above. In particular, the server 250 can form a new training data set that contains a plurality of new training digital objects, each of which contains at least (a) an indication of a new digital item from the new portion 275 of digital items and (b) at least one indication of a synthesized user response determined as described above. In addition, the server 250, based on the plurality of new training digital objects, can train the MLA algorithm 280 to determine a probability value of a positive action by the user 220 with the new digital item and, based on this probability value, determine a set of recommended digital items from among other items from the old and new portions 255, 275 of digital items.

[0164] In non-limiting embodiments of the present technology, the MLA algorithm 280 may be based on neural networks (NN), convolutional NN (CNN), transformer-based networks, decision tree models, a gradient boosted decision tree MLA algorithm, an association rule learning MLA algorithm, a deep learning MLA algorithm, an inductive logic programming MLA algorithm, a support vector machine MLA algorithm, a clustering MLA algorithm, Bayesian networks, a reinforcement learning MLA algorithm, a representation learning MLA algorithm, a similarity and metrics learning MLA algorithm, a sparse dictionary learning MLA algorithm, a genetic algorithm MLA algorithm, etc. To train algorithm 280, server 250 may use a supervised learning approach without going beyond the boundaries of the current technology.

[0165] Below, with reference to Fig. 7, an example of a machine learning model on the basis of which the MLA algorithm 280 can be implemented, as well as a process for training it, according to at least some non-limiting embodiments of the present technology, is described.

Machine learning model architecture

[0166] Fig. 7 is a diagram of a decision tree model 700 that can be used with at least some non-limiting embodiments of the present technology. The decision tree model 700 is intended to illustrate a general-purpose decision tree model that can be modified to meet the requirements of a particular prediction model, such as the MLA algorithm 280. For example, such modifications can include, among other things, adding or removing one or more levels of the tree, adding or removing nodes (i.e., features and associated branches), adding or removing branches connecting nodes and/or leaves of the tree.

[0167] The decision tree model 700 may be a trained tree model. In some non-limiting embodiments of the present technology, the generated decision tree model 700 may be updated and/or modified, for example, to improve the accuracy of the machine learning model and/or to expand the scope of application of the machine learning model. In particular, in some embodiments of the invention, the decision tree model 700 may be used, among other things, to determine the values of the probability of a positive action of users of the online recommender system 260, such as the user 220, with each element of the plurality 235 digital elements posted on the online recommender platform 260. Other areas of application of the decision tree model 700 are also possible without going beyond the scope of the present technology.

[0168] The decision tree model 700 comprises a first node 702 associated with a first feature f1. The first node 702 forms a first level of the decision tree model 700. The first node 702 is connected by branches to a second node 704 and to a third node 706. The second node 704 and the third node 706 are associated with a second feature f2. The second node 704 and the third node 706 form a second level of the decision tree model 700. In some non-limiting embodiments of the present technology, the first feature f1 (and the branching for the first feature f1) is selected from a set of features for placement at the first level of the decision tree model 700 based on a plurality of training digital objects. For example, this feature may characterize the digital item and/or the user who interacted with it and/or the method of the user interacting with the item (described in more detail below).

[0169] The first feature f1 is selected such that for a digital object, the value of the parameter associated with the first feature f1 determines whether the digital object should be associated with the second node 704 or with the third node 706. For example, if the value is less than the value f1, the digital object is associated with the second node 704. In another example, if the value is greater than the value f1, the digital object is associated with the third node 706.

[0170] In turn, the second node 704 is connected with the fourth node 708, connected with the third feature f3, and with the fifth node 710, connected with the third feature f3. The third node 706 is connected with the sixth node 712, connected with the third feature f3, and with the seventh node 714, connected with the third feature f3. The fourth node 708, the fifth node 710, the sixth node 712 and the seventh node 714 form the third level of the decision tree model 700. As described above with respect to the first node 702, for a digital object, the value of the parameter associated with the second feature f2 determines whether this digital object should be associated with the fourth node 708 or with the fifth node 710 (if the digital object is associated with the second node 704) or with the sixth node 712 or with the seventh node 714 (if the digital object is associated with the third node 706).

[0171] In turn, all nodes from the fourth node 708, the fifth node 710, the sixth node 712 and the seventh node 714 are associated with sets of predicted parameters. In the example shown in Fig. 7, the sets of predicted parameters comprise a first set 720, a second set 722, a third set 724 and a fourth set 726. Each set from among the sets of predicted parameters comprises three target values C1, C2 and C3, for example, provided by human assessors (as described below).

[0172] As will be understood by one skilled in the art, the decision tree model 700 illustrates an embodiment of the invention, in which a particular level of the decision tree model 700 is associated with one feature. In the example shown in Fig. 7, the first level contains the first node 702 and is associated with the first feature f1, the second level contains the second node 704 and the third node 706 and is associated with the second feature f2, the third level contains the fourth node 708, the fifth node 710, the sixth node 712 and the seventh node 714 and is associated with the third feature f3.

[0173] In other words, in the embodiment of the invention shown in Fig. 7, the first level is associated with the first feature f1, the second level is associated with the second feature f2, and the third level is associated with the third feature f3. However, other embodiments of the invention are also possible. In particular, an alternative embodiment of the invention is possible, in which the generated tree model may contain different features for a certain level of the tree model. For example, the first level of such a tree model may contain a first node associated with the first feature f1, the second level may contain a second node associated with the second feature f2, and a third node associated with the third feature f3. As should be clear to a person skilled in the art, other options for associating features with a level are also possible without going beyond the scope of the present technology.

[0174] There are no restrictions on the selection of features, such as the first feature f1, the second feature f2 and the third feature f3, for forming the decision tree model 700. In some non-limiting embodiments of the present technology, the selection of features and, therefore, branches between nodes of the decision tree model 700, may include optimizing a metric indicating the quality of a feature for dividing a plurality of training digital objects into branches of the decision tree model 700 during the training process, as each object of the plurality of training digital objects is fed to the decision tree model 700. In some non-limiting embodiments of the present technology, such a metric may include an Estimate of Positive Correctness metric, a Gini Impurity metric, an Information Gain metric, and the like. In certain non-limiting embodiments of the present technology, the features may be selected based on maximizing the prediction quality parameter of the proto-trees that comprise the decision tree model 700, such as described in patent application US2019164084A1, “Method of and system for generating prediction quality parameter for a prediction model executed in a machine learning algorithm” (published May 30, 2019), the contents of which are incorporated herein by reference in their entirety.

[0175] Furthermore, according to some non-limiting embodiments of the present technology, the MLA algorithm 280 may comprise a single instance of the decision tree model 700. In this case, the MLA algorithm may be referred to as a single decision tree. In other non-limiting embodiments of the present technology, the MLA algorithm 280 may comprise an ensemble of decision trees implemented similarly to the decision tree model 700. Then, the MLA algorithm 280 may be referred to as a forest of decision trees. In this case, the final prediction of the ensemble of decision trees is based on the prediction results of each decision tree in the ensemble. There are no limitations on how the ensemble of decision trees is constructed, for example, it may include using a boosting-based approach, such as a gradient boosting-based approach.

[0176] In general, boosting is a method for constructing an ensemble of decision trees where the trees are constructed in stages (as opposed to forming a random forest, where the trees may be formed in parallel) such that the difference between the target values and the predicted output of the ensemble of decision trees is minimized at each iteration as each new decision tree is formed. For example, the difference between the target values and the prediction output of the ensemble of decision trees may be expressed by a loss function, such as the cross-entropy loss function.

[0177] It should be understood that in non-limiting embodiments of the present technology, other embodiments of the loss function are also possible, for example, a mean square error loss function, a Huber loss function, a piecewise linear loss function, etc.

[0178] In other words, using a boosting approach, the MLA algorithm 280 uses decision trees as weak learners, which the MLA algorithm 280 sequentially combines in such a way that each new learner matches the residuals of the previous iteration, improving the combined result of the entire ensemble of decision trees. In other words, each decision tree is built on the same set of training digital objects, and training digital objects for which the first decision tree made "errors" in predicting are given priority when building the second decision tree, and so on. The MLA algorithm 280 then combines the results of each iteration to form a strong learner.

[0179] In certain non-limiting embodiments of the present technology, the MLA algorithm 280 may comprise an ensemble of CatBoost decision trees, such as those described in “CatBoost: gradient boosting with categorical features support” (Dorogush et al., published by Yandex Inc.), the contents of which are incorporated herein by reference in their entirety.

[0180] In general, it can be said that the processor of the server 250 must perform two processes related to the MLA algorithm 280. The first process of these two processes is a training process, where the processor 110, for example, of the server 250, is capable of training the MLA algorithm 280 based on a plurality of training digital objects (also called a training data set) to determine the values of the probability of a positive action of the user 220 with each element of the plurality 235 of digital elements.

[0181] The second process is a process of the use stage, where the processor of the server 250 executes the MLA algorithm 280 trained in this way to determine the values of the probability of a positive action of the user 220 with each element of the plurality 235 digital elements according to non-limiting embodiments of the present technology. However, in some non-limiting embodiments of the present technology, the MLA algorithm may be trained on a third-party server, and the server 250 may execute the MLA algorithm 280.

[0182] It should be understood that in some non-limiting embodiments of the present technology, the MLA algorithm 280 may comprise several machine learning models capable of determining the values of the probability of a positive action of the user 220 with the plurality 235 digital items. In particular, in addition to the decision tree model 700 described above, the MLA algorithm 280 may comprise additional machine learning models. In various non-limiting embodiments of the present technology, the additional machine learning model may, for example, comprise an advanced structured semantic machine learning model. In various non-limiting embodiments of the present technology, the advanced structured semantic machine learning model may be trained by the server 250 based on the same or a different plurality of training digital objects used to train the decision tree model 700.

[0183] After training the MLA algorithm 280 based on the new training data set, the server 250 may present at least some items from the new portion 275 of digital items to the user 220 as part of the set 225 of recommended digital items.

[0184] After presenting a new digital item from the new portion 275 of digital items to the user 220 by assigning an indication of a synthesized user response to it, as described above, the server 250 may receive an indication of a user response from the user 220. If the indication of a user response from the user 220 differs from the indication of a synthesized user response, the server 250 may replace the latter with the former. In other non-limiting embodiments of the present technology, the server 250 may add an indication of a user response from the user 220 to the indication of a synthesized user response thus generated.

The first way

[0185] The architecture and examples described above allow for a method of generating a training data set for training an MLA algorithm, such as the MLA algorithm 280, to generate recommendations of digital items for users of the online recommendation platform 260. Fig. 8 shows a flow chart of a first method 800 according to some non-limiting embodiments of the present technology. The method 800 can be performed by the server 250.

[0186] As described above, the server 250 may generate a training data set for training the MLA algorithm by generating indications of a synthesized user response for new digital items that do not have indications of actual user responses from users of the online recommendation platform 260.

Step 802: receiving by the server indications of the old portion of the digital elements for the purpose of forming a plurality of training digital objects for evaluation, each of which contains (a) the old digital element from the old portion of the digital elements and (b) an indication of the user response received by the old digital element.

[0187] The first method 800 begins with step 802, in which the server 250 is capable of training an ML scoring model to determine indications of a synthesized user response for digital items, such as the new portion 275 of the digital items. As described above, to this end, the server 250 may: (a) train an ML scoring model to determine predicted indications of user responses for items from the old portion 255 of the digital items that do not have indications of user responses; (b) determine a distribution of responses for the predicted indications of user responses in the old portion of the digital items; and (c) based on the distribution of responses, determine the parameters of the use stage for the algorithm for generating synthesized responses, including the coefficients of equations (2) and (4), for subsequent use in determining indications of a synthesized user response for each item from the new portion 275 of the digital items.

[0188] As described above with reference to Figs. 4A and 4B, server 250 may train an ML scoring model to determine predicted user response indications, such as a predicted indication 408 for user 220, for another item from old portion 255 of digital items based on actual user response indications, such as an actual indication 406 provided by user 220 for an item from old portion 255 of digital items.

[0189] As described above, in some non-limiting embodiments of the present technology, the ML rating model comprises a matrix factorization ML model, for training which, as described with reference to Fig. 4A, the server 250 can generate an input matrix 402 comprising a plurality of training digital objects for rating. In particular, the input matrix 402 has a size of m*n , where m is the number of users of the online recommendation platform 260, n is the number of old digital items in the old portion 255 of digital items. Thus, an element of the input matrix 402 representing a training digital object for rating from the plurality of training digital objects for rating comprises: (a) an actual indication 406 of a user response provided by the user 220 for an old digital item from the old portion 255 of digital items; (b) element features for the old digital item; and (c) user attributes of the user 220 who generated the actual indication 406 under the old digital element.

[0190] The first method 800 then continues at step 804.

Step 804: inputting a plurality of digital objects for evaluation into an ML evaluation model by the server, and thereby training the ML evaluation model to determine a predicted user response indication for the digital item using the response distribution for user response indications in the old portion of the digital items as actual data.

[0191] In step 804, the server 250, based on the plurality of training digital objects for evaluation represented by the input matrix 402, may train an ML evaluation model to determine an indication of a synthesized user response for a new digital item using the distribution of responses as factual data.

[0192] To this end, as described in detail with reference to Figs. 4A and 4B, the server 250 may first train an ML scoring model to determine predicted indications of user responses by determining, based on the input matrix 402, matrices P and Q of vector representations of users and items such that the elements of the output matrix 404, defined as PQ ^T , provide a close approximation of the elements of the input matrix 402. The output matrix 404 determined in this way: (a) for an item from the old portion 255 of digital items that previously did not have an indication of a user response, contains a predicted indication 408 of a user response; and (b) for items from the old portion 255 of digital items that had actual indications of user responses, such as the old digital item associated with the actual indication 406 of a user response, approximates such indications with the required accuracy.

[0193] In addition, according to some non-limiting embodiments of the present technology, the server 250 may determine a response distribution for the predicted user response indications determined by the ML scoring model in the old portion 255 of the digital items. For example, as described above, the server 250 may determine (a) response distribution parameters, such as, among other things, the mathematical expectation, the standard deviation, the skewness coefficient, and the slope of the response distribution, and/or (b) a graphical representation of the distribution.

[0194] In addition, using the distribution of responses, the server 250 can determine the parameters of the use stage for the algorithm for generating synthesized user responses for the purpose of applying it to the new part 275 of the digital elements. In particular, according to some non-limiting embodiments of the present technology, the server 250 can determine the parameters of the use stage for the algorithm for generating synthesized user responses, such as the coefficients of equations (2) and (4), so that the indications of the synthesized user responses determined in this way are distributed in the new part 275 of the digital elements in accordance with the distribution of responses.

[0195] For this purpose, according to some non-limiting embodiments of the present technology, the server 250 may train an algorithm for generating synthesized user responses based on the old portion 255 of digital items. In particular, the server 250 may train the algorithm for generating synthesized user responses so that, given a particular approach to selecting old digital items from the old portion 255 of digital items, the algorithm for generating synthesized user responses is capable of: (a) determining, for an old digital item, such as an item from the set 257 of old digital items associated with the provider 247 of old items, a value of the probability of the old digital item receiving, for example, a positive user response; and (b) assigning, based on this probability value, an indication of a synthesized user response for the old digital item so that the indications of the synthesized user responses are distributed in the set 257 of old digital items according to a distribution of responses as described above.

[0196] For example, server 250 may assign an indication of a synthesized user response for an old digital item based on a probability value associated therewith and predetermined upper and lower probability thresholds, as described above.

[0197] According to some non-limiting embodiments of the present technology, since the features of the item for the new portion 275 of the digital items may not be available, the server 250 may determine other features for the old portion 255 of the digital items for the purpose of training the algorithm for generating synthesized user responses, as also described above. In particular, for an old digital item from the old portion 255 of the digital items, the server 250 may determine: (a) an assessment of the similarity of the supplier 247 of the old items of this old digital item relative to other suppliers from the plurality 245 of old suppliers of old items and (b) a value of the popularity parameter.

[0198] According to some non-limiting embodiments of the present technology, the server 250 may determine a similarity score associated with the old item provider 247 in the item provider similarity graph 500, as described in detail with reference to Fig. 5. As described above, the server 250 may determine a popularity parameter value associated with the old digital item based on data from sources outside the online recommendation platform 260.

[0199] In addition, in some non-limiting embodiments of the present technology, the server 250 may determine a probability value of the old digital item receiving an indication of a positive user response based on (a) a first probability value determined based on a similarity score associated with the old item provider 247 according to equation (2), and/or (b) a second probability value determined based on a popularity parameter value associated with the old digital item according to equation (4).

[0200] In addition, the server 250 may determine the coefficients a, b, c and d of equations (2) and (4), which are the parameters of the use stage for the algorithm for generating synthesized user responses, using the distribution of responses for the predicted indications of user responses in the old portion 255 of the digital items as actual data. As described in detail above, for this purpose, according to some non-limiting embodiments of the present technology, the server 250 may: (a) initialize the parameters of the use stage for the algorithm for generating synthesized user responses, for example, randomly; (b) iteratively execute the algorithm for generating synthesized user responses, selecting different items from the old portion 255 of the digital items; (c) after a predetermined number of iterations of executing the algorithm for generating synthesized user responses, determine a training distribution of synthesized responses for the thus determined indications of synthesized user responses in the old portion 255 of the digital items; and (d) adjust the parameters of the use stage for the algorithm for generating synthesized user responses such that the training distribution of the synthesized responses matches, with a required accuracy, the distribution of responses for the predicted user response indications determined by the ML scoring model.

[0201] Thus, the server 250 may determine the parameters of the use stage for the algorithm for generating synthesized user responses such that equations (2) and (4) are represented by equations (5), which in this case may be used to determine the indications of synthesized user responses for each element of the new portion 275 of digital elements.

[0202] The first method 800 then continues at step 806.

Step 806: receiving by the server an indication of a new item provider from a plurality of new item providers who has uploaded a plurality of new digital items from a new part of digital items to the online recommendation platform, and applying by the server an ML evaluation model with respect to the new digital item from the plurality of new digital items of the new item provider to determine an indication of a synthesized user response for the new digital item.

[0203] According to some non-limiting embodiments of the present technology, at step 806, the server 250 may receive from a plurality 265 suppliers of new items a new portion 275 of digital items and apply to them an algorithm for generating synthesized user responses with the parameters determined at step 804.

[0204] For this purpose, the server 250 may: (a) determine for the new digital item a first and/or second value of the probability of the usage stage for the new digital item to receive an indication of a positive user response according to equations (5); (b) based on the first and/or second values of the probability of the usage stage, determine a value of the probability of the usage stage for the new digital item to receive an indication of a positive user response; and (c) based on the value of the probability of the usage stage, determine an indication of a negative or positive synthesized user response, as described above in relation to training the algorithm for generating synthesized user responses.

[0205] Similar to what is described above, the server 250 may determine a first value of the usage stage probability for the new digital item to receive an indication of a positive user response based on a usage stage similarity score for the new item provider 267. In some non-limiting embodiments of the present technology, the server 250 may determine a usage stage similarity score associated with the new item provider 267 based on a usage stage item provider similarity graph formed similar to the item provider similarity graph 500, but based on the similarity parameter values of the sets 245, 265 of old and new item providers received from human raters, as described in detail with reference to FIG. 6. Having determined the usage stage similarity score, the server 250 may determine a first value of the usage stage probability for the new digital item to receive an indication of a positive user response using equations (5).

[0206] In addition, the server 250 may (a) determine for the new digital item a usage stage popularity parameter value and (b) by applying equations (5) with respect to the usage stage popularity parameter value, determine a second usage stage probability value for the new digital item to receive an indication of a positive user response.

[0207] In addition, as also described above, based on the first and/or second values of the probability of the stage of use, the server 250 can determine a value of the probability of the stage of use for receiving an indication of a positive user response by the new digital item, based on which the server 250 can determine an indication of a synthesized user response for the new digital item.

[0208] Accordingly, by iteratively executing the algorithm for generating synthesized user responses for a predetermined number of iterations and selecting at each iteration a different new digital element from the new portion 275 of digital elements, as described above in relation to the old portion 255 of digital elements, the server 250 can generate an indication of a synthesized user response for each element from the new portion 275 of digital elements.

[0209] Next, the first method 800 proceeds to step 808.

Step 808: generating by the server a training data set containing a plurality of training digital objects, each of which contains (a) one of a plurality of new digital elements of a supplier of new elements and (b) an indication of a synthesized user response.

[0210] Having determined an indication of a synthesized user response for each element from the new portion 275 of digital elements, in step 808 the server 250 may use the new portion 275 of digital elements with the thus determined indications of the synthesized user responses for the purpose of forming a new training data set comprising a plurality of new training digital objects, each of which comprises at least (a) an indication of a new digital element from the new portion 275 of digital elements and (b) at least one indication of a synthesized user response determined as described above.

[0211] The first method 800 then continues at step 810.

Step 810: inputting a plurality of training digital objects into the MLA algorithm by the server to train the MLA algorithm to generate recommendations of digital items.

[0212] In step 810, the server 250 may, based on the plurality of new digital items, train the MLA algorithm 280 to determine the set 225 of recommended digital items, as described above. In particular, the server 250 may, based on the plurality of new training digital objects, train the MLA algorithm 280 to determine the probability value of a positive action by the user 220 with the new digital item and, based on this probability value, determine a set of recommended digital items from among other items from the old and new parts 255, 275 of digital items, an example of which is described in detail above with reference to Fig. 7.

[0213] This completes the execution of the first method 800.

The second way

[0214] The architecture and examples described above allow for another method of generating a training data set for training the MLA algorithm 280 to generate recommendations of digital items for users of the online recommendation platform 260. Fig. 9 shows a flow chart of a second method 900 according to some non-limiting embodiments of the present technology. The method 900 can also be performed by the server 250.

[0215] Similar to the first method 800 described above, the second method 900 is designed to generate a training data set for training the MLA algorithm by generating indications of synthesized user responses for a new portion 275 of digital items that do not have indications of actual user responses from users of the online recommendation platform 260.

Step 902: receiving an indication by the server of a new item provider that has a plurality of new digital items to be uploaded to the online recommendation platform and that have no previously received user responses.

[0216] The second method 900 begins with step 902, in which the server 250 may receive an indication of a provider 267 of new items that has uploaded a plurality 277 of new digital items to the online recommendation platform 260, as described above.

[0217] The second method 900 then continues at step 904.

Step 904: receiving by the server an indication of a plurality of legacy item providers that have uploaded a plurality of legacy digital items, at least some of which have received a user response from at least one user of the online recommendation platform.

[0218] At step 904, the server 250 may receive an indication of a plurality 245 of providers of old elements of the online recommendation platform 260, where at least some of the old digital elements have at least one actual indication of a user response.

[0219] The second method 900 then continues at step 906.

Step 906: analyzing the new item provider and the plurality of old item providers by the server to determine a reference old item provider based on a similarity parameter of the new item provider with each of the plurality of old item providers.

[0220] In step 906, the server 250 may determine, among the plurality 245 suppliers of old items, a reference supplier of old items for the supplier 267 of new items. In some non-limiting embodiments of the present technology, the server 250 may determine the reference supplier of old items as the supplier from the plurality 245 suppliers of old items that is most similar to the supplier 267 of new items. For example, in these embodiments of the invention, the server 250 may determine the supplier from the plurality 245 suppliers of old items that is most similar to the supplier 267 of new items based on the labels of the human raters, as described above with reference to FIG. 6.

[0221] In other non-limiting embodiments of the present technology, the server 250 may determine the reference supplier of old items for the supplier 267 of new items as the most popular supplier from the plurality 245 suppliers of old items, as described above with reference to Fig. 5.

[0222] The second method 900 then continues at step 908.

Step 908: determining, by the server, a distribution of responses for indications of user responses in the plurality of legacy digital items associated with the reference legacy item provider.

[0223] In step 908, the server 250 may determine a local distribution of responses for user response indications in the plurality of old digital items associated with the reference provider of old items. In some non-limiting embodiments of the present technology, the server 250 may determine a local distribution of responses only for actual user response indications in the plurality of old digital items associated with the reference provider of old items. In other non-limiting embodiments of the present technology, the server 250 may determine a local distribution of responses for predicted user response indications in the plurality of old digital items determined by the ML scoring model, as described above.

[0224] There is no limitation on the method by which the server 250 determines the local distribution of responses. It may include any of the approaches described above in relation to the distribution of responses.

[0225] The second method then proceeds to step 910.

Step 910: assigning, by the server, indications of the synthesized user responses to the plurality of new digital items of the new item provider according to the distribution of responses for indications of the user responses in the plurality of old digital items of the reference old item provider.

[0226] At step 910, the server 250 may assign indications of the synthesized user responses to the set 277 new digital items of the provider 267 new items according to a local distribution of responses for indications of the user responses in the set of old digital items of the reference provider of old items.

[0227] For example, it can be assumed that the set of old digital elements of the reference supplier of old elements contains five old digital elements with indications of a positive or negative user response distributed therein as follows: [100/25; 108000/580; 9995874/36954; 135001/398; 127/24]. Thus, using this local distribution of responses, the server 250 can assign indications of synthesized user responses to the set 277 of new digital elements so as to minimize the differences between parameters, such as the mathematical expectation and the standard deviation, for the local distribution of responses and for the thus formed distribution of indications of a synthesized user response in the set 277 of new digital elements (described above). Thus, if the set 277 of new digital elements also contains five digital elements, then the server 250 can determine for it indications of synthesized user responses that coincide with the local distribution of responses from the example described above. As a result, it can be said that the server 250 simulates the local distribution of responses for indications of user responses in the set of old digital elements in order to generate indications of synthesized user responses for each element of the set 277 of new digital elements.

[0228] The second method 900 then continues at steps 912 and 914, which are implemented similarly to steps 808 and 810, respectively, of the first method 800.

[0229] This completes the execution of the second method 900.

[0230] After training the MLA algorithm 280 based on the new training data set generated according to the first or second methods 800, 900, the server 250 may present to the user 220 at least some items from the new portion 275 of digital items as part of the set 225 of recommended digital items.

[0231] After presenting a new digital item from the new portion 275 of digital items to the user 220 by assigning an indication of a synthesized user response to it, as described above, the server 250 may receive an indication of a user response from the user 220. If the indication of a user response from the user 220 differs from the indication of a synthesized user response, the server 250 may replace the latter with the former. In other non-limiting embodiments of the present technology, the server 250 may add an indication of a user response from the user 220 to the indication of a synthesized user response thus generated.

[0232] Thus, some non-limiting embodiments of the present technology allow new digital items to be presented to users of the online recommendation platform 260 with limited data about these new digital items, which may contribute to increased satisfaction of users and a plurality of providers of new items.

[0233] Possible changes and improvements to the above-described embodiments of the present technology may be obvious to a person skilled in the art. The preceding description is provided for illustrative purposes only and is not intended to limit the scope of the invention. The scope of protection of the present technology is determined solely by the scope of the appended claims.

[0234] Although the embodiments described above are described with reference to specific steps performed in a particular order, it should be understood that these steps may be combined, separated, or that their order may be changed without departing from the scope of the present technology. Accordingly, the order and grouping of the steps are not limiting to the present technology.

Claims (60)

1. A method for forming a training data set for training a machine learning algorithm to form recommendations of digital elements for users of an online recommendation platform that has a plurality of digital elements, containing: (1) an old portion of digital items, each of which was uploaded to the online recommendation platform by one of a plurality of old item providers, wherein at least some of the old digital items in the old portion of digital items have received a user response from at least one user of the online recommendation platform, and indications of the user responses of at least some of the old digital items are distributed in the old portion of digital items according to the distribution of responses; and (2) a new portion of digital items, each of which was uploaded to the online recommendation platform by a new item provider from a plurality of new item providers and has no user feedback, in this case, the formation involves the formation of synthesized user responses for new digital elements, and the method is performed by the server on which the online recommendation platform is located, and includes: - at the first stage - training the server of the machine learning evaluation model to determine the predicted indication of the user response for a digital element, including: - receiving by the server indications of the old portion of digital elements for the purpose of forming a plurality of training digital objects for assessment, each of which contains (a) the old digital element from the old portion of digital elements and (b) an indication of the user response received by the old digital element; and - inputting by the server a plurality of digital objects for evaluation into an evaluation machine learning model and thereby training the evaluation machine learning model to determine a predicted user response indication for the digital item using the response distribution for user response indications in the old portion of the digital items as actual data; - at the second stage, following the first stage: - receiving by the server an indication of a supplier of new elements from a plurality of suppliers of new elements who has uploaded a plurality of new digital elements from a new part of digital elements to the online recommendation platform; - applying by the server an evaluation machine learning model in relation to a new digital item from a plurality of new digital items of a supplier of new items in order to determine an indication of a synthesized user response for the new digital item; - the formation by the server of a training data set containing a plurality of training digital objects, wherein the training digital object contains (a) one of a plurality of new digital elements of the supplier of new elements and (b) an indication of a synthesized user response; and - input by the server of a set of training digital objects into the machine learning algorithm to train the machine learning algorithm to generate recommendations of digital elements. 2. The method according to claim 1, characterized in that the use of the machine learning evaluation model includes determining a first value of the probability of receiving an indication of a positive user response by a new digital element of a supplier of new digital elements, wherein the determination is based on an assessment of the similarity of the supplier of new elements with a set of suppliers of old elements. 3. The method according to paragraph 2, characterized in that the first probability value is determined by the formula: , where artistScore is the similarity score of the new item supplier to the set of old item suppliers, a and b are coefficients determined by training the machine learning scoring model. 4. The method according to paragraph 2, characterized in that it additionally includes determining by the server an assessment of the similarity of the supplier of new elements with a set of suppliers of old elements. 5. The method according to paragraph 4, characterized in that determining the similarity assessment of the supplier of new elements includes: - obtaining for the supplier of old elements from the set of suppliers of old elements the value of the similarity parameter of the supplier of old elements with the supplier of new elements; - the formation by the server, based on the values of the similarity parameter of each supplier from the set of suppliers of old elements with the supplier of new elements, of an oriented graph structure, in which a vertex of the oriented graph structure represents a supplier of elements from the set of suppliers of old elements or from the set of suppliers of new elements, and an edge connecting a pair of vertices represents a non-zero value of the similarity parameter of the supplier of new elements with one of the set of suppliers of old elements; - determining by the server for the edge a weight value indicating the value of the similarity parameter of the supplier of new elements with one of the plurality of suppliers of old elements; and - determination by the server of the similarity score of the supplier of new elements as the maximum total weight value when moving from the vertex of the directed graph structure associated with the supplier of new elements to its initial vertex. 6. The method according to paragraph 5, characterized in that determining the weight value includes determining the inverse value for the value of the similarity parameter of the supplier of new elements with one of the plurality of suppliers of old elements. 7. The method according to claim 5, characterized in that the similarity parameter represents the degree of similarity of the supplier of new elements with each of the plurality of suppliers of old elements. 8. The method according to claim 7, characterized in that it additionally includes obtaining from a human evaluator the degree of similarity of the supplier of new elements with each of the plurality of suppliers of old elements. 9. The method according to claim 2, characterized in that the use of the machine learning evaluation model additionally includes determining a second value of the probability of receiving an indication of a positive user response by a new digital element of a supplier of new digital elements, wherein the determination is based on the value of the popularity of the new digital element in a given area. 10. The method according to paragraph 9, characterized in that the second probability value is determined by the formula: , where Popularity is the popularity value of a new digital element in a given area, maxPopularity is the popularity value of the most popular new digital element in a given area from the corresponding set of digital elements of the supplier of new elements, minPopularity is the popularity value of the least popular new digital element in a given area from the corresponding set of digital elements of the supplier of new elements, c and d are coefficients determined by training the evaluating machine learning model. 11. The method according to claim 9, characterized in that the new digital element is an audio stream, and the popularity value of the new digital element indicates at least one of the following: (a) the number of streams of the audio stream on the media resource; (b) the number of requests to play the audio stream on the media resource; (c) the position of the audio stream in the hit parade; (d) the number of albums sold containing the audio stream. 12. The method according to paragraph 9, characterized in that it additionally includes: - assigning by the server to the new digital element an indication of a positive synthesized user response if the first and/or second probability value is greater than the upper probability threshold; and - assigning by the server to the new digital element an indication of a negative synthesized user response if the first and/or second probability value is less than the lower probability threshold, which is less than the upper probability threshold. 13. The method according to claim 12, characterized in that before applying the machine learning evaluation model, it additionally includes the server selecting a new digital element from a plurality of new digital elements. 14. The method according to claim 13, characterized in that the selection includes selecting a new digital element from a plurality of new digital elements in one of the following ways: (a) randomly; (b) using uniform selection probabilities; (c) using a selection probability value proportional to the popularity value of the new digital element in the given area; and (d) using a selection probability value proportional to the square root of the popularity value of the new digital element in the given area. 15. The method according to claim 13, characterized in that the selection and application are performed iteratively over a predetermined number of iterations before forming the training data set. 16. The method according to claim 1, characterized in that it additionally includes updating by the server the training digital object from the plurality of training digital objects by replacing in it the indication of the synthesized user response with an indication of the user response, if the server has determined that a new digital element from the plurality of new digital elements has received a user response. 17. The method according to claim 1, characterized in that the evaluating machine learning model is a matrix factorization machine learning model. 18. The method according to claim 1, characterized in that the machine learning algorithm is a machine learning algorithm based on decision trees, a machine learning algorithm based on a transformer, or a machine learning algorithm based on deep semantic similarity. 19. A method for forming a training data set for training a machine learning algorithm to form recommendations of digital elements for users of an online recommendation platform that has a plurality of digital elements, wherein the formation involves the formation of synthesized user responses for new digital elements, and the method is performed by a server on which the online recommendation platform is hosted, and includes: - receiving by the server an indication of a supplier of new elements from among suppliers of new elements, having a plurality of new digital elements subject to uploading to the online recommendation platform, wherein the new digital element from the plurality of new digital elements does not have a previously received user response; - receiving by the server an indication of a plurality of legacy item providers that have uploaded a plurality of legacy digital items, at least some of which have received a user response from at least one user of the online recommendation platform; - analysis by the server of the supplier of new elements and the set of suppliers of old elements in order to determine the reference supplier of old elements based on the similarity parameter of the supplier of new elements with each of the set of suppliers of old elements; - determining by the server synthesized user responses for a plurality of new digital elements of a supplier of new elements based on user responses received by a plurality of old digital elements of a reference supplier of old elements, including: - determining by the server a distribution of responses for indications of user responses in a set of legacy digital items associated with a reference legacy item provider; - assignment by the server of indications of synthesized user responses to a set of new digital elements of a supplier of new elements according to the distribution of responses for indications of user responses in a set of old digital elements of a reference supplier of old elements; - the formation by the server of a training data set containing a plurality of training digital objects, wherein the training digital object contains (a) one of a plurality of new digital elements of the supplier of new elements and (b) an indication of a synthesized user response; and - input by the server of a set of training digital objects into the machine learning algorithm to train the machine learning algorithm to generate recommendations of digital elements. 20. A server for generating a training data set for training a machine learning algorithm to generate recommendations of digital elements for users of an online recommendation platform located on the server and having a plurality of digital elements, containing: (1) an old portion of digital items, each of which was uploaded to the online recommendation platform by one of a plurality of old item providers, wherein at least some of the old digital items in the old portion of digital items have received a user response from at least one user of the online recommendation platform, and indications of the user responses of at least some of the old digital items are distributed in the old portion of digital items according to the distribution of responses; and (2) a new portion of digital items, each of which was uploaded to the online recommendation platform by a new item provider from a plurality of new item providers and has no user feedback, in this case, the formation involves the formation of synthesized user responses for new digital elements, and the server contains machine-readable physical memory that stores commands, and a processor capable of performing the following actions when executing commands: - at the first stage - training the machine learning evaluation model to determine the predicted user response indication for a digital element by: - receiving indications of an old portion of digital elements to form a plurality of training digital objects for assessment, each of which comprises (a) an old digital element from the old portion of digital elements and (b) an indication of a user response received by the old digital element; and - inputting a plurality of digital objects for evaluation into a machine learning evaluation model and training the machine learning evaluation model to determine a predicted user response indication for the digital item using the response distribution for user response indications in the old portion of the digital items as actual data; - at the second stage, following the first stage: - receiving an indication of a supplier of new elements from a plurality of suppliers of new elements who has uploaded a plurality of new digital elements from a new part of digital elements to the online recommendation platform; - applying a machine learning evaluation model to a new digital item from a plurality of new digital items of a supplier of new items in order to determine an indication of a synthesized user response for the new digital item; - forming a training data set containing a plurality of training digital objects, wherein the training digital object contains (a) one of a plurality of new digital elements of a supplier of new elements and (b) an indication of a synthesized user response; and - inputting a set of training digital objects into a machine learning algorithm to train the machine learning algorithm to generate recommendations for digital elements.