HK1202961B - Unified marketplace for advertisements and content in an online system - Google Patents

Abstract

A server system of an online information system displays advertising items and content items retrieved from storage devices as a stream viewable by a user on a user device. The advertisement items and the content items are ordered in the stream by a ranking score computed for each of the advertisement items and each of the content items. The server system transmits a web page including the stream to a user device over a network. In this manner, advertising items and content items compete in a unified marketplace for inclusion in the stream for viewing by the end user.

Description

Unified marketplace for advertisements and content in online systems

Background

The present application relates generally to data processing systems. More particularly, the present application relates to systems and methods for online display of revenue generating information (e.g., advertisements) and non-revenue generating information (e.g., content) together.

Online advertising is becoming increasingly popular as a way for advertisers to promote goods and services information to potential customers and clients. Advertisers may use the internet-accessible facilities of online providers (e.g., Yahoo | Inc.). Online providers are used to connect advertisers with users who access online resources, such as search engines and news and information websites. An advertiser's advertisement ("ad") is provided to the user to inform and attract the user's attention.

Some online providers provide a stream of content and other information on a web page. A user may access a web page on a device, such as a desktop computer, a portable computer (e.g., a laptop computer), a handheld device (e.g., a tablet computer and a smart phone), or a variety of media devices (e.g., a television). As the stream is browsed on the display of the device, it is presented on the web page item by item (e.g. downwards on the web page) in the sequence of items displayed. In some cases, the stream may be updated with new content at the top or bottom of the page based on certain events, such as the passage of a certain period of time, the scrolling of a mouse, or the clicking of a space bar.

Advertising items (also referred to herein as "stream ads") are inserted within the content stream to supplement the sequence of items. The stream advertisement may be formatted to visually match the surrounding content stream so as to appear to originate from the stream. Alternatively, the stream advertisement may be formatted to complement the surrounding content stream for greater eye-catching.

Streams have become commonplace on online presentations, in part because they provide more flexibility to web site designers and advertisers. If the stream is not used to render data on the web page, the web page must have a predefined portion. Only certain types of information having a specified size, shape and content may be presented on the predefined portion. A stream allows for content including any number, size, and shape. When the viewer processes information associated with different content or advertising items, the flow reduces the viewer's cognitive load by clearing cognitive overload associated with switching to a different visual format or angle.

It is desirable to manage the traffic of content and advertisements in a stream to further manage the experience of users and advertisers interacting with online providers. Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with some aspects of the present disclosure as set forth in the remainder of the present application with reference to the drawings.

Disclosure of Invention

In accordance with the systems, products, and methods disclosed herein, an online provider may control the location, quantity, and spatial and temporal frequency of stream advertisements within a content stream that is viewable by a user on a web page. A stream may be viewed as a unified marketplace (unified markplace) in which both revenue generating items and non-revenue generating items (e.g., advertising items and content items) compete for presentation in the stream, respectively. Scoring, ranking, and pricing techniques allow for a commensurate processing of all items (whether or not there is revenue generation for the online provider). Other business rules for content items and advertising items provide further freedom for online providers in determining how content items and advertising items are presented in a stream.

For online providers, controlling the location, quantity, and frequency of advertisements in a stream may help manage the user and advertiser experience with the online provider's website. Providing too many advertisements may result in a user experience that is less than satisfactory. Providing too few advertisements may reduce or eliminate advertiser participation in the website. Providing the most appropriate content items and advertising items for the user may keep the user engaged with the website and ensure that the user will return to the website. The participation of the user in turn drives the confidence and participation of the advertiser who placed the streaming advertisement on the website. The present disclosure generally describes a unified marketplace where each item of information presented by an online provider (from revenue generating advertisements to paid content) is scored, priced according to explicit or implicit bids, and ranked for presentation in a unified format, such as streaming.

Drawings

FIG. 1 is a block diagram of an example online information system;

FIG. 2 is an example of a display advertisement modified for display in a streaming display;

FIG. 3 is a flow diagram illustrating one embodiment of a method for ranking and displaying a stream of advertisement items and content items in an online information system; and

FIG. 4 is an example process for displaying content in a streaming media feed according to a quality score calculated using clickability and a post-click satisfaction score.

Detailed Description

The subject matter now will be described more fully hereinafter with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific exemplary embodiments. The subject matter, however, may be embodied in many different forms and, thus, the encompassed or claimed subject matter is intended to be construed as being limited to any example embodiment set forth herein; the example embodiments provided are merely illustrative. Also, a reasonably broad scope of claimed or encompassed subject matter is intended. For example, the subject matter may be implemented as a method, device, component, or system, among others. The following detailed description is, therefore, not intended to limit the scope of the claims.

Throughout this specification and claims, terms may have in context implications or implications of subtle meanings beyond those explicitly set forth. Likewise, the phrase "in one embodiment" as used herein does not necessarily refer to the same embodiment, and the phrase "in another embodiment" as used herein does not necessarily refer to a different embodiment. For example, it is intended that claimed subject matter include all or a combination of some of the example embodiments.

In general, terms may be understood as usage that results at least in part from the context. For example, as used herein, terms such as "and," "or," or "and/or" may include various meanings that may depend, at least in part, on the context in which such terms are used. Typically, if used to associate a list such as A, B or C, an "or" is intended herein in the inclusive sense to mean A, B and C, and herein in the exclusive sense to mean A, B or C. Furthermore, the term "one or more" as used herein may be used to describe any feature, structure, or characteristic in the singular or may be used to describe a combination of features, structures, or characteristics in the plural, depending, at least in part, on the context. Similarly, terms such as "a," "an," "the," and the like may be understood to convey singular usage or to convey plural usage, depending, at least in part, on the context. Further, again depending at least in part on the context, the term "based on" may be understood to not necessarily be intended to convey an exclusive set of factors, and may instead allow for the presence of additional factors that are not necessarily explicitly described.

Online information systems place advertisers' advertisements within content services (such as web pages, mobile applications ("apps"), TV applications, or other audio or visual content services) that are available to end users. The advertisement is provided with other content. Other content may include any combination of text, graphics, audio, video, or links to such content. Advertisements are conventionally selected based on various criteria including those specified by the advertiser. Advertisers routinely define advertising campaigns to control how and when advertisements are made available to users and to specify the content of those advertisements.

Streams are becoming commonplace in online presentations because they provide flexibility to content providers that provide content items of the stream, advertisers that provide advertising items of the stream, and online providers that combine content items and advertising items to produce the stream. Streaming allows any number and size and shape of content items and advertising items to be included in the streaming. The elements in the stream may be ordered by relevance or by any suitable parameter. When the viewer processes information associated with different content or advertising items, the flow reduces the viewer's cognitive load by clearing cognitive overload associated with switching to a different visual format or angle.

The stream may be viewed as a unified marketplace, where content items and advertising items compete for placement or are included in the stream. Participants in the marketplace are advertisers who originate or provide advertising items and content providers who originate or provide content items. The streams and markets may be hosted or managed by online providers such as Yahoo corporation. The online provider may also provide advertisements for its own products and services or its own content items into the stream.

Advertisers interact with online provider's equipment to create or provide online advertisements. The online advertisement includes advertising content and one or more bid amounts stored in association with an identification of the advertiser in a database or other storage. The advertising content may include text or graphics or both, as well as a link to a landing page to which the user's browser is redirected when the link is clicked. The bid amount represents an amount that the advertiser will pay for an event related to the advertisement. The event may be an impression or view of the advertisement by the user, a click and other selection of the advertisement by the user viewing the advertisement, or an action after viewing the advertisement, such as providing credit card information or an email address. The bid amount may be used to determine the position of the advertisement in the stream in a manner to be described below. The online advertisement may include other data as well as data defining how the advertisement appears in the stream.

The content item includes information about topics that may be of interest to the user. This information may include a link to another web page that provides more information for the topic and a summary of the information for the topic. In some embodiments, the content provider will correlate the bid amount with the content item. Similar to the bid amount for the advertisement, the bid amount for the content item may be based on an impression, a click-through, or another action. Also, this bid amount may be used to determine the location of the content item in the stream in a manner to be described below. Alternatively, a software-based bidding agent may be used to automatically bid on behalf of a content item.

The content item and the advertising item compete for inclusion in the stream. The competition for slots in the stream can be cleared using a generalized second-level price (GSP) auction mechanism. In a GSP auction, the highest bidder obtains a first position, the second highest bidder obtains a second position, and so on. However, the highest bidder then pays the price bid by the second highest bidder. This is similar to the paid search marketplace, although bids in paid searches are formulated differently and competition in the paid search marketplace is only between advertisements.

In one embodiment, advertisers provide targeted forecasts, advertising segments, and bids. In some embodiments, an advertiser may provide a triple budget, referred to as triple targeting. The targeting predictions can be based on any type of market segment (segment) of interest to the advertiser, including in one example, a demographic market, a gender or age based market segment, a behavioral segment based on user profile information, or a geographic market. The bid may be a cost-per-click (CPC) bid, a cost-per-impression (CPM) bid, or a cost-per-action (CPA) bid. The online provider may choose not to support all bid types in all markets.

What advertisers are allowed to bid largely determines their bidding behavior. For online providers managing a unified market, there is a tradeoff between allowing advertisers to bid on very specific goals versus allowing advertisers to bid on a more broad range of goals.

Online providers prefer a dense market with many competing advertisers over a light market with few advertisers. The denser the market, the greater the potential for revenue increase for online providers. However, many advertisers are very interested in a particular user type. These narrowly focused users will likely be placed outside the market unless they are allowed to bid more narrowly. The broad targeting reduces the average value obtained by advertisers because their advertisements may be displayed to users who may not be interested in their products. Lower expected values result in lower bids.

Some of these tradeoffs can be mitigated by pricing performance, by using excellent scoring algorithms, and by preventing low relevance ads from being displayed in the stream. Pricing for performance implies that a charge is only made when the user responds to the advertisement. Advertisers will prefer to pay when a user is compliant (e.g., pays for a product or service). However, defining and tracking the compliance and estimating the compliance rate can be difficult to make reliably, so market operators prefer to charge for clicks that are easier to track and estimate. Charging per click presents challenges. For example, not all clicks from the user translate into a sale by the advertiser. In the event that too many clicks do not produce a transition, a low quality score for the advertisement may result.

Extensive targeting requires accurate scoring methods to maintain a good user and advertiser experience. Scoring is the process of assigning a value to an advertisement or content item, which can then be used to determine which item should be included in the stream. This accurate scoring may require the online provider to examine not only the advertisement segments but also the landing page's content. In some embodiments, the advertisement may include additional information, such as metadata that is automatically collected or manually provided by the advertiser and used as a signal to the scoring function.

Extensive targeting also increases the difficulty of pricing CPC advertisements. In pricing advertisements, it is important to distinguish between the quality of the match between the keyword and the search term and the quality of the advertisement. The online operator may choose to discount advertisers for low quality matches, which is the responsibility of the operator of the online marketplace for matching. The online operator may choose to charge an additional fee for low quality advertisements that are the responsibility of the advertiser.

An exemplary system will now be described in which aspects of a unified market for advertising items and content items may be presented and described. Further details and optional embodiments will be provided in connection with the accompanying drawings.

FIG. 1 is a block diagram of a presence information system 100. The online information system 100 in the exemplary embodiment of FIG. 1 includes an account server 102, and account database 104, search engine 106, advertisement (ad) server 108, advertisement database 110, content database 114, content server 112, and ranking engine 116. The online information system 100 may be accessed by one or more advertiser devices (e.g., advertiser device 122) and one or more user devices (e.g., user device 124) over the network 120. In examples of such online information systems, users may search for and obtain content from multiple sources or from content database 114 via network 120. Advertisers may provide advertisements for placement in web pages and other communications sent over a network to user devices, such as user device 124. In one example, the online information system is deployed and operated by an online provider (e.g., Yahoo corporation).

The account server 102 stores account information for advertisers. The account server 102 is in data communication with an account database 104. The account information may include a corresponding one or more database records associated with each advertiser. The account management server 102 may store, maintain, update, and read any suitable information from the content database 104. Examples include advertiser identification information, advertiser security information (such as passwords and other security credentials), and account balance information.

The account server 102 may be implemented using any suitable device. The account management server 102 may be implemented as a single server, multiple servers, or any other type of computing device known in the art. Access to the account server 102 may preferably be accomplished through a firewall (not shown) that protects the account management program and account information from external tampering. Additional security may be provided by enhancing standard communication protocols, such as secure HTTP or secure sockets layer.

The account server 102 may provide an advertiser front end to simplify the process of accessing account information for advertisers. The advertiser front end may be a program, application, or software routine that forms a user interface. In a particular embodiment, the advertiser front end is accessible as a website having access to one or more web pages that an advertiser may view on an advertiser device, such as advertiser device 122. The advertiser can browse and edit account data and advertisement data using the advertiser front end. After editing the advertisement data, the account data may then be saved to the account database 104.

Search engine 106 may be a computer system, one or more servers, or any other computing device known in the art. Alternatively, search engine 106 may be a computer program, instructions, or software code stored on a computer-readable storage medium that runs on a processor of a single server, multiple servers, or any other type of computing device known in the art. For example, user devices operated by a user (e.g., user device 124) may access search engine 106 via network 120. User device 124 communicates the user query to search engine 106. The search engine 106 locates the matching information using any suitable protocol or algorithm and returns the information to the user device 124. The search engine 106 may be designed to assist users in finding information located on the internet or an intranet. In one particular example, the search engine 106 may also provide web pages to the user device 124 over the network 120 with content that includes: search results, information matching the context of the user query, links to other network destinations or files of information and information of interest to the user operating user device 124, and streams of content items and advertising items selected for display to the user.

Search engine 106 may enable a device (e.g., user device 124 or any other client device) to search for files of interest using a search query. Typically, a client device may access search engine 106 through one or more servers or directly through network 120. For example, in one illustrative embodiment, the search engine 106 can include a web crawler component, an indexer component, an index store component, a search component, a ranking component, a cache, a profile store component, a login component, a profile builder, and one or more Application Program Interfaces (APIs). The search engine 106 may be deployed in a distributed manner, such as through a collection of distributed servers, for example. Components may be replicated within the network, such as for redundant or better access.

The ad server 108 operates to serve ads to user devices, such as user device 124. The advertisement includes data defining advertisement information of interest to a user of the user device. The advertisement may include text data, graphics data, image data, video data, or audio data. The advertisement may further include data defining one or more links to other network resources that provide such data. The other location may be other locations on the internet, other locations on an advertiser-operated intranet, or any access.

For online information providers, advertisements may be displayed on web pages resulting from user-defined searches based at least in part on one or more search terms. If the displayed advertisement is relevant to one or more of the user's interests, the advertisement may be beneficial to the user, advertiser, or web portal. Accordingly, various techniques have been developed to infer user interests, user intent, or to subsequently target related advertisements to consumers.

One method of presenting targeted advertisements includes using demographic characteristics (e.g., age, income, gender, occupation, etc.) for predicting user behavior, such as by group. An advertisement may be presented to a user in the target audience based at least in part on the predicted user row.

Another approach includes profile advertising targeting. In the present approach, a user-specific user profile may be generated to model user behavior, for example, by tracking the user's path through a website or network of sites, and compiling the profile based at least in part on the final delivered web page or advertisement. Correlations may be identified, for example, as being for what the user purchased. The identified correlations may be used to target potential purchasers by targeting content or advertisements to specific users.

Yet another method includes targeting based on content of a web page requested by a user. Advertisements may be placed on web pages or associated with other content that is relevant to the subject matter of the advertisements. The relationship between the content and the advertisement may be determined in any suitable manner. The overall theme of a particular web page may be determined, for example, by analyzing content presented within the web page. In addition, a variety of techniques have been developed for displaying advertisements that are appropriate for the particular portion of the article that the user is currently browsing. Accordingly, advertisements may be selected by matching the advertisements to keywords and/or phrases within the web page. An exemplary system and method is disclosed in pending U.S. patent application No. 13/836,052 entitled Efficient Matching of User Profiles for viewer purchases to viewer partitions filed on 15/3.2013 (effective Matching of User Profiles with Audio Segments for Audio Buy). This application is incorporated herein by reference in its entirety.

The ad server 108 includes logic and data operative to format ad data for transmission to user devices. The ad server 108 is in data communication with an ad database 110. The advertisement database 110 stores information including data defining advertisements to be served to user devices. The advertisement data may be stored in the advertisement database 110 by another data processing device or advertiser. The advertising data can include data defining an advertising creative and a bid amount for a corresponding advertisement.

For example, the advertisement data may be formatted as advertisement items that may be included within a content item and advertisement item stream provided to the user device. The formatted advertising items are specified by the appearance, size, shape, text formatting, graphics formatting, and included information, all of which may be standardized to provide a consistent look for all advertising items in the stream. At least some of the advertising items may have an associated bid amount and may be considered revenue generating items. The ad server 108 then provides these ad items to other network devices (e.g., the ranking engine 116).

Further, the ad server 108 is in data communication with a network 120. The ad server 108 transmits the ad data and other information to the devices over the network 120. This information may include advertisement data that is transmitted to the user device. This information may also include advertisement data and other information transmitted with advertiser devices, such as advertiser device 122. Advertisers operating advertiser devices may access information including advertisement data through a network access advertisement server 108. This access may include developing ad creatives, editing ad data, deleting ad data, setting and adjusting bid amounts, and other activities.

The ad server 108 may provide an advertiser front end to simplify the process of accessing the advertiser's ad data. The advertiser front end may be a program, application, or software routine that forms a user interface. In a particular embodiment, the advertiser front end is accessible as a website having one or more web pages that an accessing advertiser may view on an advertiser device. Advertisers can browse and edit advertisement data using the advertiser front end. After the ad data is edited, the ad data may then be saved to the ad database 110 for subsequent delivery to the user device in the form of an ad.

The ad server 108 may be a computer system, one or more servers, or any other computing device known in the art. Alternatively, the search engine 108 may be a computer program, instructions, and/or software code stored on a computer-readable storage medium that runs on a processor of a single server, multiple servers, or any other type of computing device known in the art.

The content server 112 is in data communication with a content database 114, the advertisement server 108, and a ranking engine 116. The content server 112 may access information for the content item from the content database 114 or from another location accessible through the network 120. The content server 112 transmits data defining the content item and other information to the devices over the network 120. This information may include content data that is transmitted to the user device. This information may also include content data and other information transmitted with the content provider operating the content provider device. Content providers operating provider devices may access information including content data through network 120 to access content server 112. This access may include developing content items, editing content items, deleting content items, setting and adjusting bid amounts, and other activities.

The content server 112 may provide a content provider front end to simplify the process of accessing content data of a content provider. The content provider front end may be a program, application, or software routine that forms a user interface. In a particular embodiment, the content provider front end is accessible as a website having one or more web pages that an accessing content provider may browse on a content provider device. The content provider may browse and edit the content data using the content provider front end. After the content data is edited, the content data is then saved to the content database 114 for subsequent transmission to the user device.

The content server 112 includes logic and data operable to format content data and other information for delivery to user devices. For example, the content data may be formatted as content items that may be included within a stream of content items and advertising items provided to the user device. The formatted content items are specified by appearance, size, shape, text formatting, graphics formatting, and included information, all of which may be standardized to provide a consistent look for all content items in the stream. In some embodiments, the content items have associated bid amounts that are used to rank or locate content items in an item stream presented to the user device. In other embodiments, the content items do not include a bid amount, or the bid amount is not used to rank the content items. Such content items may be considered non-revenue generating items. The content server 112 then provides these content items to other network devices (e.g., the ad server 108 and the ranking engine 116).

The ranking engine 116 is in data communication with the advertisement server 108, the advertisement database 110, the content server 112, and the content database 114. The ranking engine 118 is configured to identify items to be included in a stream of content items and advertising items to be provided to a user device, such as user device 124. The ranking engine 118 may thus be configured for determining which advertising items and which content items qualify for inclusion in the stream and for scoring and for ordering the individual advertising items and individual content items in the stream.

In one embodiment, the ranking engine 116 is configured to calculate a ranking score for each of a plurality of advertising items using bid values retrieved from the advertising database 110. The ranking engine 116 is further configured to calculate a ranking score for each of the plurality of content items using the bid values obtained from the content database 114. The ranking engine 116 may use other information available from the ad server 108, ad database 110, content server 112, and content database 114, and the account database 104 when determining the ranking score. Other embodiments and other details of exemplary operation of the online information system 100 including the ranking engine will be described below.

The account server 102, search engine 106, advertisement server 108, content server 112, and ranking engine 114 may be implemented as any suitable computing device. The computing device may be capable of sending or receiving signals, such as over a wired or wireless network, or may be capable of processing signals or storing signals in memory, such as a physical memory state, and thus may operate as a server. Thus, a device capable of operating as a server may include, by way of example, a dedicated rack-mounted server, a desktop computer, a laptop computer, a set-top box, an integrated device combining multiple features of two or more of the above-described devices, and so forth.

Servers can vary widely in configuration or capability, but in general a server can include one or more central processing units and memory. The server may also include one or more mass storage devices, one or more power supplies, one or more wired or wireless network interfaces, one or more input/output interfaces, or one or more operating systems, such as a Windows server, Mac OS X, Unix, Linux, FreeBSD, etc.

The account server 102, search engine 106, advertisement server 108, content server 112, and ranking engine 114 may be implemented as or may be in communication with an online server system. The online server system may include a device including a configuration to provide content to another device over a network, including in response to a received request for page browsing. The online server system may, for example, host a website, such as a social networking website, examples of which may include, but are not limited to, a web photo album (Flickr), Twitter (Twitter), Facebook (Facebook), human relationship web (LinkedIn), or an individual user website (e.g., blog, video blog, online dating website, etc.). The online server system may also host various other websites including, but not limited to, commerce websites, educational websites, dictionary websites, encyclopedia websites, wikipedia, financial websites, government websites, and the like.

The online server system may further provide various services including, but not limited to, web services, third party services, audio services, video services, email services, Instant Messaging (IM) services, SMS services, MMS services, FTP services, Voice Over IP (VOIP) services, calendar services, camera services, and the like. Examples of content may include text, images, audio, video, etc., which may be processed in the form of physical signals (e.g., as electrical signals), or which may be stored, for example, in a memory as physical states. Examples of devices that may operate as an online server system include desktop computers, multiprocessor systems, microprocessor-based or programmable consumer electronics, and the like. The online server system may not be under common ownership or control with the ad server 108, the content server 112, or the ranking engine 116.

Network 120 may include any data communication network or combination of networks. A network may connect multiple devices such that communications may be interchanged, such as between a server and a client device or other type of device, including between wireless devices connected via a wireless network, for example. The network may also include mass storage, such as Network Attached Storage (NAS), a Storage Area Network (SAN), or other forms of computer or machine readable media, for example. The network may include the internet, one or more Local Area Networks (LANs), one or more Wide Area Networks (WANs), wired connections, wireless connections, or any combination thereof. Likewise, sub-networks (e.g., which may use different architectures or may conform to or be compatible with different protocols) may interoperate within a larger network (e.g., network 120). Various types of devices may be available, for example, to provide interoperable capabilities for different architectures or protocols. As one illustrative example, a router may provide links between otherwise separate and independent LANs. The communication link or channel may include, for example, an analog telephone line (e.g., twisted pair, coaxial cable), all or a portion of a digital line including T1, T2, T3, or T4 types of lines, an Integrated Services Digital Network (ISDN), a Digital Subscriber Line (DSL), a wireless link including a satellite link, or any other communication link or channel as may be known to those of skill in the art. Further, for example, the computing device or other related electronic devices may be remotely connected to a network, such as by a telephone line or connection.

Advertiser devices 122 include any data processing device that may access online information system 100 via network 120. Advertiser devices 122 operate to interact with account server 102, search engine 106, advertisement server 108, ranking engine 116, content servers, and other data processing systems via network 120. Advertiser device 122 may, for example, implement a web browser for browsing web pages and submitting user requests. The advertiser device 122 may transmit data to the online information system 100, including data defining web pages and other information. The advertiser device 122 may receive communications from the online information system 100 including data defining a web page and an advertising creative.

In some embodiments, a content provider may access the online information system 100 with a content provider device that is generally similar in structure and function to an advertiser device. The content provider provides access to content data in the content database 114, for example.

The user device 124 comprises any data processing device that can access the online information system 100 through the network 120. User device 124 is operative to interact with search engine 106 via network 120. User device 124 may, for example, implement a web browser for browsing web pages and submitting user requests. A user operating the user device 124 may enter a search request and transmit the search request to the online information system 100. The search engine processes the search request and the search results are returned to the user device 124. In other examples, a user of the user device 124 may request data, such as an information page, from the online information handling system 100. Alternatively, the data may be provided in another environment, such as a local mobile application, a TV application, or an audio application. The online information handling system 100 may provide this data or redirect the browser to another website. Further, the advertisement server may select advertisements from the advertisement database 110 and include data defining the advertisements in the data provided to the user device 124.

When accessing information on the online information system 100, the advertiser device 122 and the user device 124 operate as client devices. Client devices, such as advertiser device 122 and user device 124, may include computing devices capable of sending or receiving signals, such as over a wired or wireless network. The client devices may include, for example, desktop or portable devices (e.g., cellular phones, smart phones, display pagers), Radio Frequency (RF) devices, Infrared (IR) devices, Personal Digital Assistants (PDAs), handheld computers, tablet computers, laptop computers, set-top boxes, wearable computers, integrated devices combining various features (such as those of the devices described above), and the like. In the example of fig. 1, both the laptop 126 and the smartphone 128 may be operated as advertiser devices or user devices.

Client devices may differ in capabilities and features. The claimed subject matter is intended to cover a wide range of possible variations. For example, a cell phone may include a numeric keypad or a limited functionality display, such as a black and white Liquid Crystal Display (LCD) for displaying text. In contrast, however, as another example, a network-enabled client device may include one or more physical or virtual keyboards, mass storage, one or more accelerometers, one or more gyroscopes, Global Positioning System (GPS) or other location-identifying type capabilities, or a display with a high degree of functionality, such as a touch-sensitive color 2D or 3D display, for example. The client devices (e.g., advertiser device 122 and user device 124) may include or may execute various operating systems, including a personal computer operating system, such as a Windows, iOS, or Linux operating system, or a mobile operating system, such as an iOS, Android (Android), or Windows mobile operating system, among others. The client device may include or may execute a variety of possible applications, such as a client software application capable of communicating with other devices, such as transmitting one or more messages (e.g., via email, Short Message Service (SMS), or Multimedia Message Service (MMS)), including via a network, such as a social network, including, for example, Facebook, linkedln, Twitter, Flickr, or Google (Google +), to provide just a few possible examples. The client device may also include or execute an application to transfer content, such as, for example, textual content, multimedia content, and the like. The client device may also include or execute an application that performs various possible tasks, such as browsing, searching, playing various forms of content, including locally stored or streaming video, or gaming. The above is provided to illustrate that the claimed subject matter is intended to include a wide range of possible features or capabilities.

Fig. 2 shows a flow of content items and data items displayed on a selected user device. In FIG. 2, the display advertisement 202 is illustrated as being displayed on a variety of displays, including a mobile network device display 204, a mobile application display 206, and a personal computer display 208. The mobile network device display 204 may be shown on the display of a mobile handset (e.g., a smartphone). The mobile application display 206 may be shown on a display screen of a portable device, such as a tablet computer. The personal computer display 208 may be displayed on a display screen of a Personal Computer (PC).

An example of a display ad 202 formatted for display on a user device, but not as part of a stream, for presenting the content of such a display ad is shown in fig. 2. The display advertisement 202 includes text 212, a graphical image 214, and a defined boundary 216. The display advertisement 202 is developed by an advertiser for delivery to a web page that is sent to a user device operated by a user. The display ad 202 may be placed in various locations on the web page. However, the defined boundary 216 and the shape of the displayed advertisement must match the space available on the web page. If the available space has the wrong shape or size, the displayed advertisement 202 may not be available.

To overcome these requirements and limitations, the display ad 202 may be reformatted or alternatively formatted for inclusion within the content item and a stream of ad items including a stream ad incorporating the content of the display ad 202.

In these examples, the display advertisement is shown as part of streams 224a, 224b, and 224 c. The streams 224a, 224b, 224c include a sequence of items that are displayed, for example, downward, item by item on web pages viewed on the mobile network device display 204, the mobile application display 206, and the personal computer display 208. The streams 224a, 224b, 224c may include any type of item. In the illustrated example, the streams 224a, 224b, 224c include content items and advertisement items. For example, stream 224a includes content items 226a and 228a and advertisement item 222 a; stream 224b includes content items 226b, 228b, 230b, 232b, 234b and advertisement item 222 b; and stream 224c includes content items 226c, 228c, 230c, 232c, 234c and advertisement item 222 c. Each of the streams 224a, 224b, 224c may include any number of content items and advertisement items. In one embodiment, the streams 224a, 224b, 224c may be arranged to appear to the user as an endless sequence of items, such that when the user of the user device on which one of the streams 224a, 224b, 224c is displayed scrolls the display, the endless sequence of items appears to appear in the displayed stream.

Content items located within any of the streams 224a, 224b, 224c may include news items, business related items, sports related items, and the like. Further, the content items in any stream may include other data, such as audio and video data or applications, in addition to textual or graphical content. Each content item may include text, graphics, other data, and links to additional information. Clicking on or otherwise selecting the link redirects the browser on the user device to a web page called a landing page containing the additional information.

Streaming advertisements like advertising items 222a, 222b, and 222c can be inserted into the content stream to complement the sequence of related items, providing a more seamless experience for the end user. Similar to the content items, the advertisement items may include textual or graphical content as well as other data such as audio and video data or applications. Each advertisement item 222a, 222b, and 222c may include text, graphics, other data, and links to additional information. Clicking on or otherwise selecting the link redirects the browser on the user device to a web page known as a landing page.

Although the example streams 224a, 224b, 224c are shown with a single visible advertising item 222a, 222b, 222c, respectively, any number of advertising items may be included within the stream of items. Traditionally, it is known to position advertising items at fixed locations. For example, in one conventional system, it is known to position the advertising item at the third item in the stream, starting from the top, at the sixteenth item in the stream and after every thirteen item in the stream. That is, in this conventional system, the advertisements are located within predefined locations (slots) in the stream. The targeting of the advertisement is the same for all users under all conditions. In this regard, the advertisements and content items are supplements within the stream. If a content item is not placed at a specified location in the stream, an advertisement is placed within that location.

In accordance with one aspect of the illustrated embodiment, the positioning of advertisements in a stream is made dynamic. Any position in the stream is subject to competition between the advertising item and the content item. A score is determined for each item. The scores of the advertisement items and the content items are matched such that the advertisement items and the content items can be ranked relative to each other and the ranking is used to populate the stream. Techniques for ranking advertisement items with content items are discussed in further detail below.

FIG. 3 is a flow diagram illustrating one embodiment of a method for ranking and displaying advertisement items and content item streams in an online information system. The method of FIG. 3 may be performed by, for example, elements of the online information system 100 of FIG. 1 including the account server 102, the search engine 106, the advertisement server 108, the content server 112, and the ranking engine 114. In other embodiments, other components may participate in performing the method of fig. 3, and some of the steps shown for the method of fig. 3 may be omitted or reordered, and different steps may be added or substituted.

The method begins at block 300. At block 302, the method loops to receive a web browsing request. The web browsing request is a data communication received over a network, such as network 120 of fig. 1. The data communication includes data specifying a web page to be browsed. For example, the web browsing request may specify a Uniform Resource Locator (URL) of an online provider (e.g., yahoo). The requested web page is one that may be filled or partially filled by a stream that includes at least two different types of items. In the example shown here, the types of items included in the stream are content items and advertising items as generally shown in the example embodiment of FIG. 2. However, in other embodiments, other types of items may be provided in the stream, and the type of item or category of item may be selected according to any convenient or useful criteria. For example, instead of streaming content items and advertisement items as shown in FIG. 2, streams may also be streamed with sports-related content items and news-related content items. In another example, instead of scoring and ranking only two types of items, such as content items and advertising items, more than two types of items may be scored and ranked, including content items, CPC advertising items, and CPM advertising items. The method shown in fig. 3 can be extended to the widest variety of combinations.

After the page view request has been received, at block 304, the advertisement item and the content item are qualified such that only qualified items are the subject of further processing. In one example, the items selected for qualification are contained within the advertisement database 110 and the content database 114 of the online information system 100 of FIG. 1.

Qualification may be performed on any suitable basis using any suitable input. For example, the advertisement item and the content item may be qualified based on identification information of the user from whom the web browsing request was received. If the online information system already stores information of the interests and preferences of the identified user, this information may be used to qualify the advertising items and content items. Also, if the advertiser has specified targeting constraints, such as gender, age, and geographic location, those constraints may be applied to known information about the user for qualifying the advertising and content items. Still further, if the page request includes information specifying a device type or platform of the user device, the platform information may be used to qualify the advertisement item and the content item for further processing. Some content providers may limit the content items they will send to a particular platform or format content items into a particular format based on platform information. Similarly, some advertisers may direct a particular advertising item only to a desktop computer or tablet computer. Once the content item and the advertising item have been qualified, processing continues to block 306.

At block 306, a clickthrough score is calculated for each advertisement item and a clickthrough score is calculated for each content item. Clicktability is a measure of the number of clicks a given advertising or content item gets more than the average advertising or content item. In one example, clicktability is a function of: the number of clicks an advertisement item or content item receives for all users and the number of impressions or views an advertisement receives for all users, and the click-through rate (CTR) of an advertisement item or content item for all users. Clickability is location independent.

Click-through rate is defined as the ratio of the click-through rate received by an advertisement item or a content item to the number of impressions. Click-through rates may be dynamically determined using stored data, such as statistics about the performance of advertisements within an online information system. For example, each time a particular advertising item or a particular content item is displayed or viewed by an advertiser, the data item representing the viewed amount or number of impressions for that item is incremented. Similarly, each time a particular advertisement item or a particular content item is clicked on or otherwise selected by a user, the data item representing the click through or click through amount for that item is incremented. The data items may be stored in an advertisement database, a content database, or any other suitable storage device, such as advertisement database 110 and content database 114 of FIG. 1. Similarly, the mathematical processing may be performed, for example, by the ad server 108, the content server 112, or the ranking engine 116 of FIG. 1.

In one example, the click-through rate is defined as:

wherein, C_i,tRepresenting the number of click-throughs an advertisement item or a content item received at a particular location i within time t. Time t represents a discrete time increment, the width of which may vary. Typically, each increment of t corresponds to a second or minute. Position i refers to a position within the stream, where i-1 denotes the first position, i-2 denotes the second position, and so on. In one embodiment, V_i,tRepresenting the amount of browsing or number of impressions received by an advertisement item or content item at a particular location i within a time t. In another embodiment, V_i,tRepresents the sum of the number of clicks and the number of skips at time t position i. The number of skips for position i may be counted each time the user clicks on an item in a position below position i, or it may be adjusted by a factor to register a score count. For example, if the user clicks on the number i of 4, the number of skips is incremented by the number i of 1,2,3, while the click count is incremented by the number i of 4. The exponential term with the coefficient gamma (γ) specifies the decay rate, which is typically longer for advertising items than for content. These indices introduce temporal dependencies such that if an event, such as a click or browse, occurs recently in the past, it is given more weight than an earlier event.

In one embodiment, the coefficient gamma (γ) may be calculated or adjusted based on a periodic comparison of the click-through rate of a given item j at location i and time t +1 with the click-through rate of the same item j at location i and the immediately previous time increment (refCTR) and clickability — for example, by plotting refCTR (i) clickability (j, t) -CTR (i, j, t + 1). The reference CTR and clickable function are described further below. In another embodiment, gamma (γ) is calculated by summing the errors and then minimizing them.

γ＝∑V_i，t[refCTR(i)*clickability(j，t)-CTR(i，j，t+1)]²

To calculate the click-through rate, the following decomposition is performed for the advertisement item and the content item:

CTR (advertisement/content, user, location, configuration) ═ clicktability (advertisement/content, user) × reference CTR (location, configuration)

Thus, the click-through rate CTR is specified for a specific advertisement item or content item, a specific user, a specific location in the stream, and a specific device configuration. Examples of device configurations include handheld devices, tablets, and desktop computers. Other configurations and techniques may also be suitable and may be used to characterize CRTs or other user data. The relationship between clickthrough and click-through rate is specified by a reference curve, i.e. a reference CTR, which varies with the location of the content item or advertisement item within the stream and the user device configuration. Thus, clickability explicitly represents a location-independent CTR.

According to a corollary, the reference CTR represents the probability that a user will click on a particular advertisement item or a particular content item at a particular location i, independent of any impact on the desirable (or undesirable) click-through rate for the particular advertisement or content item. This reference CTR can be calculated by running a random bucket that displays the click-through rate for random advertisement items or content items across all locations, with the advertisement/content ratio being the same as within the main bucket.

It is desirable that the clicktability value of an advertisement item or content item be location independent. Clickthrough eliminates any effect of the location of the content item or advertisement item within the stream, but instead focuses on the quality of the content item or advertisement item.

Returning to step 306 of FIG. 3, the clickthrough score for item i in the stream can thus be calculated using the following general relationship.

However, in some embodiments, it may be desirable to estimate or measure clickthrough of items specific to a particular user or a particular market segment, which may be given any index tag j. In such an embodiment, clickability may be calculated based on relationships specific to the user or market segment j.

Thus, in embodiments, to determine clickthrough, one may sum the click volumes at all locations and divide by the product of the view volume or impression count at all locations and the reference CTR.

In some embodiments, personalization is then introduced as an additional factor that contributes to the overall click-through rate for a given item, user/zone, location, and device configuration.

CTR (advertisement/content, user/partition, location, configuration) — reference CTR (location, configuration) × clicktability (advertisement/content, user/partition) × affinity (advertisement/content, user/partition).

For content items, the affinity between the user and the content, normalized by the number of clicks observed from a reference user interval, is estimated from historical data using the number of clicks observed for the user for content items having similar content characteristics (where similarity is determined, for example, using algorithmic analysis of similarity to context or known content classifications). In one embodiment, this uses naive Bayes (A), (B), (C), (DBayes) approximation. Many months of user history data may be used to reliably estimate this affinity of the content item.

For advertising items, the affinity between the user and the advertisement may be more difficult to estimate from historical data. The user's purchase intent with respect to the advertised product or service is likely to be less long than the user's general interest in the content item (e.g., news story). Thus, a shorter history window may be used for advertising items. An exponential change using gamma (γ) is introduced to adjust the time window for which the quantity is being calculated. The historical data for advertising items also tends to be much more sparse. In some embodiments, search history profiles, email, or other application activities may be used to expand the user behavior pool. Regardless of the data source, once the data is collected, affinity scores are computed by plotting the user data and items within a high-dimensional vector space defined by identifying features within an existing content network or taxonomy. A description of such a rendering is provided in co-pending U.S. patent application entitled "Method and System for Multi-Phase Ranking for Content Personalization" 13/839,169 and "Method and System for building a User Profile by Mapping Third Party interests into a common Space of Interest" (Method and System for User Profiling Via Mapping Third Party Interest a Universal Interest Space) "13/837,357, the entire contents of both documents being incorporated herein by reference.

After clickthrough is calculated for each advertisement item and each content item at block 306, a satisfaction score is calculated for each advertisement item and each content item at block 308. Satisfaction may also be referred to as post-click satisfaction and is defined by some measure of user satisfaction after the user has interacted with the advertisement item and the content item. Clickability generally measures only the user's propensity to click on advertisement items and content items. Satisfaction assigns a numerical value to the user's likelihood of returning to an online provider or marketplace based on the user's overall experience. In one example, the satisfaction may be set to a value between 0 and 1, where 1 indicates that the user is fully satisfied and willing to come back, and 0 indicates that the user is fully not satisfied and the lost user is unlikely to come back any more.

In some examples, satisfaction with an advertising item may be calculated using a conversion data. However, such data is relatively sparse and may not be reliably comparable across advertising items. In another example, dwell time may be used as a measure of satisfaction. The dwell time is an indication of the amount of time the user browses the clicked advertisement after clicking on the advertisement item and browses the content after clicking on the content item. In some embodiments, the satisfaction may be set to a constant value (e.g., 1.0) for the content item, the advertisement item, or both when insufficient data makes the estimate unreliable.

In other embodiments, additional factors for popularity may be included in the CTR model:

CTR (advertisement/content, user/partition, location, configuration) — reference CTR (location, configuration) — clicktability (advertisement/content, user/partition) — affinity (advertisement/content, user/partition) — popularity (advertisement/content, user/partition).

The popularity score reflects a measure of the total interest in the particular item within the time window. The popularity score may be calculated, for example, based on a simple ranking of the highest click-through rate of the advertisement or content within the time window. The ranking may then be normalized or adjusted by a coefficient reflecting the relative importance of popularity to total CTR.

At block 310, a bid amount for each content item is calculated. For advertising items, the bid amount is set by the advertiser and stored, for example, within the advertisement data, along with the remaining data defining the advertisement. For content items and advertising items to be ranked together for inclusion within a stream, there must be a simulation of the bid amount for the content item for the advertising item. In some embodiments, if the content providers participate in the unified marketplace, they may provide a bid amount. However, in other embodiments, there may be no content bids. The automated method may not generate bids that allow content items to be ranked together with advertising items until publishers and other content providers explicitly bid to position their content items within the stream. In other embodiments, the content bids may reflect actual amounts paid to content contributors who are partners with the online provider.

One example for determining a content bid is shown herein. For each user or user zone, the following table 1 may be constructed. Here the user zones may be based on a user targeting profile (i.e., a combination of user zones that advertisers may bid). That is, if data for a given user is not available, the data for the table may be obtained at a general level based on the user separation.

TABLE 1

Table 1 stores content quality scores and percentile (percentile) values of historical bids for advertisements. These percentiles are not weighted for impression. Thus, the percentile distribution of quality scores across a particular set of content items is just like the distribution of bids across a particular set of advertisements. The percentile score may be obtained by selecting all content items within the advertisement database that are qualified for the user, ranking the selected content items by quality score. The ranked selected items are then separated according to their tenth and twentieth percentiles. Thus, the table translates the quality scores of the content into bids for the content. In this way, the quality scores are independent of the bid data for the content items and the advertising items. The quality scores of the content and the bid data of the advertisements are paired or associated based only on the relative percentiles.

In other embodiments of the unified marketplace, advertisements and content may be paired, associated, or synchronized in other ways. For example, ads and content may be bundled based on similar topics to fit the bundle's estimated total clickthrough, which is calibrated to protect the quality of the end user experience.

In one embodiment, the quality scores entered in Table 1 are a function of clickability and post-click satisfaction. One technique for calculating a mass fraction is described below in conjunction with fig. 4. The historical period of data may be limited to'd' days (where'd' is an external parameter). The rightmost column of Table 1 carries the conversion of historical bids. An exemplary conversion function of bid amounts that may be used is as follows:

where M ∈ (0.. M), where M is an input parameter. The value m <1 gives an advantage to the content item and the value m >1 gives an advantage to the advertisement item. Typically, m will have a value of approximately 1. The online provider may update the data in table 1 daily.

After a content item has been selected for a stream, when a page view event is performed, step 310 of the exemplary method of FIG. 3 includes determining a quality score, q, by comparing the quality score to a threshold value_cServing as a clue to the table to look up each content item from the table. If q is found_cThen the corresponding bid b is looked up from the third column_a. If q is not found_cThen the more highly scored entry in the table, denoted as q, is found immediately_c ^h. Then find the corresponding q in the table_c ^hBid b of_a. The converted historical bids can then be calculated as follows: b_c＝k*f[b_a]Where k is an external parameter.

Accordingly, bids for the advertising item are used to determine a bid for the content item. The present technique is merely exemplary, but it ensures that the automatically determined bids for content items are commensurate with the bids for advertising items, and that each content item will achieve a unique bid. These benefits are important for ranking content items for inclusion in a stream. Further, the bids for content items are proportional to their quality scores. Thus, only high quality advertising items with high quality scores will rise to the top of the stream, and at the same time content items with low quality scores will not replace high quality advertising items. Since the content bid is proportional to the quality score, it reflects both the long term user value as well as the instant short term revenue. Still further, the content bids are proportional to the user's monetary value, as the advertiser's bid can be safely considered to reflect the user's value. Finally, the technique shown is adaptive. That is, as the bid for an advertising item increases, the bid for the content item will also increase. To some extent, this will cause the advertiser to bid higher.

The external parameters k and m may be selected in any suitable manner. In one example, these parameters may be selected such that most of the time the content item gets the top position in the stream, the advertisement items are interspersed within the content item without being aggregated together. Also, after the initial start-up phase, the market should achieve a certain level of stability. This implies that advertisers should not have to continually raise their bids for advertising items to maintain their position. One exception to this rule is when the market is growing and new advertisers are entering the market.

At block 312, a ranking score is calculated for each advertisement item and a ranking score is calculated for each content item. In one embodiment, the content item is presented, for the content item,

ranking score_c＝b_c*c_c*s_c

Wherein b is_cCalculated bids for content items, c_cBeing clickable of content items and_cis a satisfaction score of the content item. In one embodiment, the advertisement is presented for an advertising item,

rank score b_a*c_a*s_a

Wherein b is_aA bid specified by an advertiser for an advertising item, c_aBeing clickable of an advertising item and_ais a satisfaction score for the advertising item.

After the ranking scores are computed at block 312, the advertisement items and content items are ordered at block 314 using the computed ranking scores. Because the ranking scores have been calculated using similar values, the ranking scores are commensurate and can be reliably staggered. The result of the ranking is a miscellaneous list of advertisement items and content items.

After the sorting step, the miscellaneous list may be used to fill the stream. However, in some embodiments, at block 316, it may be preferable to process the miscellaneous list for diversity. Diversity is applied to the content to prevent too many similar content items from being located in proximity to each other. One example is news articles on the same topic, which may be ranked close to each other. Only content items are affected by the diversity process and this will cause some content items to drop in the ranking.

One example of a diversity algorithm includes first scoring by rank (b)_c*c_c*s_c) The content items are ordered such that position 1 corresponds to the highest score. Second, content items at positions 2-N (where N is in the range of 200-300) are compared to content items at position 1. Third, if any item at location 2-N matches any feature of the item at location 1, then the ranking score of the item is multiplied by a diversity score, such as the range [0-1 ]]Depending on the type of feature. Fourth, content items at positions 2-N may be ordered relative to content items at position 1 using a new ranking score that is penalized by diversity. Fifth, steps two through four may be repeated for content items at locations 3-N compared to content items at location 2. And sixth, step five may be repeated for content items at locations 3-N, for example a minimum of 20 times.

In addition to processing the content items for diversity, specific rules may also be applied to the advertising items after the sorting step. For example, in one embodiment, a rule like a guardrail may be applied, preventing advertising items from being positioned in positions 1 and 2 of the stream. If the advertising item is in either of the first two positions after the sorting process, the advertising item may be moved to a lower position, such as position 3. In another embodiment, rules are established that hold at least 9 content items between each advertisement item in the stream. If less than 9 content items are present between two advertising items, the lower ranked advertising item may be moved to a lower position. Other values may be selected for these rules, or other rules may be established.

At block 318, pricing is calculated for the advertisement items and the content items. Pricing refers to an amount charged to an account of an advertiser associated with an advertised item in response to a user selection of the advertised item, such as by a click. Similarly, pricing of a content item refers to an amount charged to an account of a publisher or content provider associated with the content item in response to a user selecting the content item, such as by clicking through the content item.

In one embodiment, a rule called generalized second-order price (GSP) is employed for block 318. Under this rule, if a user clicks on or otherwise selects an advertising item, the advertiser associated with that advertising item will be charged an amount equal to the minimum bid required to win that position. Specifically, let i denote the advertisement item a under consideration, and let i +1 be a content item or an advertisement item in a position below the advertisement item a. The price that would be charged (if clicked) to the advertiser associated with advertising item a would then be item i r_iThe reserve price and the maximum of the product of the bid, clickable, and satisfaction for the item at i +1 and the quotient of clickable and satisfaction for the item at i.

Price p_iAlways less than the initial bid b; because b is_i+1*c_i+1*s_i+1<b_i*c_i*s_i. Price p_iThere is also a desirable feature that prohibits the advertiser's follow-up repentance. I.e. if the advertiser would otherwise choose to be less than b_iThe advertiser will still win the same position. As long as the advertiser's bid is greater than p_iThis is true. If the advertiser is charged the bid amount b_iThen after that fact, the advertiser will regret without a bid p_iPlus one cent. Alternatively, if we charge p from the advertiser_iInitially, the motivation to guess the system is reduced and advertiser satisfaction is improved. However, the advertiser will still need to check if a different location will be more appropriate. Periodic bid adjustments are suggested.

In some embodiments, only cost-per-click (CPC) bids are accepted for advertising items and content items. However, in other embodiments, a cost-per-impression (CPM) bid may also be accepted. When accepting a CPM bid, the desired bid allocation for a location is required to place an advertisement with a higher expected revenue in a higher location, where a CPC advertisement a placed in location k is considered to have an expected revenue b_a*c_aReference ctr (k). When all bids are CPC bids, by according to b_a*c_aSimple ordering of the advertisement items achieves this allocation goal, since the reference ctr (k) is unrelated to the advertisement and can therefore be ignored. This makes the calculation of the allocation of bids relatively fast and straightforward.

On the other hand, revenue for a CPM advertisement does not depend in any way on its clickthrough or its location. Advertisers pay their bids each time an advertisement is displayed. Simply ordering the CPC and CPM bids according to bid value (or bid amount multiplied by clickability) will no longer satisfy the characteristics of higher position in the higher revenue bid acquisition stream.

However, the following method can be used to accomplish joint ranking of two types of advertising items in multiple locations in one pass, still meeting the revenue requirements described above, and without slowing down the computation. Therefore, there is no need to have any technical limitation on the permission to provide both CPM and CPC advertisements.

The proposed algorithm is as follows:

inputting:

1. ordered CPC list (in bid x clickable order), x1> x2

2. Reference CTR (location dependent, advertisement independent) a1> a2>.

3. The ordered CPM list (in bid order), Y > Y2

For each position k (starting with the highest position 1),

let X be the highest ranking CPC advertising item (which has not yet been assigned)

Let Y be the highest ranked CPM advertisement item (which has not yet been assigned)

If X a_k>Y, then X is assigned to position k, otherwise Y is assigned to position k.

Pricing techniques may also differ for the inclusion of content for CPM advertisements. For a CPM advertisement, the price is set to (bid for the CPM advertisement item or underlying content item or underlying CPM advertisement). For the CPC advertisement, if the advertisement item is below the CPC, the price is set to (bid of the following advertisement item or content item ×/clickable of the CPC advertisement item), or if the following advertisement is the CPM, the price is set to (bid of the following advertisement item/reference CTR)/(clickable of the CPC advertisement item).

This technique meets the above-described revenue ranking requirement. Run time and latency are not affected because here sort time is the dominant factor and even sorting is done for CPC-only bids.

An example process for displaying content in a streaming media feed according to quality scores is shown in fig. 4. An example process for determining a quality score is also presented. In one example, a user of an electronic device (e.g., a mobile device) may be browsing content in a streaming media feed, such as a news article. The content (e.g., news articles) may be interleaved with advertisements in a feed as in the exemplary illustration of fig. 2. For example, a user may browse a news article, and while browsing the article or after, the user may request a new article; however, before the next article, an advertisement may appear in the feed. In general, the order in which advertisements or articles appear in a feed may be arbitrary or determined by factors such as quality scores.

In one example, the ranking engine 116 of the online system 100 implements a Quality Scoring System (QSS), the operation of which is schematically illustrated in FIG. 4. The processor of the ranking engine 115 cooperates with data stored in, for example, the advertisement database 110 and the content database 114 to perform the following data processing operations of the QSS.

The QSS receives information (designated as current content 402 in fig. 4) such as advertisements or articles to be scored by the QQS. The current content 402 may be received from a content source over a network, such as the internet. The QSS may include an interface, such as an optical transceiver or an electrical transceiver, configured to receive data defining the current content 402 from a streaming media feed or any other type of online feed. The content source may be any source of advertising or multimedia content, such as a network of servers hosting content, configured to feed online content.

When current content in a streaming media feed is received, for example, by an interface of a QSS, a processor of the QSS communicatively connected to the interface may determine or be configured to determine a first probability at 404 that a user will select to interact with a current content item in the streaming media feed. In one example, the current content item may be an advertisement item or a content item as described and illustrated herein, for example, in connection with FIG. 2.

Regarding the determination of the first probability at 404, selecting to interact with the current advertisement or article may include clicking on the advertisement or article or clicking on a hyperlink of the advertisement or article. Further, such selection may include a gesture made with respect to the advertisement or article or a gesture made with respect to a hyperlink of the advertisement or article.

As shown, the first probability may be based on data 406 corresponding to characteristics of the current content item that are associated with previous interactions and/or preferences of the user with content similar to the current content. These interactions and/or preferences may come from user profiles stored in a database, such as database 408. The user profile may include, for example, parameters associated with the user, for example, regarding browsing of streaming media content, and the data 406 may be received by the processor from the database 408 of the QSS.

Upon receiving the current content, the processor may determine or be configured to determine a second probability 410 that the user will select to interact with the content (e.g., advertisements or articles) in the streaming media feed. Data 412 used as input to determine the second probability may be received by the processor from the database 408 of the QSS.

The processor may also determine or be configured to determine an affinity score for the current content. The affinity score is a relationship between the first probability determined at 414 and the second probability determined. For example, the relationship may be represented by a ratio or another type of numerical score. Also, the affinity score may represent a relevance between the user and the current content item based on characteristics of the current content item that match user profile parameters associated with the user.

The affinity score may be independent of the configuration of the current content item in the streaming media feed. For example, the affinity score may be independent of the location of the current content item in the streaming media feed. In other words, the affinity scores may not take into account the order in which the current content item appears relative to other content items in the streaming media feed.

In one example of the feed, the current content item may be a first content item, such as an advertisement, and the second content item may be an article, such as a news article. In this case, the first and second content items may comprise categorically similar topics, and the determination of the affinity score may comprise determining an affinity score for the second content item based on the affinity score of the first content item. In such an example, the similarity in the classification of topics may be identified by matching metadata elements embedded in the first and second content items. Matching metadata elements may include matching categories of content classifications and/or correlations with web pages in a series of web pages provided by a network content provider. In one example, a web page in a series of web pages provided by a website content provider may be an online encyclopedia or dictionary entry, such as a WIKIPEDIA (WIKIPEDIA) entry.

In another example of the feed, the determination of the affinity score may include a determination of an affinity score between a user device (e.g., a mobile device) and the current content. For example, this affinity score between the user device and the current content may be based on text in the current content and the current geographic location of the mobile device. This affinity score may also be based on any other attributes of the mobile device (such as the telecommunication service operator associated with the device) and any other attributes of the current content (such as the video element of the current content).

In this other example, the processor may determine or be configured to determine the first probability based on a probability that a user of the device will select to view the current content item based on characteristics of the current content item matching profile parameters associated with the user device and/or the user. With respect to the second probability, the processor can determine the second probability based on a probability that a user using the device will select to browse content items in a typical streaming media feed. Finally, the processor may determine an affinity score based on a relationship between the determined first probability and the determined second probability.

In yet another example, the processor may determine or be configured to determine the first or second probability using machine learning techniques. For example, the processor may use a boosted decision tree or another form of artificial intelligence to determine the first or second probabilities.

The processor may also identify or be configured to identify data 416 corresponding to post-interaction satisfaction with previous content, such as data regarding post-interaction satisfaction with previously viewed advertisements that are categorically matched with current content. The data 416 may then be derived as post-interaction satisfaction scores. Categorical matching may include matching by category of content categorization and/or web page relevance in a series of web pages provided by a network content provider, such as a provider of an online encyclopedia or dictionary. Data 416 may include data associated with posts (including social media posts) about current content or similar content. The data 416 may also include data regarding mouse clicks on content or similar content or links to such content. And, data 416 may include data regarding: the amount of browsing of such content, the length of browsing such content, the amount of registration or subscription to browse such content, the amount of sharing of such content to other users, and the linking of such content by the user's own content.

Based on the affinity score and the post-interaction satisfaction data 416 or post-interaction satisfaction score, the processor may determine or be configured to determine a quality score at 418. For example, the quality score may be determined by calculating the product of the affinity and the post-interaction satisfaction score.

When the quality score is determined, the processor may use the score as a basis for displaying and/or configuring the current content for the feed, such as the order in which the current content is displayed relative to the other content in the feed. Also, for example, the quality score may be displayed or used to generate a report for an administrator.

As can be seen from the foregoing, the present disclosure provides various techniques for online providers to control the location, quantity, and density of stream advertisements within a content stream viewable by a user on a web page. Streaming can be viewed as a unified marketplace, where both content items and advertising items compete for placement or are included in the stream. Scoring and ranking techniques allow for a commensurate ranking of both content items and advertising items. Additional business rules for the content items and the advertising items may further control the relative positions of the content items and the advertising items in the stream.

The disclosed methods and systems may be implemented in part within a server, a client device, a cloud computing environment, in part within a server and in part within a client device, or a combination of a server, a cloud computing environment, and a client device.

It is therefore intended that the foregoing detailed description be regarded as illustrative rather than limiting, and that it be understood that it is the following claims, including all equivalents, that are intended to define the spirit and scope of this disclosure.

Claims (18)

1. A system for providing a unified marketplace of advertisements and content, comprising:

a processor and a memory;

an advertisement storage device configured to store data defining a plurality of advertisement items originating from advertisers;

a content storage device configured to store a plurality of content items originating from a source other than an advertiser; and

a server system in data communication with the advertisement storage device and the content storage device and configured to display advertisement items retrieved from the advertisement storage device and content items retrieved from the content storage device as streams having graphical or textual items arranged in order on a web page and viewable by a user on a user device, the server system being further configured to determine a corresponding bid amount for each advertisement item and a corresponding content bid amount for each content item to enable ranking of each advertisement item and each content item together, the server system being further configured to calculate a corresponding ranking score for each advertisement item using the determined corresponding bid amounts and to calculate a corresponding ranking score for each content item using the determined corresponding content bid amounts, the server system is further configured to order the ranked advertising items and the ranked content items together in the stream using the corresponding ranking scores computed for each of the plurality of advertising items and each of the plurality of content items, the server system further configured to transmit a web page including the stream to a user device over a network.

2. The system of claim 1, wherein the server system is further configured to calculate a respective ranking score for each of the plurality of advertising items as a product of at least a bid amount and a clickable score that reflects a likelihood of being clicked on by a user of the user device, regardless of a location occupied within the stream.

3. The system of claim 2, wherein the server system is further configured to calculate the corresponding ranking score as a product of the clickable scores using a reference click through value.

4. The system of claim 2, wherein the server system is further configured to calculate the corresponding ranking score as a product of a bid amount, a clickable score, and a satisfaction score for each advertising item.

5. The system of claim 4, wherein the server system is further configured to determine a quality score for each content item and use the quality score to determine a corresponding content bid amount for each content item.

6. The system of claim 5, wherein the server system is further configured to use the quality score to determine a percentile score and to use the percentile score to select a bid amount for an advertising item having the same percentile score as a corresponding content bid amount for each content item.

7. The system of claim 2, wherein the server system is configured to detect selection of a ranked advertising item at the user device and, in response, charge an account of an advertiser associated with the selected advertising item an amount equal to a bid amount of a next lowest ranked content item or advertising item in the stream.

8. The system of claim 1, further comprising:

an account database configured to store account data defining account information for a plurality of advertisers associated with the plurality of advertising items; and

an account server in data communication with the account database and the server system, the account server configured to respond to a detected selection at a user device of a ranked advertising item in the stream by: an account of the advertiser associated with the selected advertising item is charged an amount equal to the bid amount for the next lowest ranked content item or advertising item in the stream.

9. The system of claim 1, wherein the server system is further configured to:

calculating clickable scores and post-click satisfaction scores of each content item and each advertisement item;

calculating a corresponding ranking score for each advertising item and each content item using the calculated clickable scores and the calculated post-click satisfaction scores for each advertising item and for each content item using the corresponding content bid amounts determined for each content item and using bid amounts specified by advertisers for each advertising item; and

using the calculated ranking scores for each advertising item and each content item, positioning individual advertising items having scores greater than the processed content item ahead of the processed content item;

the individual advertisement items and the individual content items are displayed as positioned using the calculated ranking scores.

10. A method for providing a unified marketplace of advertisements and content, comprising:

in the case of a server system, it is,

retrieving a stored advertising item from an advertising storage device, the advertising item originating from an advertiser;

retrieving a stored content item from a content storage device, the content item originating from a source other than an advertiser;

determining a corresponding content bid amount for each retrieved content item to enable ranking of each retrieved advertising item and each retrieved content item together; determining, by the processor, a content ranking score for each retrieved content item using each content bid amount;

determining, by the processor, an advertisement ranking score for each retrieved advertisement item using each advertisement bid amount;

ranking, by the processor, each retrieved advertisement item and each retrieved content item using the content ranking score and the advertisement ranking score;

formatting a web page with a sequence of items that are visible when the web page is displayed to a user device, the sequence of items including ranked advertising items and ranked content items that are accordingly located on the web page in the sequence of items according to a ranking; and

the formatted web page is transmitted to the user.

11. The method of claim 10, wherein ranking each retrieved advertising item and each retrieved content item comprises ranking each retrieved advertising item and each retrieved content item

Scaling each retrieved advertising item to determine a corresponding advertising score for each retrieved advertising item;

scaling each retrieved content item to determine a corresponding content score for each retrieved content item;

ranking each retrieved advertisement item and each retrieved content item on the web page using the corresponding advertisement score and the corresponding content score.

12. The method of claim 10, wherein formatting the web page comprises: arranging a stream of advertisement items and content items on the web page, ordering respective retrieved advertisement items and respective retrieved content items according to the ranking.

13. The method of claim 10, further comprising:

determining, for each retrieved advertisement item, a corresponding advertisement score using the corresponding advertisement clickable value;

determining, for each retrieved content item, a corresponding content ranking score using the corresponding advertisement clickable value; and

wherein ranking each retrieved advertisement item and each retrieved content item comprises using the determined corresponding advertisement score and the determined corresponding content score.

14. The method of claim 13, further comprising:

determining, for each retrieved advertisement item, a corresponding advertisement clickthrough value as a ratio of a number by which each retrieved advertisement item will receive more clicks to an average number of clicks received by an inventory of advertisement items; and

for each retrieved content item, a corresponding content clickthrough value is determined as a ratio of the number of clicks that each retrieved content item will receive more to the average number of clicks received by the inventory of content items.

15. The method of claim 13, wherein determining a corresponding content score comprises determining the corresponding content score as a product of a clickable score and a post-click satisfaction score.

16. The method of claim 14, further comprising:

determining a dwell time for each retrieved content item; and

determining the post-click satisfaction score using the dwell time.

17. The method of claim 10, further comprising:

aiming at each retrieved advertisement item, determining a corresponding advertisement bid amount, an advertisement clickable score and an advertisement satisfaction score;

determining the corresponding advertisement ranking score as the product of the corresponding advertisement bid amount, the corresponding advertisement clickable score, and the corresponding advertisement satisfaction score;

determining a corresponding content clickable score and a corresponding content satisfaction score for each retrieved content item;

determining the corresponding content ranking score as the product of the corresponding content bid amount, the corresponding content clickable score, and the corresponding content satisfaction score; and

ranking the respective retrieved advertising items and the respective retrieved content items by comparing the determined respective advertising scores to the determined respective content scores to order the respective retrieved advertising items and the respective retrieved content items on the web page.

18. The method of claim 17, further comprising:

determining a corresponding advertisement score for each retrieved advertising item using the bid amount associated with each advertisement;

ordering each retrieved advertising item and each retrieved content item in a stream on the web page using the corresponding advertising score and the corresponding content score;

detecting click-throughs for each advertisement by a user browsing the webpage; and

the advertiser's account associated with each advertisement clicked on is charged an amount equal to the bid amount for the next lowest ranked content item or advertisement item.