WO2001018684A2 - System and method for profiling a web site - Google Patents

Abstract

A system and technique for profiling a website is provided. Users who wish to make a purchase consult commercial websites on the Internet. Following these consultations, information is transmitted to a profile server which creates a site profile, or site map. This site profile includes information on the layout of the various web pages, as well as instructions for navigating the pages of that website. The interaction between the user and the various web pages informs about the actions of the user. For example, if a frame known to be a name frame (based on information on the web profile) has data, it can be assumed that it is the user's name, in which case this information should be stored. So when the user interacts with other web pages, and if information on the user's profile and information on the site profile is available, this information can be entered automatically in the appropriate boxes. As it is stored in a profile server, the information (including scripts for interacting with the site) is available to other users.

Description

SYSTEM AND METHOD FOR PROFILING A WEB SITE

FIELD OF THE INVENTION

The invention relates generally to electronic commerce. More particularly, the invention relates to systems and methods for creating and using merchant profiles.

BACKGROUND OF THE INVENTION

Merchants often desired a profile of consumers on their buying habits and product preferences. In the past, consumer profiling has been solved in the brick and mortar context by having customers complete surveys, as well as by tracking inventory to determine buying habits and product preferences. However, for convenience, consumers have continually sought alternatives to the standard brick and mortar stores for their purchases. For example, consumers often make purchases by telephone or mail order. Most recently, consumers have turned to the Internet as an alternative method of making purchases. Merchants selling over the Internet also desire a mechanism for gathering information about consumer buying habits. Conventional techniques present a "preferences" screen to the user for completion. The consumer completes this screen prior to purchase or as part of a merchant poll or survey or promotional "give-away". Using techniques, such as surveys, to gather this information presents several problems. Most people do not like to spend the time required to complete these surveys. Thus, the survey is not representative of consumers as a whole. For those consumers that do complete the surveys, there is no way of validating the truth of the responses. Thus, a need exists for a system and method for gathering preference information as an additional element of the purchase experience. Such data illuminates what customers do, not what they might indicate in a survey or a poll. Moreover, in order to attract more customers and to increase customer spending, merchants seek ways to increase the ease of use of their sites, in particular, the specific web pages that support customer purchases. Conventional techniques allow customers to navigate through one or more of a series of Web pages and information is requested and entered in appropriate form fields (e.g., product identification, quantity, shipping address, etc.). Although the functions performed at merchant sites are very similar, the format of the page layouts (e.g., forms) which require data entry can be vastly different. Navigation between pages, and the collection of pages that make up the site also vary greatly even when they support nearly identical functions, thereby rendering automated mapping of a site difficult. Thus, there is a need to simplify the ways in which users interact with sites through automated and semi- automate mapp ng o s tes e.g., to create a pro i e o a merc ant s te . Ven ors ex st t at se information about merchant sites. For example, Globeset Corporation of Austin, Texas, offers a subscription of profiles on merchants. However, purchasing of subscriptions limits the merchant profiles available to those provided by the subscription vendor. Thus, a need exists to profile any desired merchant site, e.g., those actually used by consumers who will make use of the merchant profiling system.

SUMMARY OF THE INVENTION

In exemplary embodiments of the invention, users view pages from merchant Web sites over the Internet to make a purchase. As the user views the Web pages, information is sent to a profile server which creates a site profile or a site map. The site profile includes information about the layout of the various Web pages, as well as information about how to navigate among the various pages within the Web site.

As the user interacts with the Web pages, information about the user is obtained from the user's actions. For example, if a field known to be a name field (by the site profile information) has data entered into it, it can be presumed that this is the user's name and such information about the user is also be stored. As such, when the user interacts with other Web pages, if site profile information is available and user profile information is available, user data can automatically be entered in the appropriate fields.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention are hereinafter described in the following detailed description of illustrative embodiments to be read in conjunction with the accompanying drawing figures, wherein like reference numerals are used to identify the same or similar parts in the similar views, and:

Figure 1-3 are block diagrams of various embodiments of an exemplary merchant profiling system;

Figures 4-5 are block diagrams illustrating exemplary embodiments of the Merchant Profile Server shown in Figures 1-3; Figure 6 is a flow diagram illustrating exemplary logic performed on a user computer to profile a site in accordance with the present invention;

Figures 7-10 are flow diagrams illustrating exemplary logic for performing merchant profiling in accordance with the present invention.

Figures 11-20 illustrate exemplary data structures for a site model; gures - i us ra e exemp ary a a s ruc ures or scπp a e in erac ions, Figures 25-28 illustrate exemplary data structures for an exemplary user profile, and Figures 29-34 illustrate exemplary displays for a user interface for initiating site profiling in accordance with the present invention

DETAILED DESCRIPTION OF PREFERRED EXEMPLARY EMBODIMENTS

In general, the present invention performs site profiling by storing information about

Web pages within a Web site and capturing information when a user interacts with the site For example, the site information (or site map) includes information on various form fields on a page within the site (for example, name field, address field, e-mail address field, etc ) and the site information is stored on a server so that it is available to other users As a user enters information into form fields, the user information (e g , user's name, address, e-mail address, etc ) is also stored (e g , a user profile is created) The user information can then automatically be entered into appropriate fields of sites that have been profiled In exemplary embodiments described herein, the sites profiled are merchant sites However, it will be appreciated that profiling can be performed on virtually any type of Web site, database or network

An exemplary merchant profiling system of the present invention creates merchant profiles based on user interactions with sites, wherein the user interactions are typically purchases Merchant profiles describe a merchant's site such as, for example, the format of a checkout page When a customer makes an online purchase, information about the merchant site is stored and this information can be made available to other users Information about a user is also stored and by having information about the user, as well as information about the merchant site, much of the information that a user is normally required to enter can automatically be entered for the user The exemplary system thereby decreases the amount of time that a user must spend in conducting an online purchase, which will in turn increase the likelihood that the user will make online purchases

Figure 1 is an exemplary block diagram illustrating an exemplary merchant profiling system 100 formed in accordance with the present invention A user or customer 120 communicates with a merchant 125 over a network 130, such as the Internet A Merchant Profile Server 110 communicates with one or more customers 120 over a network 130, such as the Internet While the embodiment illustrated in Figure 1 employs the Internet as the communication medium, it will be appreciated that other communication media are also possible, for example, digital wireless media It will be appreciated that the system includes a plurality of users and a plurality of merchants Moreover, the customer mteraction with the merchant is known in the art For examp e, a customer 110 uses a We rowser, suc as Netscape Navigator® or Microsoft Internet Explorer® to navigate to a merchant Web site The Web browser contains a plug-in or similar component that can interact with the browser as well as the Merchant Profile Server 110 The component may include, for example, ActiveX, Java, a browser-specific plug-in, scripting, or server-side components The user navigates through the merchant's site performing various functions, for example, viewing product descriptions, placing products in a shopping cart and proceeding to a checkout to purchase the products in the shopping cart

As shown in Figure 2, various embodiments may also include a Partner Server 140 in addition to the components illustrated in Figure 1 The Partner Server 140 exchanges data with the Merchant Profile Server 110 and uses merchant profile data to support needs of partner customers Needs of partner customers may be similar to the needs of customer 120 except that the customer client component interacts with a server other than (or in addition to) Merchant Profile Server 100 Partner Server 140 can exchange data with the Merchant Profile Server 110 via an Internet connection 130 In various embodiments, SSL is used m combination with server-to-server authentication using X 509 certificates (from both the client and the server host) in order to achieve encryption, authentication, and cryptographic protection from replay attacks and other types of security threats Alternative embodiments are possible, such as the use of Virtual Private Network, (VPN) or reduced security (e g , use of SSL only for encryption but not for server-to-server authentication) Alternatively, a dedicated secure connection 150 may be used to exchange data between the Merchant Profile Server 110 and the Partner Server 140 as shown in Figure 3

Figure 4 illustrates exemplary components of Merchant Profile Server 110 The Merchant Profile Server 110 may include several different hosts Alternatively, the described functionality may be contained on a single host, or on multiple hosts that operate in parallel to achieve fault tolerance and load balancing An unlimited number of configurations are possible by varying the numbers of each type of host and the ways in which described functionality can be deployed onto single hosts Exemplary host may also be interpreted as individual computer chassis or central processing units (CPUs), as might occur in Symmetric Multi Processor (SMP) computer architectures The functionality itself consists of a merchant profile application server 200, a merchant profile database 210, and optionally, a user profile database 220, as well as a Web server 230 It will be appreciated that one profile database can include both merchant profile data and user profile data The merchant profile database includes information about merchant sites, for example, a list of fields (with associated types and default values) on a c ec u page, p us n ma on . profile database 220 includes information about the user, for example, name address, purchase preferences, shipping preferences and payment preferences. If the user profile database is omitted, user profile data may be stored on the merchant profile database. Exemplary application server 200 services request events originating from the Web server 230, in response to user interaction with the merchant Web site. Application server 200 also mediates interaction with profile database 220 and may also perform performance enhancing functions, such as result set caching. Application server 200 may also exchange data with profile database 220. The combined data access by application server 200 to merchant profile database 210 and profile database 220 allows application server 200 to execute logic that ultimately results in one or more of a number of actions. These actions may, for example, include updates to data residing in merchant profile database 210 and/or user profile database 220, and the submission of data to Web server 230. The submission of data to Web server 230 may result in Web server 230 formatting data with other Web content. The resultant Web pages will be displayed to users 120. Web server 230 also communicates with any number of form-fill mechanisms that operate on the forms and pages that customer 120 may view and manipulate on the site of a merchant 150. For example, Web server 230 may send form fill data to a client software component that operates within the user's browser, for example, a Java applet or an ActiveX control. Figure 5 shows an alternative exemplary configuration of Merchant Profile Server 110.

In embodiments such as the one shown in Figure 5, the Merchant Profile application server 200 can exchange data with a converter or other translator, for example, an Electronic Commerce Modeling Language (ECML) converter. The ECML converter is based on XML. However, it will be appreciated that other translators or converters that server to convert to a from a predetermined data interchange standard can be implemented in the present invention instead of or in addition to ECML 250. Standards to be converted may be either open standards or proprietary standards. The ECML converter can then send information about merchant site profiles to partners 140 that participate in a special program for this exchange. In the illustrated embodiment, information is communicated from the ECML converter 250 to the partner server 140 via Web server 230. It will be appreciated that in alternative embodiments information can be communicated directly from ECML converter 250 to partner server 140. In the support of various functions (for example, viewing available scripts, using a public script, validating a public script, profiling a page, sending a site map, receiving a site map, etc.) which might be initiated by any one of the components (e.g., merchant profile server 110, partner 140 or customer 230), site profile data can be sent eit er from or to the Merchant Pro ile app ication server 200 using an ECML format that is suitably encoded or decoded by the ECML converter 250 in moving data into or out of the processing logic that resides on the Merchant Profile application server 200 This logic may also exchange data with the various databases and profile data stores that in some embodiments are associated with the Merchant Profile Server 110

As described above with reference to Figure 5, various embodiments may include an ECML converter In these embodiments, ECML can be used to define interchange of data with merchants who have enrolled in the merchant profile exchange program An appropriate Web site allows merchants to enroll in a program Enrolled merchants can use the Web site to perform various functions, for example, accessing raw data for scripts and site profiles and providing links and other mechanisms that provide their customers with easy access to the profiling capability

Figure 6 is a flow diagram illustrating exemplary logic performed by a user to perform merchant profiling in accordance with the present invention In exemplary embodiments, the user visits Web sites using a Web browser, such as Netscape Navigator® or Microsoft Internet Explorer® Client software on the user's computer, for example a browser plug-in, performs merchant profiling in the background while the user is visiting merchant sites The logic of Figure 6 moves from a start block to block 300 where the merchant profiling function is activated In exemplary embodiments, profiling may be activated by using a digital wallet In other embodiments, profiling may occur when the Web browser is run (e g , always active) In yet other embodiments, a user may be required to initiate merchant profiling

Next, the logic moves to block 302 where a Web page is retrieved and displayed This is known in the art For example, a user enters a Uniform Resource Locator (URL) or clicks on a hyperlink The appropriate Web page is retrieved from a Web server (such as merchant 125) Next, the logic moves to decision block 304 where a test is made to determine if there is profile information for the page For example, when a page request is sent to the Web server, a request can simultaneously be sent to the Merchant Profile Server 110 The Merchant Profile Server 1 10 checks the merchant profile database for information for the site If there is information available for the site, the logic moves to decision block 308 to determme if there is information available for the user (in the user profile database) If there is profile information for the site (yes in decision block 304) and profile information for the user (yes in decision block 308), the logic move to block 310 where appropriate fields are filled For example, the site information may include data indicating that there is a name field The user's name is ret eve an sp aye n t e appropr ate e t ere s not pro e n ormat on or t e page (no in decision block 304) or there is not profile information for the user (no in decision block 308), form fill can not be performed and the logic skips block 310

Next, the logic moves to decision block 312 to determine if there is new profile information available In exemplary embodiments, various page contents, such as a POST action, indicate that potential profile information may be extracted If there is potentially profile information available, the logic moves to block 314 where merchant profiling is performed From the perspective of the software on the user's computer, this means that potential profile data is sent to the Merchant Profile Server As described above, this data may be sent based on an action, such as a POST In alternative embodiments, information about every page can be sent to the Merchant Profile Server 110 as potential profile information Next, the logic moves to block 316 where user profile information is saved, for example to the User Profile database, if required In exemplary embodiments, this step is combined with the merchant profiling of block 314 After performing profiling (possibly saving merchant profile information in block 314) and/or user information (in block 316) or if there is no new profile information available (no in decision block 312), the logic moves to decision block 318 to determine if it is time to exit (for example, by a user selecting the Exit menu option in the browser If it is not time to exit, the logic returns to block 302 to wait for the next page request When it is time to exit (yes in decision block 318), the logic of Figure 6 ends. Figures 7-10 are flow diagrams illustrating exemplary logic for performing the site profiling function on the profile server 110 (e g , merchant profile server) in accordance with the present invention In exemplary embodiments, merchant profiling will be performed automatically (e g , without any user interaction) In alternative embodiments, user interaction may be required to initiate site profiling (for example, see exemplary user interfaces shown in Figures 28-33) In exemplary embodiments, the pages processed are Web pages using HyperText Markup Language (HTML) which are transmitted to a user 120 from a merchant over the Internet 130 using the HTTP protocol A browser, such as Netscape Navigator® or Microsoft Internet Explorer® is used to view the Web pages It will be appreciated that the invention can be used to read information in formats other than HTML, for example, Extensible Markup Language (XML) or Wireless Markup Language (WML) It will also be appreciated that the information can be communicated using protocols other than HTTP, for example, Wireless Access Protocol (WAP)

The model of a Web site effectively consists of a collection of pages Each page has an associated vector that characterizes its fields and navigational options (for example, links to other pages, such as a home page or a chec out page within a merc ant site). The n ng o a set of values to these vectors prescribes a specific interaction with the site modeled. It will be appreciated that this description assumes a specification of the starting page for the interaction, and facilities for managing cookies, SSL connections, and any other activities that might be part of the typical user interaction using a browser (which might include Javascript, Java, ActiveX, and other functionality). For example, a model of a Web site might be characterized by three pages, each of which includes a number of editable fields, hyperlinks and submit actions that navigate to one of the other pages. A "binding" of values to these form fields and actions might be provided as a combination of look-ups to a table of user preferences. Starting on the first page, the values are bound to each element of the site model in the following sequence: form fields of the first page, navigation act(s) (e.g., hyperlink or submit) of the first page, form fields of the second page, navigation act(s) of the second page, form fields of the third page and navigation act(s) of the third page.

The basic theory described above identifies how repeatable Web interactions can be achieved. However, it is often not desirable to only be able to precisely repeat an exact sequence of events, such as the purchase of a specific item at an E-Commerce Site. It is often more useful to parameterize these interactions in such a way as to allow limited user interaction to elicit input for necessary variables, such as the product being purchased in the preceding example. The extension of the basic theory is that the vector of user interaction contains a number of "unfilled" values or input variables that the user still must provide. In an exemplary embodiment, the user enters these values. In other embodiments, these values are retrieved by other means , and possibly in combination. Values may be retrieved from user profiles, computed based on available information (e.g., deliver within five days of today's date), computed based on user profile information (e.g., ship to billing address unless the item is gift wrapped), or derived via a combination of automated and non-automated means (e.g., ship to billing address but allow user interaction to override address to identify gift delivery). The means by which values are retrieved can be represented as "meta-values" and stored in an appropriate merchant profile table or user profile table. The term "meta" as used herein denotes the fact that the values are not literal constants, but rather, prescribe a means by which some expression or data can be evaluated so as to provide a value.

In exemplary embodiments, the automated or semi-automated merchant profile function of the present invention is implemented using scripts. A script references specific merchant site model elements (such as vectors of field names and attributes, as well as action information) and prescribes bindings of values that are used for specific types of interaction. The site model escr es t e syntact c e ements t at res e on pages o a s te an t e user pro i e escr es attributes of a user (including preferences) A script describes the semantic relationship between the users' profiles and the site model for various interactions For example, a given page (or pages) of a merchant site may allow a user to perform a variety of functions, such as change address and change payment vehicle (e.g., via a specific credit card or using Automated Clearing House (ACH) facilities) In exemplary embodiments, at least two scripts would be required, one for the change address function and one for the change payment vehicle function It will be appreciated that several scripts may be employed to support a given function, for example, the change address function may include a script to use business address for shipping and one to use home address for shipping The merchant profile system of the present invention provides means for managing scripts that are available to all users (public scripts) as well as scripts that users explicitly create for their own use (private scripts) Merchant partners may choose to utilize this data to improve site usability or to understand typical user interactions and desires Links and other mechanisms that provide a merchant's customers with easy access to the profiling capability may provide either the ability to use profiles, and/or to modify profiles

Vectors are employed for performing the logic in the exemplary embodiment described herein However, it will be appreciated that various other implementations are possible Vectors in exemplary embodiments include, a profile customization vector, a tables vector, an edit fields vector, a field heuristics vector, an actions vector, an actions heuristics vector, a field heuristics strategy patterns vector and an action heuristics strategy patterns vector The profile customization vector stores data that drives the heuristics when user feedback or precise processing directives have previously been provided Such directives can allow certain heuristics to always be favored over others, or for the association of certain names within the application of a specific heuristic to always be favored For example, the association of a specific element of HTML can be assigned a high "goodness" weight to ensure that the heuristics favor the interpretation of that text element as being a field label of a given field The tables vector stores pointers and information about tables that are contained in HTML or other page representation language to support iterative or recursive processing, as well as any other processing that differs between tables and more linear text flows The edit fields vector stores pointers and information about user editable and hidden form fields The field heuristics vector stores information about the application of heuristics of individual fields The actions vector stores pointers and information about form submit and action tags, as well as hyperlinks. The action heuristics vector stores information about the application of heuristics to individual actions an/or hyperlinks The field heuristics strategy patterns vector stores command data and processing instructions that define general heuristics that may be applied to fields in a form to be filled The actions heuristics strategy patterns vector stores command data and processing instructions that define general heuristics that may be applied to actions and navigation items Application of heuristics to a field vector element will initialize an element (or part therein) of the field heuristics vector It will be appreciated that in various embodiments other vectors and heuristics may be used

Figure 7 is a flow diagram illustrating exemplary logic performed by a Merchant Profile Server 110 for profiling a single page As described above with reference to Figure 6, when potential profile information is available, it is sent to the Merchant Profile Server 110 In exemplary embodiments, the data sent to the Merchant Profile Server 110 is a Web page, for example, HyperText Markup Language (HTML) The logic of Figure 7 moves from a start block to block 400 where initialization is performed, for example, data structures are initialized Exemplary data structures are illustrated in Figures 11-28 It will be appreciated that various embodiments may include additional data structures, as well as variations of the data structures illustrated in Figures 11-28 Initializing of data structures may include loading vectors with initial values which may be zero or null values, or objects (such as instances of heuristic command patterns classes) Vectors used in exemplary embodiments of the invention are described above After initialization has been performed, the logic moves to block 402 where the page source is scanned as shown in detail in Figure 8 and described below After the page source has been scanned, the logic moves to block 404 where form field heuristics are performed on the page as shown in detail in Figure 9 and described later Next, the logic moves to block 406 where action heuristics are performed as shown in detail in Figure 10 and described later The logic of Figure 7 then ends The logic illustrated in Figure 7 addresses the scanning of a single page It will be appreciated that the logic that triggers the scanning of a page may also trigger the scanning of other pages In various embodiments, a "spider" scans all pages in a given site by performing a recursive traversal of all links contained in any given page

Figure 8 is a flow diagram illustrating exemplary logic for scanning the source of a page (402 of Figure 7) The logic of Figure 8 moves from a start block to block 420 where the first tag of the page is read The tag determines the processing that will occur If the tag is a hyperlink (yes in decision block 422), the logic moves to block 424 where the tag information is saved in a Page Action Table (such as the one shown in Figure 20) and an offset to the navigation items vector is saved to allow subsequent processing by heuristics to quickly find surroun ing text w en searc ing or possi e e names I t e tag is a orm e yes in decision block 426), the tag information is saved to the field data vector, an offset is saved to the edit fields vector (block 428) and an element of the field heuristics vector is initialized (block 430) If the tag is an action tag (yes in decision block 432), for example, submit, or invokes client-side scripting functionality (e g , a javascript function), the information is saved to the actions vector (block 434) and an element of the action heuristics vector is initialized (block 436)

After the appropriate processing for the tag has been performed (e g , block 424 for a hyperlink, blocks 428 and 430 for a form field or blocks 434 and 436 for an action), the logic moves to decision block 438 to determine if there are more tags in the page to process If so, the logic moves to block 440 to read the next tag The processing then returns to decision block 422 to determine the appropriate processing for the tag If there are no more tags (no in decision block 438), the logic of Figure 8 ends and processing returns to Figure 7 In this manner, each tag in the page is read and processed It will be understood that the logic of Figure 8 is exemplary and many variations are possible For example, if the page contains a table (e g , an HTML or similar table), each tag in the table is processed For nested tables, recursive processing may be employed

Figure 9 is a flow diagram illustrating exemplary logic for performing field heuristics for a page (404 of Figure 7) Performing field heuristics (as in Figure 9) allows for the identification of a meaningful name for each field in a page The name can be used to associate bindings to elements of scripts The logic iterates over each field element Within each of these iterations, a set of heuristics are iterated over The heuristics are encapsulated as "strategy design patterns" (as characterized by "Design Patterns Elements of Reusable Object Oriented Software" by Erich Gamma, John Vhssides, Ralph Johnson, Richard Helm (Addison Wesley Longman, Inc (June, 1994) pages 315-324) Each of the heuristics identifies the best candidate name for a field Many of the heuristics will operate by generating a list of candidate names and then selecting the best possible name from the list of candidate names by applying a weighting function to each candidate name This processing is described in further detail next with reference to Figure 9 The logic of Figure 9 moves from a start block to block 450 where the first element of the edit fields vector is retrieved (block 228 of Figure 8) Next, the logic moves to block 452 where the first heuristics strategy is retrieved For example, a combination of means, such as static initialization or database retrieval (tables not shown here) can be used to initially create instances of the heuristics strategies The strategies specify methods of defining the site map, or examp e, ase on tag name, ase on spat a prox m ty o e to text, ase on spat a proximity with formatting, based on natural language parsing, etc. The logic then moves to block 454 where candidate names are identified by the algorithm. A "goodness" measure is calculated for the heuristics strategy for each element of the edit fields vector. The heuristic seeks to find the best name by identifying candidate names and applying the goodness measure to each. The logic then moves to block 456 where the results (e.g., the candidate names and "goodness measures") are stored to the field heuristics vector. The logic then moves to decision block 458 where a test is made to determine if there are more heuristic strategies to process for the current element of the edit fields vector. If there are more heuristic strategies to process (yes in decision block 458), the logic moves to block 460 to get the next heuristics strategy. The logic then returns to block 454. The logic of blocks 454-458 is repeated for each heuristics strategy for the element of the edit fields vector currently being processed. When all of the heuristic strategies have been processed for the element (no in decision block 458), the logic moves to block 462 where a heuristics weighting function is applied. The best heuristic is identified for each field, and the name that is favored by that heuristic becomes the designated name for that field. The name is stored in the field vector. The logic then moves to block 464 where the best name (identified in block 462) is stored to the edit fields vector. The logic then moves to decision block 466 where a test is made to determine if there are more elements in the edit fields vector. If so, the logic moves to block 468 where the next element of the edit fields vector is retrieved. The logic then returns to block 452. The logic of blocks 452-466 is repeated until there are no more elements in the edit fields vector to be processed. When there are no more elements in the edit fields vector to process (no in decision block 466), the logic of Figure 9 ends and processing returns to Figure 7.

Figure 10 is a flow diagram illustrating exemplary logic for performing action heuristics for a page (406 of Figure 7). The logic for performing action heuristics is similar to that for performing field heuristics (described above with reference to Figure 9)except that the logic addresses the association of name to an action vector. The logic of Figure 10 moves from a start block to block 280 where the first element of the actions vector is retrieved. Next, the logic moves to block 482 where the first heuristics strategy is retrieved. The logic then moves to block 484 where candidate names are identified. A "goodness" measure is calculated for a the heuristics strategy for the element of the actions vector. The logic then moves to block 486 where the results (e.g., the candidate names and "goodness measures" are stored to the field heuristics vector. The logic then moves to decision block 288 where a test is made to determine if there are more heuristic strategies to process for the current element of the actions vector. If t ere are more eur st c strateg es to process yes n ec s on oc 88 , t e og c moves to block 290 to get the next heuristics strategy The logic then returns to block 484 The logic of blocks 484-488 is repeated for each heuristics strategy for the element of the actions vector currently being processed When all of the heuristic strategies have been processed for the element (no in decision block 488), the logic moves to block 492 where a heuristics weighting function is applied The best name is identified for each field The logic then moves to block 494 where the best name (identified in block 492) is stored to the actions vector The logic then moves to decision block 496 where a test is made to determine if there are more elements in the actions vector If so, the logic moves to block 498 where the next element of the actions vector is retrieved The logic then returns to block 482 The logic of blocks 482-496 is repeated until there are no more elements in the actions vector to be processed When there are no more elements in the actions vector to process (no in decision block 496), the logic of Figure 10 ends and processing returns to Figure 7

The present invention utilizes a clickstream of a user (e g , a user's actions in a browser), in combination with the model of a site, to formulate a discrete state space that can be characterized by a set of mathematical vectors Exemplary data structures for a site model are shown in Figures 11-20 and described below Given a mathematically precise description of a Web site (model), and a prescribed sequence of user interactions (vector of actions) with that site, a specific activity (for example, filling in fields of pages and navigation among pages) may result, which in many cases is repeatable and serves some desirable end, such as placing an item in a shopping cart, checking order status or customizing products/services

The tables shown in Figures 11-20 illustrate exemplary data structures for a site model A site table, such as the one shown in Figure 11, associates a site to a meaningful name and a domain name system (DNS) name The site ID is the primary key, and is used to link this table to other tables Additional fields may be supplied in alternative embodiments to characterize an entire site (e g., geographic location, information about redirection or site availability) A site- merchant association table, such as the one shown in Figure 12, associates a site to a specific merchant This table is necessary because of the arbitrariness of the cardinality of association (many-to-many) between merchant and site That is, a given merchant might have multiple sites and/or a given site might service multiple merchants In an alternative embodiment, a merchant ID is used as primary key and a separate table characterizes merchant attributes, such as name A site-subsite association table, such as the one shown in Figure 13, provides a means to break a site into subsites that describe special security and navigation attributes In an alternative embodiment, a subsite ID can be used interchangeably with site ID as a foreign key nto ot er ta es, suc as a su s te c aracter st cs ta e. n exemp ary su s te c aracter s cs table is shown in Figure 14. The subsite characteristics table describes characteristics of a subsite, such as whether it is dynamic (requiring special URL handling, cookies, etc), how cookies are used (and possibly identifying a specific cookie or set of cookies that are stored in other tables that an implementer(s) of this design would create), and other behaviors such as whether a secure session times out, etc. The partial URL defines where in the site this subsite exists. The login field can provide data that is used to determine how to handle logon, for example, it may indicate that there is a logon ID and logon password stored in the user profile database. As such, it would need to identify the profile field names that are specific to this subsite. It will be appreciated that various other subsite characteristics can be included in the subsite characteristics table. A subsite-pages association table, such as the one shown in Figure 15, enumerates pages that can be characterized within a given site model. Page ID is a primary key, and is used as foreign key in many other tables in various embodiments of the invention. A page characteristics table, such as the one shown in Figure 16, characterizes individual pages. Since there is a many-to-one relationship between fields and pages, a foreign key into a field map is used to link specific field data to a page.

A field map table, such as the one shown in Figure 17, associates individual fields, in a specific sequence, with a specific field map (as referenced by a page characteristics table row). A field table, such as the one shown in Figure 18, defines attributes of specific fields. Figure 19 illustrates an exemplary page action map table which identifies actions that are available on a page, such as how data can be submitted after performing a form fill or client- side scripting functionality that can be invoked when a user clicks a button. Figure 20 illustrates an exemplary Page Action Table which provides information that gives meaning to an individual action within a page (name and action information). Figure 21-24 are exemplary data structures defining scriptable interactions. Scriptable interaction ('scripts') composed of sequences of discrete steps are stored in a script flow table (Figure 21), using sequence numbers to set the order. Other tables capture relations between: scripts and pages; pages and sites or parts of sites (to identify operational characteristics such as logon, cookies, etc.); and sites and merchants (to allow for the fact that merchants may use third party sites or have arbitrary cardinality of association with sites). A scriptable interactions table, such as the one shown in Figure 22, defines a script (or scriptable interaction), and uses a script ID as primary key. This key is used as foreign key throughout the system. Since scripts may be public or private, and may originate from numerous sources (e.g., a user or a partner), fields can be provided to identify source and usage. A script flow table, such as the one shown in Figure 21, defines an interaction with a sequence of pages It uses the primary key (script ID), and defines the essential interaction with a Web site which includes an identification of the start page, and an ordered sequence of actions that are linked by an action ID foreign key to a page actions table (shown in Figure 23) An end page ID is also included Since any page can include navigation to any other page, the term "end" is defined to be the last defined action (e.g , navigation is not necessarily sequential among the various pages within a site A sequence of one or more rows from the script flow table defines a script The page actions table shown in Figure 23 defines individual actions The primary key is the action ID which is also a foreign key into other tables In exemplary embodiments, click data and fill data are also provided Fill data can be linked by foreign key to a form fill table (such as the one shown in Figure 24)

A form fill table, such as the one shown in Figure 24, contains the appropriate meta fill value and associated data for each field The meta value prescribes the evaluation and population of a field given access to a user's profile, database tables, and interaction with the user For example, a meta value might specify that associated data is to be treated as a constant to be automatically filled, or that there is no valuation for automated form fill, thereby necessitating that a user fill the field.

Figures 25-28 are exemplary data structures for defining a user profile A user table, such as the one shown in Figure 25 associates a globally unique ID (GUID) used to identify users in the merchant profile system of the present invention with a user profile that has been captured elsewhere (if available), for example, during the process of user interaction with a site or from profile data that the user has provided during self-profiling This can be used to provide an incentive to users to provide feedback and update (validate) the merchant profile scripts A user-feedback association table is shown in Figure 26 This table associates a user to a feedback ID (foreign key into a merchant profile incentives table (such as the one shown in Figure 27) Windows that allow a user to interact with an existing script can also allow the user to enter feedback in a variety of ways This may include general comments or new scripts that the users post, for example, to rectify problems or improve the general behavior of an existing public script A merchant profile incentives table, such as the one shown in Figure 27, stores incentives-based information for a user by tracking the status of a script that has been posted This table is also used to support a description of the relationship between personal and public scripts (which are defined elsewhere) For example, this preserves knowledge that a given public script was originally created as a private script from a given user Figure 28 lustrates an exemp ary user pro e ata ta e T e user pro e ata ta e associates a profile ID (as linked to the GUID in the user table of Figure 25) to a collection of attribute-value pairs Attribute-value pairs provide named values that can be referenced by other programming logic and data For example, a meta fill value of the form fill table (e.g , Figure 24) might reference a named value that is to be filled into a specific form field of a given page during script execution In various embodiments, the user profile database may be a relational database, a Lightweight Directory Access Protocol (LDAP), or some other type of database

Exemplary embodiments identify individual pages within a site, however, alternative embodiments may define the model of a Web site in different ways Each page is associated with a portion of the site (if necessary) in order to characterize security and other aspects of the site that affect how the automated interaction occurs (e g., providing an ID and password and then dispensing a cookie that contains a security token) For example, the view of the join between the site table, site-subsite association table, and subsite characteristics tables (shown in Figures 11, 13 and 14) stores cookies, logon and timeout data for partial URLs In exemplary embodiments, pages are characterized by vectors of fields that can be filled For example, field maps can be stored as records of the view of the join between the field map and field table (Figures 17 and 18). wherein each record represents a field that is contained within a field map. Field maps are further associated with pages by the join between the page characteristics table and the field map table (Figures 16 and 17) Additionally, pages can be characterized by how they can be submitted back to the hosting web server (e.g., HTTP POST, GET, and various SUBMIT actions and values of associated HTML parameters) For example, this could be represented in a join between a page characteristics table, such as the one shown in Figure 16 and a page action map table, such as the one shown in Figure 20 Meaningful sequences of interactions (as may be associated with the user clickstream) are stored and manipulated by the system.

As described above, exemplary embodiments automatically perform merchant profiling without user initiation In alternative embodiments, merchant profiling is initiated by the user Figures 28-33 illustrate exemplary user interface functionality for performing merchant profiling based on user initiation For example, the windows illustrated in Figures 28-33 may be viewed by a user in a browser

In exemplary embodiments, when a user interacts with a merchant site that has been profiled, the form fill values and actions that the user selects during the interaction with the site are also processed by the merchant profile system of the present invention This results in au oma e s orage o e new user pro i e a a in e user pro e a a ase n o er embodiments, pre-existing user profile data may be used exclusively The pre-existing data may have been collected, for example, during enrollment at a merchant site or at a digital wallet site New user profile data is stored in a manner that allows it to be retrieved upon subsequent interactions with merchant sites For example, the user may enter a shipping address This shipping address can thereafter be used to populate shipping address form fields (on this page, on other pages or in the shipping address field of other sites)

Figure 28 is an exemplary main window for user interaction an exemplary merchant profiling system In an exemplary embodiment, the user may initiate merchant profile interaction via a digital wallet interface or from a merchant site (e g , a hyperlink on a Web page of a partner merchant site The main user interface allows the user to specify whether to engage a public script or a personal script, and how to utilize the script, as in validation/editing, creation of a new script, or profiling of a page (as might occur when creating a new script)

A script is a sequence of actions that entail form fill of pages and navigation between different pages Scripts can automate any number of user interface actions, where in some instances a sequence of pages may be submitted to a Web server in a wholly automated fashion and in other instances some manual form fill and/or navigation is required

A public script can be accessed by different users A personal script is one that can only be accessed by the user that created it In some embodiments a script can be "posted" to the merchant profile server 1 10 or to a merchant partner 140 as a candidate to become a public script A suitable incentive-awarding mechanism provides users who successfully create a script that is "accepted" to become public a reward of some sort, for example, bonus rewards miles, coupons, special offers, gifts, etc

In exemplary embodiments a user can make a copy of any script (e g , any public script or any personal script) for the purpose of further editing that script In exemplary embodiments, users are not be permitted to directly modify public scripts Rather, they copy a selected public script and then edit the public script, and re-post it for review (and possibly an award of additional incentives) A validate action allows users to provide feedback on public scripts, and to post recommended variations or corrections as described above (and possibly an award of additional incentives)

The action of creating a new script may also involve creating an initial page profile This automated process identifies fields that a user can edit, how the page can be posted to the server for processing, and links contained within the page This action, "profile page", can be employed by itself or as part of the new script action not er exemp ary too or creat ng a new scr pt s to turn on a macro recor er w c records user interactions with a site Instead of displaying information about the site (as occurs for "profile page"), the macro recorder observes and records the user clickstream and the page context, as well as other interactions such as exchange of cookies A script is effectively generated automatically, although all of the values for the form fields are automatically populated and the script effectively has no variables input to it See the following discussion of variables and interaction space

As described above, Figure 28 is an exemplary user interface that allows a user to select the functionality described above A control allows the user to effectively exit the special modes ("no special mode") When the user selects "no special mode," the merchant profile system pauses until another event (e g , the user de-selecting no special mode) causes merchant profiling to resume

Upon selection of "Use a Public Script" or "Use a Personal Script", a suitable user interface, such as the one shown in Figure 29 allows the user to select a script from the currently available scripts Upon selection of the desired script (e g , pressing an OK button), the user is returned to the main merchant profiling window (Figure 28)

Once a script is selected, the user can perform actions, such as validate or edit) on the script If the user selects Validate Public Script, a suitable user interface (such as the exemplary

Validate Public Script interface shown in Figure 30) is displayed In the exemplary user interface shown in Figure 30, the user can either View Actions associated with the selected script or View Properties associated with the selected script.

Figure 31 is an exemplary illustration showing how a user can view the sequence of actions associated with a specific script Suitable buttons (e.g , delete, edit) are provided to allow the user to further interact with specific actions displayed Figure 32 is an exemplary user interface that may be displayed when a user elects to view a selected action (from Figure 31) In exemplary embodiments, field values stored in a corresponding database tables may be displayed and/or edited, for example, the name of the action, the page on which the action can be performed, how the page is submitted to the Web server, etc Appropriate controls (e g , buttons) are provided for various functions, for example to cancel the edit activity, delete the currently viewed action, save the edits to the action, or allow the user to edit the detailed form fill actions

If, in Figure 32, the user chooses to edit the detailed form fill actions, a suitable user interface, such as the one shown in Figure 33 is displayed The exemplary user interface shown in Figure 33 displays a tabular view of data about each field of a form, including the names an va ues assoc ate w t eac e eta- ata s prov e n t e way o e type Additionally, a meta-value is displayed, which defines the evaluation that leads to the creation of the field value For example, the meta-value might specify a constant, or retrieval from a user profile or database Users can create new scripts in a variety of ways, such as modifying existing ones or performing macro recording In a exemplary embodiments, after running macro recording, the user can select heuristic processing that can distinguish the various meta fill values that populate each field and generate a useful script that requires the user to input only that information that varies from one invocation to the next Such a heuristic can, for example, identify that fields such as "SKU" or "product JO" would vary with each script invocation, but recognize that "e-mail address" would not Some fields might have default values that are the same for all users (e g , "gift" would be "no" by default) Some fields might have default values that are unique to each user (e.g , "e-mail address" or a payment vehicle and account/card number) In these latter two cases, the heuristic processing might identify these fields as ones that allow the user to manually override the default values

Alternatively, a user may create a new script simply by identifying a sequence of one or more pages that correspond to some useful interaction In this approach, a default set of form- fill values will be provided (typically null), and the user will then need to edit data for the field and action maps manually For example, the user will need to identify for each field whether the data is constant, is derived from a profile, or originates from some other source (such as evaluation of a more complex expression that can obtain data from user profiles) The user will similarly identify any additional actions such as submit buttons If the profile spans multiple pages, the user will also identify the next page that appears after a specific action. This latter functionality involves a higher level of expertise by a user, and may not be offered to end users in the various embodiments However, it will be appreciated that this functionality will also be needed for the maintenance and validation of scripts that are to become public scripts accessible to end users For example, the commercial organization that manages the system may employ individuals who create and maintain public scripts using tools and methods, such as those described above Various embodiments of the present invention include aggregation capabilities The aggregation of interactions of multiple users, either by macro record or script, in itself and apart from the creation of specific interaction vectors that are defined as personal or public scripts For example, scripts may be captured, stored, and later used for meaningful and repeatable interactions Aggregated interaction data collected through these means may also be stored and use or a var ety o purposes. t ona ta es may e store to represent str ut ons o various field values and actions entered by a population of users. For example, there may be meaningful information in knowing that 80% of users enter the same address for shipping and billing and that 10% of those that enter different addresses for these two fields are sending gift purchases. In this example, one might conclude that the remaining 10% of users use alternate addresses for billing. This fact may be of significance to businesses, for the improvement of the merchant profile system, for the improvement of merchant sites, or some combination thereof.

The present invention may be described herein in terms of functional block components, screen shots, optional selections and various processing steps. It should be appreciated that such functional blocks may be realized by any number of hardware and/or software components configured to perform the specified functions. For example, the present invention may employ various integrated circuit components, e.g., memory elements, processing elements, logic elements, look-up tables, and the like, which may carry out a variety of functions under the control of one or more microprocessors or other control devices. Similarly, the software elements of the present invention may be implemented with any programming or scripting language such as C, C++, Java, COBOL, assembler, PERL, or the like, with the various algorithms being implemented with any combination of data structures, objects, processes, routines or other programming elements. Further, it should be noted that the present invention may employ any number of conventional techniques for data transmission, signaling, data processing, network control, and the like. For a basic introduction of cryptography, the reader is directed to a text written by Bruce Schneider which is entitled "Applied Cryptography: Protocols, Algorithms, And Source Code In C," published by John Wiley & Sons (second edition, 1996), which is hereby incorporated by reference.

It should be appreciated that the particular implementations shown and described herein are illustrative of the invention and its best mode and are not intended to otherwise limit the scope of the present invention in any way. Indeed, for the sake of brevity, conventional data networking, application development and other functional aspects of the systems (and components of the individual operating components of the systems) may not be described in detail herein. Furthermore, the connecting lines shown in the various figures contained herein are intended to represent exemplary functional relationships and/or physical couplings between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in a practical electronic transaction system. o simp i y t e escription o t e exemp ary em o iments, t e invention is requent y described as pertaining to a profiling system It will be appreciated, however, that many applications of the present invention could be formulated One skilled in the art will appreciate that the network may include any system for exchanging data or transacting business, such as the Internet, an intranet, an extranet, WAN, LAN, satellite communications, and/or the like The users may interact with the system via any input device such as a keyboard, mouse, kiosk, personal digital assistant, handheld computer (e g , Palm Pilot®), cellular phone and/or the like Similarly, the invention could be used in conjunction with any type of personal computer, network computer, workstation, minicomputer, mainframe, or the like running any operating system such as any version of Windows, Windows NT, Wιndows2000, Windows 98, Windows 95, MacOS, OS/2, BeOS, Linux, UNIX, or the like Moreover, although the invention is frequently described herein as being implemented with TCP/IP communications protocols, it will be readily understood that the invention could also be implemented using IPX, Appletalk, IP-6, NetBIOS, OSI or any number of existing or future protocols Moreover, while the exemplary embodiment will be described as a profiling system, the system contemplates the use, sale or distribution of any goods, services or information over any network having similar functionality described herein

The customer and merchant may represent individual people, entities, or business The bank may represent other types of card issuing institutions, such as credit card companies, card sponsoring companies, or third party issuers under contract with financial institutions It is further noted that other participants may be involved in some phases of the transaction, such as an intermediary settlement institution, but these participants are not shown

Each participant is equipped with a computing system to facilitate online commerce transactions The customer has a computing unit in the form of a personal computer, although other types of computing units may be used including laptops, notebooks, hand held computers, set-top boxes, and the like The merchant has a computing unit implemented in the form of a computer-server, although other implementations are possible The bank has a computing center shown as a main frame computer However, the bank computmg center may be implemented in other forms, such as a mini-computer, a PC server, a network set of computers,

The computing units are connected with each other via a data communication network The network is a public network and assumed to be insecure and open to eavesdroppers In the illustrated implementation, the network is embodied as the internet In this context, the computers may or may not be connected to the internet at all times For instance, the customer computer may emp oy a mo em to occas ona y connect to t e nternet, w ereas t e an computing center might maintain a permanent connection to the internet It is noted that the network may be implemented as other types of networks, such as an interactive television (ITV) network Any merchant computer and bank computer are interconnected via a second network, referred to as a payment network The payment network represents existing proprietary networks that presently accommodate transactions for credit cards, debit cards, and other types of financial/banking cards The payment network is a closed network that is assumed to be secure from eavesdroppers Examples of the payment network include the American Express®, VisaNet® and the Veriphone® network In an exemplary embodiment, the electronic commerce system is implemented at the customer and issuing bank In an exemplary implementation, the electronic commerce system is implemented as computer software modules loaded onto the customer computer and the banking computing center The merchant computer does not require any additional software to participate in the online commerce transactions supported by the online commerce system

The corresponding structures, materials, acts and equivalents of all elements in the claims below are intended to include any structure, material or acts for performing the functions in combination with other claimed elements as specifically claimed The scope of the invention should be determined by the allowed claims and their legal equivalents, rather than by the examples given above

Claims

What is claimed is

1 A method for profiling a site, the method comprising a in response to a request by a user, retrieving data from a site, b displaying the retrieved data, c determining if there is site profile data stored for the retrieved data, d if there is site profile data stored for the retrieved data, determining if the stored site profile data needs to be updated, e if the stored site profile data needs to be updated, updating the stored site profile data, and f if there is no stored site profile data for the retrieved data, storing the site profile data

2 The method of Claim 1, further comprising a determining if there is user profile information for the user, b if there is site profile information for the retrieved page of data and there is user profile information for the user, displaying user data in the retrieved data as appropriate

The method of Claim 1, wherein the retrieved data is page data

The method of Claim 3, wherein the page data is a plurality of pages of page data

The method of Claim 1, further comprising a. if there is site profile data for the retrieved data and there is user profile information for the user, displaying user data in the retrieved data, as appropriate

The method of Claim 1, where site profile data is stored as a site model

The method of Claim 6, wherein the site model includes page field information

The method of Claim 6, wherein the site model includes site navigation information The method of Claim 1, further comprising a determining if new user profile information is available, b if new user profile information is available, storing the new user profile information

The method of Claim 1, wherein the site information is available to a plurality of users

The method of Claim 1, wherein the site is a merchant site

A system for profiling a site, the system comprising a at least one user computer, b at least one site computer, wherein said site computer includes at least one site page and wherein a user views the at least site page on said at least one user computer, c a profile server, wherein said profile server stores information about the at least one site page, and d a network, wherein communications among said user computer, said at least one site computer and said profile server occur over said network

The system of Claim 12, wherein said profile server further stores information about at least one user

The system of Claim 12, further comprising a partner server

The system of Claim 12, wherein said merchant profile server comprises a an application server, and b a web server

The system of Claim 15, wherein said merchant profile server further comprises a translator for translating profile data

The system of Claim 16, wherein said translator translates ECML data

. e system o am , w eren sa ste computer s a merc ant ste compuer.