WO2001088848A2 - Computer-aided method and apparatus for directly assisting people in problem solving - Google Patents

Abstract

A new generation computer-assisted problem solving tool that helps meet human problem solving needs and captures problem solving patterns and preferences across multiple professions, industries, sets and problem-solving scales. The interface makes it possible to introduce essential decision-making factors in terms of problem solving, relating to the choice of solutions and / or partial preferences, in computer processes driving an intelligent engine intended to choose a fully integrated set of relevant solutions ( e.g. ideas, knowledge, people, resources, products), and can be layered around any data extraction mechanism. The solver computer system is installed as part of a web interface that shows the degree of sophistication that can be achieved by professional problem solvers, both individually and collectively. The solver computer system is controlled by the predetermined resolution preferences of any individual or professional solver to directly assist a solver in a problem solving process based on </ = assisted intelligence> / =.

Description

COMPUTER-AIDED METHOD AND APPARATUS FOR DIRECTLY ASSISTING PEOPLE IN PROBLEM SOLVING

SPECIFICATION

FIELD OF THE INVENTION

Our invention relates to computer-assisted human problem-solving and decision-making that create a context for automatic information processing, manipulation and organization.

BACKGROUND AND SUMMARY

We live in an era where information technology is our key tool, knowledge is a critical asset and problem solving is a paramount skill. Unfortunately, any problem solver that wants to enhance his or her ability to solve problems and thrive in this new environment is inhibited from the start. The problem solver may need to: ask for expert advice and assistance from others or purchase it from consultants, get published content and training information (e.g., training courses, magazines and other publications) from his or her community, receive online learning resources or knowledge in databases, and use certain automated software tools such as workflow software, desktop applications or expert systems.

In the past, all these had to be gathered from different sources by different means. Time is wasted. Opportunities are lost. Attention is fractured.

There have in the past been some attempts to integrate some of these different types of resources using a computer interface such as an information portal. Generally, the best that prior computing systems offer in terms of integrated, intelligent assistance is either expert advisory systems that help in very specific types of issues, or search engines that offer an overload of information on everything. There is a need for a method and system that can simultaneously cure information overload and enhance problem solver capability with expert assistance across a broad range of problems and opportunities.

Our invention directly addresses this challenge. We have developed an approach that systematizes the methods and approaches to human problem solving - in fact any existing problem solving frame of reference ~ making it possible to integrate these approaches in a systematic way with computer-based decision theories/tools (software) and to link those problem-solving and decision-making tools to knowledge objects for purposes of processing and manipulation. We have invented what amounts to a new type of problem solving "power tool."

A path to resolving any problem or opportunity can be thought of as a journey through a multidimensional space of possible routes and destinations. Tough, multidisciplinary, large-scale problems are never cracked with a single solution or "silver bullet." Nor are complex decision paths generally predictable enough to program a computer to move through the problem space in a deterministic fashion using current technologies. To complete a journey successfully when tackling the richer, more varied problem sets that make up most of professional work, individuals and teams make hundreds or thousands of decisions/choices about partial solutions (e.g., knowledge, ideas, tools, people, resources and techniques) and how to apply them to define and achieve a goal. Each of those choices is based on a pool of knowledge objects that represent and are tied to means of marshalling a real partial solution (e.g. ordering, payment, fulfillment and delivery).

To address this, we have developed a full integration of mindware (i.e. analog human expertise), knowledgeware (i.e. digital knowledge and learning systems), software (i.e. digital problem solving systems) and content resources, providing a novel, next generation computer aided problem-solving, decision-making and knowledge organization capability (i.e. "power tool"). In one place, we provide any human problem solver with the right knowledge, capability and direct expert assistance at the right time during every step in any problem solving process. (See Figure 1A-1) With an approach that allows more user involvement, less constraining methods and a more flexible, generally applicable knowledge organization scheme, it is possible to push the state of the art past both automated artificial intelligence and systematic human intelligence into an area we call "assisted intelligence." The promise of "assisted intelligence" is to bridge a large gap that currently exists and bring into being a whole new set of possibilities for creating and realizing human potential. (See Figure 1 A-2) The different elements of this opportunity involve both advancing and then integrating the arts of mindware, software and knowledgeware to achieve a new problem solving approach, system and method.

An Illustrative Example Problem Solving Process

Suppose a person has a problem he or she wants to solve. The problem can be of any sort — for example, planning a party, building an information system, constructing a an addition to a house, or developing a business strategy. Anyone who has ever done any of these things knows how difficult it is to find the right information, locate the right people, and make all the arrangements without forgetting anything. Figure IB (and the Figures that follow) illustrates how a non-limiting, illustrative example embodiment of our invention can provide assisted intelligence to help solve any sort of problem. Example First Step

Looking at Figure IB, the person 10 may first decide what problem he or she should solve (Step 14). This may also include a choice about which specific method the person wishes to use in solving the problem. The person can use a computer 12 (for example, a personal computer, a handheld computer, or a web enabled appliance), mediated by an interface 12A (for example, a web browser or a voice-activated portal) using any type of human-computer communication device (e.g. keyboard, mouse, voice response system) to connect to a solver computing system and method provided by our invention. Our solver computing system and method provides resources and functions that help the person solve his or her problem.

Figure 1C shows an example introductory "home screen" that our solver computing system and method may display on the person's computer display. The various problem solving functions can be presented on the computer screen as part of a "one stop shop" or "portal" for solving problems. The main screen can provide a variety of links to various resources of interest including communities, knowledge, problems, learning sources and other links. It is here, for instance, that a person would orient themselves and get up-to-date on whether colleagues are involved in solving something similar. For example, in planning a party, the computing system would present a screen with different types of communities already involved in planning a party (e.g. fundraisers), knowledge about planning parties (e.g. books, articles) and/or online course that would teach certain elements of party execution (e.g., techniques for decoration).

The particular example screen of Figure 1C is based on a particular problem set within the Department of Defense community. However, different communities can have different screens. For example, a "party planning" community and a "residential building addition" community may each have their own customized screens and list of options. The person can customize this main screen as he or she desires to include the resources of most interest at a particular time. Because the invention provides a general solution capable of solving a wide range of problems spanning across a number of different domains, professions, industries or sectors, the underlying methodology and computing system can remain the same across a wide range of particular problem types and areas. Example Second Step The person may next choose preferences (Figure IB - step 14A). These preferences may be a set of computerized value judgments that apply in automated problem solving. The person can specify that his or her own value judgments should be used ~ or he or she can ask that someone else's value judgments (e.g., those of an expert problem solver) should be applied. For example, the person may choose from a variety of different types of expertise — or even a particular expert (for example, a famous college professor, military leader or business executive). The chosen expertise can be that of one or more colleagues or friends who have solved this type of problem before, or could be based on the person's own expertise in solving other problems in the past. Figure ID shows an example computer screen display the person can use to choose a set of preferences (104). The lower right-hand corner of the screen under "Solver Personalities" includes a "pull-down" menu (down arrow) 20. When the person clicks on the down arrow 20, he or she may be presented with a variety of different expert personalities (a "personality" represents one or more preference sets) for selection. The person may also be able to create a new personality or edit an existing personality. A "Personality Filter" tool 21 can be used to help the person find the personality he or she is looking for. For example, in planning a party, the solver computing system might present the person with the option of choosing a party planning expert personality such as a "Martha Stewart" personality or a "Julie Rosso" personality.

In this example, the person is not choosing an actual human being to help solve the problem, but rather, is choosing a computer imprint of a personality that our solver computing system and method can use in an automated way. Our solver computing system and method uses the chosen personality to distill expert value judgments from a computer-based problem-solving model 103a (see Figure IB).

The computer problem solving model 103a in this example represents general problem solving goals that can be applied to help solve virtually any problem. These goals can include objectives such as, for example:

• the best ideas,

• the best knowledge, • the best relationships,

• the best direction,

• the best capability, and

• the best execution.

Each of these goals can be broken down into any number of subgoals. When the preferences are applied to the problem solving model

103a, the result is a more specific computer representation of individual preferences called a "preference set" 104. The preference set 104 embodies the preferences of a particular individual human problem solver.

As a result, there are many possible "personality types." As an example, a particular personality might value knowledge more than relationships when solving a particular type of problem. The preference set 104 takes such value judgments into account. Example Third Step The person then sets a context for his or her decision (step 14B). In effect, the person tells our solver computing system and method where the person is currently within the problem's life cycle and/or what decision- making mode they are in. Problems can be thought of as having a life cycle or time-progression filled with different types of activities. These activities may involve a progression of activities or "modes" including, for example:

• coming up with ideas,

• gathering knowledge,

• forming relationships, • identifying problems,

• developing capabilities, and

• obtaining results.

The person may progress through these different problem solving modes in different time phases or stages. For example, a person solving a problem may first need to orient himself or herself to the part of the problem he or she is solving right now. He or she may then select certain resources, and initiate certain problem solving activities. The person may then execute and complete the task, and evaluate the results. Looking ahead to Figure 3A, you can see how a particular problem life cycle extends across the different domains of a so-called "context matrix" that correlates problem solving modes and problem solving phases within a multidimensional problem solving framework.

In step 14B of Figure IB, our solving computing system and method learns where the person is in the problem solving progression or life cycle. Our solver computing system and method can learn this information in a variety of ways. One way is for a person to simply declare this information. For example, the example computer screen display shown in Figure ID shows a "dashboard" type display listing a first "gauge" 702 for problem solving mode and a second "gauge" 704 for problem solving phase. The person can "click on" a part of the "dashboard" to specify the mode and/or phase he or she is working within at the time. Another way is for the solver computing system or method to infer this information from the person's past and/or current activities, or the activities of others they are working with. Any combination of these approaches can be used.

Referring again to Figure IB, the person may frame a decision or ask a question (step 14B) relevant to the problem he or she wants to solve. Figures 1C and IE show example computer screen displays including a "solver criteria" space 22 where the person can input a query or frame a decision, and Figure IF is a close-up of this same display. In this particular example, the person has typed in the question "architecture", but it could be any sort of inquiry (for example, "party planning", "roofing", or just about any problem solving inquiry you can think of). Our solver computing system and method use the person's question and/or decision context to automatically search through resources (i.e. partial solutions) that might help to solve the problem. The resources can include partial solutions of any type, for example:

• tools • methods

• courses

• human resources (consultants, for example)

• books, magazines or other printed information

• videos, audio presentations and/or multimedia presentations • other sorts of knowledge objects

• any other type of resource you can think of.

The resources can be stored within computer 12, or they can be stored remotely. They can be accessed via any communications mechanism, distributed or otherwise (e.g. the Internet, a local or wide area network or any other telecommunications means.) They can be prequalified in terms of potential usefulness to a particular community of interest and/or problem domain, but they need not be. Our solver computing system and method can use a variety of different data retrieval techniques (e.g. search engines) to find resources that may be responsive to the person's question or may help make a decision.

In this example, our solver computing system and method does not overload the person with an indiscriminate listing of all of the various partial solutions and resources the search engine returns. Instead, once the resources have been found, our solver computing system and method calculates how useful each resource may be in solving the person's problem. This "utility computation" is based on the preference set 104 and the problem solving progression (mode and phase) information. In one example, each resource is tagged with information called "metadata" that allows our solver computing system and method to easily and efficiently match the resource with the preference set 104, and to calculate the resource's utility based on the person's position within the problem solving progression or "life cycle." These same tags may also categorize the resource by "use type" (for example, briefing, measures, articles, sources, models and concepts as shown in Figure IE).

In the example of party planning, the solver system might return different types of partial solutions, such as:

• ideas (e.g. innovative types of parties and themes),

• knowledge (e.g. books and articles on parties),

• relationships (e.g. caterers and other party suppliers),

• planning (e.g. tools for planning the party and defining objectives), • resources (e.g. utensils, dance floor, bands, food) and

• implementation results (e.g. step-by-step recipes for food).

In this example, our solver computing system and method ranks the listing in order of utility or usefulness, and presents the person with a listing 16 of some or all of the resources for the user to look at (step 14C). The person may specify the maximum number of resources he or she wants to see (for example, no more than 20). The example listing is very useful because it is ranked in order of utility (e.g., relevance) and may include only the most useful resources retrieved based on the person's inquiry. In this example the person can also choose to filter the resources based on problem solving mode. As shown in Figures IE and IF, the person can select any mode (or all of the modes) by "clicking on" a "display mode" selection 722.

The resulting listing can be presented on the computer 12 (e.g. on the computer screen or via a voice interface) for easy processing (i.e. viewing) and selection. See Figure IE. The person may select any of these resources (for example, by "clicking on" associated links or symbols displayed on the computer's screen). This selection can result in the resource being automatically delivered to the person. For example, "clicking on" a link to a magazine article or video presentation may instantly display the article or video on the person's computer screen. In some cases, the person may need to order and pay for the resource to access it (see for example the "xNet Consortium" item listed on Figure IE. For example, the person might be able to sign up for a course or seminar, pay for it, and get instructions about where and when to go to attend.

Example Fourth and Fifth Steps

Referring again to Figure IB, our solver computing system and method can also provide tools to help the person collaborate with other people (step 14D) using the selected partial solutions or other resources. Many times, other people who have confronted the same problem in the past will have useful advice. The person may also want to communicate with other people helping him or her to solve the problem. Figures 1 C and ID (and see also Figure IG) show an example list of collaboration type tools 692 such as for example:

• email,

• workflow,

• people, • chat,

• video conferencing,

• other collaboration tools.

The various steps shown in Figure IB can be performed over and over again as the user attacks different parts of the problem and makes decisions. The user can set a new context as he or she moves to different parts of the problem life cycle (step 14B), and evaluate resources directed to those parts of the problem (step 14C). These steps can be repeated many times until the user actually solves his or her problem (see example fifth step ~ step 14E).

Example Solver Computing System

Figure 1H shows an example solver computing system 50 having six major components:

• solution system 100

• solver engine 200 • solution well 300

• solving toolkit 400

• prequalified content 500 • portal infrastructure 600.

Each of the six components may play a role in closely integrating the different sources of problem solving as shown in Figure 1 A -1. Four of these components - the Solution System 100, Prequalified content 500, the Solution Well 300 and the Solver Toolkit 400 - provide a means of adapting each of the four domains of Figure 1A-1 so that they can be blended together. For example:

• The Solution System 100 allows any problem solving method or set of expert preferences to be engineered into a computer model to determine the problem solving context for decisions or questions.

• Prequalified content 500 ensures a bounded set of high quality knowledge objects are available to a problem solver.

• The Solution Well 300 arranges those knowledge objects by unique types associated with certain problem solving decision modes, so that the amount of information provided can be dramatically reduced and fit a problem-solving context precisely.

• The Solving toolkit 400 provides a common set of easily available software tools that can be used to collaborate with other problem solvers.

The other two components - the Solver engine 200 and the Solver Portal infrastructure 600 - provide the means of integrating these both with one another and any other computing system. The Solver Engine 200 pulls the best sources of knowledge together at every key decision point in problem solving process - dynamically integrating them and presenting them in order of relevance. The Solver Portal infrastructure 600 provides the infrastructure to integrate software and available knowledge management with transaction and security features. An Example Computer Assisted Problem Solving Method

Figure II shows an example of how computer assisted problem solving can be accomplished using the Figure 1H solver computing system. Example First Step The overall process starts with the person choosing a problem to solve (step 51 A). The person may define the environment and/or a specific problem or opportunity to resolve. They may also choose a particular problem-solving method to use, select and purchase this method. In this example, the person can do this from a "home screen" like the one shown in Figure lC.

The process proceeds by the person orienting himself or herself in a professional community to determine what others are doing that may be useful. This may involve orienting oneself in ongoing work of the community corresponding to the particular portal or "home screen" and/or with prequalified content within the solution well 300. This operation can uses the Solver Portal 600 which draws on prequalified content from the solution well 300 (step 51B). The Figure 1C screen in the example described above may be used to perform this operation. Example Second Step Next, the person profiles himself or herself and his or her problem, while choosing and purchasing the intelligent assistance that is needed. This step may use the Solution System 100 (step 54 ~ and see step 14A of Figure IB). The profiling gives the system 50 information about who the person is and what his or her needs are so the system can best help the person. This profiling of the person and the problem allows the system to determine a menu of preference sets (and/or a mixture of more than one preference set) that is most appropriate and to narrow the bounds of relevant content from the Solution Well 300. A preference set may then be chosen or recommended and purchased. The screen of Figure ID can be used to perform this operation.

Example Third Step

At this point solving begins, with the person inputting queries in normal language to the Solver Engine 200 that return the highest priority knowledge for purchase from the Solution Well 300 given the current context of the problem solving effort (step 55 ~ and see steps 14B and 14C of Figure IB). The person can iteratively formulate queries for the solver engine 200 that return the best partial solutions. The screen of Figure IE shows an example output of the system for one such inquiry.

Example Fourth and Fifth Steps

Using the knowledge returned by the Solver Engine 200, the person can use the Solving Toolkit 400 to collaborate with colleagues in taking action (step 57 ~ and see step 14D of Figure IB). This "collaboration" step allows the person to purchase appropriate partial solutions and then apply them in concert with other problem solvers by using the solver toolkit 400 in their resolution effort. Figure IG shows an example screen with such solver toolkit 400 features.

Steps 54, 55 and 57 are iterated (repeated) as many times as necessary until reaching the fifth step (Figure IB, step 14E) when the problem is resolved.

Additional Example Details

As will now be appreciated, we have developed a system of problem solving that has been engineered to the level of detail that allows it to seamlessly interface with the logic structure of a machine while remaining accessible to large numbers of people for broad classes of problem solving. From our research base, we have distilled: • a general frame of reference;

• a universal or widely applicable set of principles, guidelines and values for making choices about potential solutions; and

• a set of decision-making modes, that can be usefully linked to certain types of knowledge objects (e.g. partial solutions).

Some of the example features provided by example systems and methods embodying the present invention include:

• a systematic approach to general problem solving and decision making integrated in an effective yet flexible manner in computer software;

• categorizing knowledge around general problem classes and use types that reflect varying degrees of usefulness in different decision making modes;

• engineering human preferences for the types of partial solutions they prefer in different phases and modes of problem solving in an effective yet flexible manner in computer software;

• using data types that allow the manipulation of data in a way more akin to how humans really use knowledge to solve problems; • making choices about knowledge objects as partial solutions to a problem solving effort; and

• a knowledge organization scheme that seamlessly allows knowledge objects to represent partial solutions in a computer- aided problem solving process that can be applied to any conceivable problem or opportunity.

Other aspects of the invention provide any or all of the following additional features: • A computing system (e.g. web-based or other server) coupled to a decentralized computer network, with a user interface (e.g. browser or voice response system) and hosting the solver engine.

• A data retrieval mechanism (e.g. search engine) interfaced to the solver engine

• A filtered and/or sorted object set presented to the user by transmitting messages to the browser via a decentralized computer network.

• A network interface that communicates with the filtered and/or sorted objects in the form of computer screens (for example web pages or other displays)

• Filtered and/or sorted objects presented to the user in such a way as to directly assist the user in achieving a changed goal state.

• A user profile (characterization) relative to the problem-solving progression.

• An elicited problem solving mode and/or phase from the user, and computed object utilities based on the user's problem solving mode and/or phase.

• The problem solving progression may comprise plural problem solving modes and plural problem solving phases, and the characterization may involve a user interface that requests the user to identify one of the plural problem solving phases and/or problem solving modes.

• Partial solutions presented to the user's problem solving activity. • A decision theoretic process such as, for example, multi-attribute utility theory, for ranking alternatives. • A multi-dimensional problem solving space defining plural problem solving modes and plural problem solving phases within each of the modes.

• A problem-solving context matrix defining plural problem solving modes along a first dimension and plural problem solving phases along a second dimension different from the first dimension.

• Eliciting problem solving preference information from at least one person, and using the elicited preference information to develop at least one preference set from a preference model (the person may be the user and/or someone different from the user, e.g., an expert problem solver).

• Mapping object attributes to the model; the object attributes can be encoded within object metatags associated with the objects, and the solver engine can map the object metatags to the model.

• Ranking the objects based on computed utility using the preferences.

• Presenting a ranked subset of the objects in an interactive digital form (e.g. web page) usable by a person. • Presenting the objects to the user based on use type.

• Producing a static stored data structure, and operating dynamically on the static structure based on the static stored data structure and user inputs.

• Developing at least some continuous utility functions, and dynamically evaluating the continuous utility functions for particular objects within the object set.

• A search engine coupled to at least one object repository that searches an object repository to provide search results; the selecting a subset of the search results based at least in part on the search results.

• Retrieving at least some of the objects over a distributed computing network (e.g. the Internet.) • Mapping objects to the model based on metatags associated with the objects.

• Presenting the filtered and/or sorted objects on a web portal.

• Developing plural specialized preference sets corresponding to different personalities, and selecting one of the plural preference sets.

• A transaction engine may be used to electronically charge the user a fee for a partial solution and/or access to a preference set based on the preference model.

• A unique interface between a solver engine and a search engine based on use types.

• E-commerce sales of deliverables including preference models, preference sets, and partial solutions.

One example aspect of our invention provides a computer-based system and method for assisting a human problem solver comprises: • developing a computer-based preference model reflecting a problem-solving progression represented by a multidimensional correlation between plural problem solving modes and plural problem solving phases;

• determining a context of the human problem solver within the multidimensional correlation; and

• using the preference model, the determined context and value judgments represented in a preference set to present partial solutions to the human problem solver. Another example aspect of our invention provides a data processing system for assisting a user engaged in a problem solving activity, comprising:

• a computer-based modeler that develops a computer-based preference model reflecting a problem-solving progression;

• an object handler that accesses a set of information objects; and

• a solver engine operatively coupled to at least the object handler, the solver engine using individual preference sets to compute utilities of objects within the object set, the solver engine filtering and/or sorting the object set based at least in part on the computed utilities and presenting the filtered and/or sorted objects to the user. Yet another aspect of the invention provides a computer-assisted method for assisting a user engaged in a problem solving activity, comprising:

• developing a computer-based preference model of a problem-solving goal hierarchy reflecting a problem-solving progression;

• accessing a set of objects; • purchasing and using an individual preference set to compute utilities of objects within the object set;

• filtering and/or sorting the object set based at least in part on the computed utilities; and

• presenting the filtered and/or sorted objects to the user. Another aspect of the invention provides a computer system that filters and ranks objects to help a user solve problems. It includes a storage mechanism for storing a data structure representing a problem solving goal hierarchy. The hierarchy corresponds to a multi-dimensional problem solving space of problem solving modes that are exercised over time. The goal hierarchy includes plural terminal nodes each of which defines decision theoretic parameters and additional information for, in use, correlating the node with a metatag associated with at least one object to be filtered. The decision theoretic parameters may comprise multi-attribute utility theory parameters. The multi-dimensional problem solving space may be defined by a context matrix having first and second dimensions, the first dimension defining problem solving modes, the second dimension defining problem solving phases. Still another aspect of the invention provides a computing system

(e.g. web portal) coupled to a distributed communications network (e.g. the Internet), comprising:

• a storage mechanism storing at least one preference model representing a problem solving goal hierarchy that models human problem solving;

• a preference set for an individual person derived using the model as it is applied to a particular problem or class of problems;

• a data retrieval mechanism (e.g. search engine) arranged to retrieve a set of objects from a data repository; • a utility computation performed relative to the goal hierarchy, user preferences and data that describes alternatives;

• a ranker that ranks the retrieved objects based on computed utility; and

• a user interface that presents the ranked objects to at least one user via a computer, an interface (e.g. browser, voice response system) and a communications network (e.g. the Internet). The data retrieval mechanism (e.g. search engine) may retrieve objects having identifying data associated therewith, and the mechanism may include an object mapper that maps the retrieved objects to the model based at least in part on the identifying data. The user interface may transmit the ranked objects encapsulated in messages (e.g. http or other messages). A filter coupled to the ranker may categorize the retrieved object based on use type.

Still another aspect of the invention provides a computer-assisted method for helping at least one user solve a problem, comprising:

(a) distilling at least one person's problem solving decision preferences into an operational model; (b) characterizing the user's problem solving activity as a progression of plural problem solving modes;

(c) selecting at least one mode and/or phase within the mode progression in response to input signals and/or stored information;

(d) performing a comparison between attributes of an object set and the model, based at least in part on the selected mode; and

(e) selecting and presenting a subset of the object set to the user based at least in part on the comparison.

The selecting and presenting step may include arranging the subset of objects on a display based on use type.

Still another aspect of the invention provides a computer-assisted method for helping at least one user solve a problem, comprising:

(a) developing a preference set based on a problem-solving model;

(b) characterizing the user's problem solving activity as a progression of plural problem solving modes in phases over time;

(c) selecting at least one mode and/or phase within the mode progression in response to input signals and/or stored information;

(d) selecting a subset of objects from an object repository based at least in part on the selected mode and the preference set; and (e) delivering the subset of objects to the user.

The preference set, preference model and/or object subsets may be delivered to the user for a fee. The object selecting step may include mapping preference attributes within the preference set to data schema associated with the objects. The data schema may comprise metatags associated with the objects.

Still another aspect of the invention may provide, in the context of a data processing system including a preference engine that develops object utility values based at least in part on a progression of problem solving modes and/or phases, a solution well comprising:

• an object repository providing a set of prequalified objects each having associated data schemata facilitating mapping of the objects to the progression of problem solving modes and/or phases; • a content management system that manages the objects within the object repository and presents objects to the preference engine; and

• a portfolio that organizes and packages objects based on content. At least some of the objects may comprise streaming objects, and at least some of the data schemata may comprise metatags. The portfolio may package objects based on problem or opportunity, and can be meta-tagged by problem. The portfolio includes a template system and a portfolio creation system. The portfolio may aggregate, organize and index diverse data types to provide domain-specific content. Still another aspect of the invention provides a method of buying or selling partial solutions comprising: (a) using a preference set based at least in part on a problem solving progression to develop a set of knowledge objects that may be used to directly assist a user in solving a particular problem; and

(b) performing an electronic, computer-based transaction that exchanges the knowledge object set for value.

A still further aspect of the invention provides a method of buying or selling preference sets comprising:

• applying domain specific parameters to a generalized preference model based on a problem solving progression to provide at least one preference set; and

• performing an electronic, computer-based transaction that exchanges the preference set for value.

A still further aspect of the invention provides a method of choosing, buying or selling computerized preference models comprising:

• Distilling at least one person or organization's problem-solving method into a context matrix that serves as the basis for a computerized operational problem solving preference model; • performing an electronic, computer-based transaction that exchanges the preference model for value. Yet another aspect of the invention provides a method of creating specialized preference sets according to different personalities comprising:

(a) developing a generalized preference model based on a progression of problem solving modes and/or phases;

(b) eliciting judgment information from a human being; and (c) applying the elicited judgment information as weighting information to the generalized preference model to produce at least one preference set.

The human being may comprise an expert problem solver, and the elicited judgment information may provide expert problem solving values. BRIEF DESCRIPTION OF THE DRAWINGS

These, as well as other features and advantages of this invention, will be more completely understood and appreciated by careful study of the following more detailed description of presently preferred exemplary and non-limiting embodiments of the invention taken in conjunction with the accompanying drawings, of which:

I. Introduction to Solver Computing System and Method:

Figure 1A-1 shows example problem solving approaches;

Figure 1 A-2 shows an example integrated, assisted intelligence domain provided by an aspect of our invention;

Figure IB shows an example problem solving procedure provided by our invention;

Figure 1C-1G show example computer screen displays;

Figure 1H shows an overall block diagram of an example Solver Computing System;

Figure II shows an example process a person might follow to solve a problem using the example Solver Computing System;

Figures 1J-1 and 1J-2 together are an example process flow diagram;

II. Example Solution System: Figure 2 is an overall block diagram of an example solution system;

Figure 3A shows an example context matrix; Figure 3B defines some of the various activities and types of decisions a human problem solver may be involved in as characterized by different cells of the example context matrix;

Figure 4 shows an example process for creating a problem-solving decision theoretic preference model(s) and preference set(s) based on a context matrix;

Figure 4A shows an example solution system architecture;

Figure 5 shows an example partial simplified goal hierarchy that reflects modes and phases of the example context matrix; Figure 6 illustrates a further example goal hierarchy;

Figure 7 shows an example general classification of different types of personalities that can be used in connection with the Figure 4 example model generation process;

Figure 8 shows an example of how preference attributes within a preference model can be mapped to attributes of partial solutions using attribute and metadata object mapping;

III. Example Solver Engine:

Figure 9 is a schematic illustration of an example solver engine;

Figure 10 shows an overall flowchart of an example process to define and operate the example solver engine;

Figure 11 is a block diagram of an example solver engine architecture;

Figure 12 shows an example utility computation architecture;

Figures 13 and 14 show example attribute entries for continuous- valued (size) and discrete- valued (semantic density) attributes;

Figure 15 shows an example object attribute data structure;

Figure 16 shows an example demultiplexing function for "interior" discrete values; Figure 17 is a block diagram of an example attribute mapper; Figure 18 is a flowchart of example steps that may be used to implement an example utility computation;

Figure 19 is a block diagram of an example filter and display block;

IV. Example Solution Well:

Figure 20 is a block diagram of an example solution well architecture;

Figures 21 A-21D show an example list of base schema metatags that can be used to associate partial solution attributes with preference attributes;

Figures 22A-22F show example use types associated with the various "modes" of the example context matrix;

V. Example Solver Portal Infrastructure:

Figure 23 shows an example solver portal architecture; and Figures 24A & 24B show a more detailed schematic illustration of an example portal architecture.

DETAILED DESCRIPTION OF THE DRAWINGS

More Detailed Statement of the Problem

In order to understand the significance of this invention, it may be helpful to better understand in more detail the unsolved problem that currently exists. As discussed above, information technology has become the primary tool for business, knowledge the key strategic asset and problem solving the paramount skill. With global integration, intensified competition, a heightened influence of technology and a more diverse, distributed workforce, flexible, scalable and generalizeable approaches and tools for integrating and enabling professional problem solving are in extremely high demand. Problem-solving ability is now the most sought-after trait in executives, according to a recent survey of 1000 CEOs published in the Wall Street Journal. In fact, as the proportion of services to manufacturing in the US and global economy grows - now about 70% in the US - that entire sector is restructuring the way it works based on the advent of openly architected, distributing computing platforms (e.g. the World Wide Web).

The essence of service is solving someone's problem. Therefore, problem solving is at the core of this entire sector of the economy. But skilled problem solvers are in short supply, and few of the resources available today (e.g., training programs, executive education, consulting services) adequately help businesses address this problem-solving skill gap.

Common pressures mean businesses face tougher, more complex problems and opportunities at an accelerating rate. Thousands of efforts are started every day, in processes involving people, technology and knowledge assets that waste resources and/or fail to deliver. The reason is a capability gap, rooted in mismanaged knowledge, hard to control technology and inadequate problem solving skills. CXOs spend billions and devote considerable effort on closing this gap to reduce waste, turn-over, opportunity cost and raise revenue. Capability gaps are especially acute in knowledge intensive industries where technology penetration rates are rising. The range of choices for CXOs and professionals are typically expensive, fragmented and of indeterminate value.

One way to attack this problem is to focus not just on information retrieval (e.g. data mining, search), or online learning, or software technology (e.g. browsers) but on the problem solving and decision-making that create a true context for the interpretation, use and value of knowledge and information.

Cutting across professions, industries and disciplines are a common set of problems and opportunities that professionals around the world are trying to solve every day. These include the following classes of medium- to-large scale problems that typically involve dozens of intelligent professionals working together over many months. These problems also involve thousands of individual decisions from the beginning - where a problem is identified and defined - to the end where resolution is reached and learning is produced. Those people draw upon and choose between many combinations of expertise, tools, knowledge or information that will help them to do the job faster, better and cheaper. In fact, the 20 million professionals currently working in the U.S. make decisions to spend over $200 billion per year on ways to improve their problem solving capability. The types of problems they solve may include:

• New product development

• Sales and service

• Acquisition of capital investment • Deals and negotiations

• Business improvement projects

• Crises and conflict resolution

• Research and analysis

• Deployments and campaigns • Repair and maintenance projects

• Applied research

• Mergers & acquisitions

• Managing company growth

For decades, authors, companies, and consultants have promulgated multi-step problem solving approaches, frameworks and methods that purport to generalize across any problem or opportunity set. Such approaches have often included an expensive set of associated training courses. Because this knowledge is primarily resident in and is transferred through human beings (i.e. in analog form), we call this mindware.

Meanwhile, the software industry continues to create a growing blizzard of technologies and tools designed to improve knowledge management and collaboration to impact both individual and group effectiveness.

Furthermore, the knowledge industry has developed massive collections of data represented in databases accessible by any number of automated search, data mining or data organization tools - what we'll call knowledgeware.

But there are problems and limitations with software, mindware and knowledgeware when it comes to enhancing problem solving, decision- making and knowledge organization in a computer aided environment. This is still the case even though many individuals have pushed the state of the art in each of these individual areas, and in many cases, binary combinations. The following is a general discussion (not intended to be exhaustive or complete) of some of these advances.

In general summary, the problems and limitations come in the following areas: 1) Solutions typically take an "information-based" rather than a

"problem-based" perspective, 2) Solutions are usually fragmented and do not allow the smooth integration of expertise, knowledge, learning and information resources 3) Solutions tend to concentrate on the computer and work at the creation of "artificial intelligence" (e.g. automating or replicating human problem solving and decision making) rather than concentrating on systematizing both an individual's and a computer's cognition to create "assisted intelligence." Mindware

Generally, mindware solutions have typically been organized at a very high level of abstraction - too high to operationalize in a computer system without drastically limiting scope, scale and generalizeability. From the beginnings of the Artificial Intelligence movement, much more effort has been put by the world's scientists into teaching machines how to think than has been invested in new ways to help humans think more systematically; so they can work more effectively with machines.

One exception to this has been the American military, which has invested hundreds of millions of dollars in the means by which soldiers can be systematically trained to think, behave and react in a disciplined fashion. This results in more effective, efficient and predictable problem solving and decision-making under pressure and life or death. Equally large efforts have been put into testing methods and diagnostics to categorize and assess individuals based on intelligence or other desirable traits on a systematic basis.

The most intensive efforts, outside the military, to influence the systematization of human thinking have generally taken place in corporate consulting, training and education environments, where numerous frameworks and techniques have been developed to structure the thinking and problem solving and decision making of individuals and groups.

One class of these efforts - the multistep approaches - generally breaks problem solving down into a series of steps that must be rigorously cycled through each time a problem is solved. Another class - the key factors approach - generally identifies a simple number of key factors that should be considered again and again throughout a problem solving effort. A third class - the capability maturity model approach - generally identifies different levels of capability dependent on the explicitness or "maturity" of the work processes used to solve problems. A fourth class generally identifies differing personality types in the hopes that by fitting more compatible teams together, problem solving will improve. A fifth class - the decision making models - has generally been based on decision theories and tools that systematize the process of making choices. Another class has generally attempted to systematize thinking and problem solving through high levels of repetition while facing certain types of problems in order to routinize pattern recognition and decision-making.

The current mindware frameworks and methods usually propagated by consulting firms, authors, training firms or within enterprises are limited by scale and their ability to operationalize in a computer system. They break down in multidisciplinary problem solving efforts or projects that are large, complex or difficult. They also falter when the variety of problems faced over time is not very narrow. These breakdowns occur for a variety of reasons. Multi-step approaches break down largely because they are linear and can only account for the time dimension of problem solving, i.e. — treating it as a sequence of steps. This contrasts with treating problem solving as a progression of phases or state changes overlaid with a parallel set of optimization choices about the types of partial solutions that are right at any one particular stage of problem solving. Conversely, the key factor approaches break down because they fail to include the time dimension.

Capability maturity model approaches have to date been generally focused only on very specific professional domains, such as software engineering, procurement and human resources management. And personality type models work well for small teams, but generally have no relationship to the division of labor or organizational structures required for larger scale efforts. Intensive training courses often fail to deliver measurable results or lasting improvements in capability unless reinforced assiduously over time after the training event to increase retention and teach application. Also, they generally tend to lack a level of systematization that doesn't have the consequence of constraining their scalability and applicability to a wide range of problems (e.g., expert systems treated below).

As a result of the breakdowns of the currently existing solutions, mindware generally has not advanced enough to bring it to the point of full integration with software. Highly skilled and successful individuals, who have very sophisticated cognitive processes but do not happen to have computer programming skills, are often completely unable to infuse software with their own preferences and thought processes for end-to-end problem solving. Programmers themselves are oftentimes constrained by what their tools will allow, as opposed to setting a context for the tools they select and use with the way they think, make decisions and problem solve.

Software

The best software products generally work well both in isolation and while interacting with other programs. But software oftentimes has trouble working in concert with other software and programs are often oriented toward information, general collaboration, tasks or very narrowly defined problem or knowledge domains rather than at problem solving in general.

Officeware has generally tended to focus on targeted tasks or functions (calculation, word processing, and databases). Groupware, for example, has generally tended to concentrate on building the ability to manage large quantities of information and messaging between widely distributed groups of people that need to collaborate in groupwork. Teamware has advanced groupware capabilities to the point that work can generally be conducted and organized around the tasks required to execute a particular project. But these categories of software generally do not have built in computer-assisted decision making ability, nor do they typically provide an integrated problem solving context that ensures a project is tackled end-to-end, from a beginning state to a desired end state. Even ERP software, which has been successful at integrating large amounts of data and work process, is typically not problem oriented and generally does not build in extensive assisted decision making capability. Decision support software automates decision support — but this support is often outside the frame of reference of a problem.

Knowledgeware

Knowledgeware solutions have generally tended to focus on ever more flexible ways of representing data, manipulating ever richer forms of information and automated querying systems for increasing the utility of those data. Database management systems have generally concentrated on ever more flexible ways to organize and manage data and information at high volumes and rates of processing, with data organization following traditional functional (e.g. financial systems) or industry templates (e.g., auto manufacturing). Knowledge organization schemes (e.g., any knowledge organization or cataloguing scheme from the library of congress cataloguing system to meta tagging schemas that drive a particular data repository) are typically organized according to categories. They generally have no reference to actual problems being solved or the decision-making modes that humans use when they solve a problem.

Automated tools, such as search engines using case-based reasoning or latent semantic indexing, typically try to make up for the lack of foresight in organizing the knowledge in the first place and attempting to infer a problem solving context, rather than explicitly working with the user to create one. Many different organizations have been busy developing controlled vocabularies, meta-data frameworks, knowledge maps and multiple vector tagging schemas - but these efforts have generally been focused either on a particular company, industry or profession.

Mindware - Software Integration:

Integration of mindware and software has generally focused on either automated tools for very limited domains or non-automated frameworks and methods that can be taught with automated courseware. General problem solving frameworks have been developed for use in a multiplicity of professions - ranging from accounting and medicine to engineering and science. Books, courses and online courseware have been created to test for and develop human abilities in problem solving with such frameworks. For example, multi- attribute utility theory (MAUT) has been used for decision and analytic decision support to help humans make more effective and more systematic decisions. Separately, certain forms of human preferences have been used for a variety of applications.

Software - KnowledgeWare Integration:

Integration of software and knowledge-ware has generally tended to focus primarily on automated means of accessing, filtering, querying, sorting and ranking search returns. Statistical analysis, thesauri based lookup, NLP/IR and agents are commonly used to search and rank objects on the web. Other past approaches have included LSI, preference based searching, and modes and steps. Intelligent agents and other sophisticated query techniques that all fall under the category of data mining have generally succeeded in increasing our ability to extract meaning from knowledge in an automated fashion and make inductive inferences based on identifiable patterns in data.

Also known are "knowledge hubs" that bring together a combination of information, knowledge and learning based on a portal infrastructure, but which generally still fail to create a context for problem solving or decision- making.

Mindware - Knowledgeware Integration:

Mindware-knowledgeware integrations have generally tended to focus on intuitive, manageable ways to guide human interaction with extremely large data sets. Extensive work on automated and human indexing of knowledge resources has generally extended to complex multimedia objects and global scale data resources. It is also known to provide relative associative ranking of search returns based on attributes that relate objects of information to one another. Cataloguing schemes have been developed to guide the users through multiple layers of knowledge with numerous paths of interest open to them at each point along the way to follow (e.g. human maintained search trees).

Integration of Mindware, Software and Knowledgeware

Mindware, software and knowledgeware approaches have not generally been integrated into a seamless approach to computer assisted knowledge management, decision making and problem solving that is not limited by the scale of problem or the need to perform in a constrained, specialized domain.

Expert systems at the leading edge of artificial intelligence have generally automated human decision making and problem solving - and links to knowledge bases - but only in highly constrained problem domains because of the rigid, formal approaches taken to systematizing decision making rules or developing mathematical operators and algorithms for problem transformations. These solutions typically do not tend to be generalizeable across any domain, industry, and profession. In short, they are generally not universally applicable to all human problem solving. The state of that art today representing perhaps the furthest degree of integration of mindware, knowledgeware and software - expert systems ~ has generally been constrained and bounded by the following:

• highly formal methods that generally cannot deal with large- scale problems and ambiguous forms of judgment and decision-making;

• high degrees of targeted automation - artificial intelligence - as opposed to seeking a blend of human and machine intelligence - assisted intelligence;

• narrowly focused problem and knowledge domains and non- standard methods that generally make it next to impossible to organize them for large-scale use.

The unsolved need

In sum, professionals have been disappointed and inhibited by the complexity and poor integration of the mindware, software and knowledgeware available to them in the marketplace. They are forced to buy products, information, knowledge, and tools all from separate sources that fracture their attention and focus in problem solving.

They spend more time trying to get these pieces they need together, compensating for the lack of true integration, adapting narrow tools for relevance in new situations, and reinventing solutions to known problems, than they do continuously improving their capacity to solve known problems and to discover new opportunities. The creation and evolution of the World Wide Web holds out the potential to resolve the current situation - by creating access to a wider universe of software and data. But to date, it has generally continued with the traditional approaches ~ typically simply transplanting them to another medium/platform rather then reinventing them in novel ways.

A problem with computer and web interfaces today is generally a focus on manipulating information out of the context of problem solving. For example, some applications are targeted to individual problem or opportunity areas (for example, a computer aided design program that helps an aviation engineer design and organize the building of an aircraft). But general purpose interfaces (e.g., Windows, Browsers) are typically not designed the way humans solve problems, they are designed to process and use information and focus on tasks ~ which is only part of how problems get solved. Generally, they are out of alignment by focusing on users of computers rather than on solvers of problems. They are generally more about doing things the way the computer works, rather than starting with how the person works and making the computer completely transparent. CXOs and business professionals are ready for a wider range of choices and quality, offered in one place, with reduced knowledge costs and higher yields. The value for customers is the capability to solve bigger, tougher problems faster, better and cheaper ~ a better, lasting, evolving bottom line.

What is needed is a computer-aided tool and interface specifically designed to manipulate information for the purpose of managing knowledge, making decisions and solving problems and opportunities, without inundating the problem solver with functionality that is extraneous to that purpose. Such an ideal tool and interface would provide exactly the right partial solutions at every decision point in the context of a problem solving process. Such an ideal tool and interface would provide an unprecedented way to get information and tools more easily available and waste less time. It would be useful both in specialized areas of knowledge as well as across broad domains of problems, both small and large scale.

Our invention solves this long felt but unsolved need With a different approach ~ one that allows more user involvement, less constraining mathematical methods and a more flexible, generally applicable knowledge organization scheme — it is possible to push the state of the art into a new space. The space between pure human problem solving and artificial intelligence is what we call "Assisted Intelligence." See Figure 1A-2 The promise of "assisted intelligence" is to bridge a large gap that currently exists and thereby bring into being a whole new set of possibilities for creating and realizing human potential.

The different elements of this opportunity involve both advancing and then integrating the arts of mindware, software and knowledgeware to achieve a new problem solving system. Different pieces of the overall problem have been tackled before. But they have never all been solved together in a fully integrated and truly generalizeable form as described herein. Advances in increasing systematization of mindware and innovative organization of knowledgeware allow seamless integration of mindware, software and knowledgeware to provide a more complete solution than has ever existed previously.

The present invention stems from a realization that a path to resolution for any problem or opportunity can be conceived as a journey through a multidimensional space of possible routes and destinations. Tough, multidisciplinary, large-scale problems are never cracked with a single solution or "silver bullet." Nor are complex decision paths predictable enough to program search algorithms to move through their problem space in a deterministic fashion using current technologies. To complete a journey successfully when tackling the richer, more varied problem sets that make up most of professional work, individuals and teams make thousands of decisions/choices about partial solutions (e.g. knowledge, ideas, tools, people, resources & techniques) and how to apply them to define and achieve a goal. Each of those choices is based on a pool of knowledge objects that represent and are tied to means of marshalling a real partial solution (e.g. ordering, payment, fulfillment and delivery). It is possible to fully integrate mindware, knowledgeware and software into a novel, next generation computer aided problem-solving, decision-making and knowledge organization.

The present invention thus provides a system of problem solving that seamlessly interfaces with the logic structure of a machine while remaining accessible to large numbers of people for broad classes of problem solving.

We have distilled: • a general frame of reference,

• a universal or widely applicable set of principles, rules and values for making choices about potential solutions, and

• Categorization and distillation of those principles into a set of decision-making modes, that can be usefully linked to certain types of partial solutions.

The present invention provides implementations that incorporate both the frame of reference and the principles into the design of a new computerized software interface (e.g. web-based portal) ~ a next generation problem solving "power tool." It provides the first interface specifically designed to engineer human decision making preferences into an intelligent tool that assists directly in human problem solving across multiple professions, industries, problem sets and scales of problem solving. It furthermore provides a way to organize and leverage large amounts of knowledge represented in digital form for the purposes of decision making or problem solving.

One component of the interface is our ability to engineer essential problem solving judgment factors about choosing partial solutions - what we call "problem solving preferences" ~ into computer-based operational models and individual sets. They are used to drive an intelligent process for choosing relevant solutions that can be layered around traditional data retrieval mechanisms (e.g., Boolean queries, search engines) and implemented in the context of any computing platform (e.g., portal). The resulting Solver Computing System implementation provides a computer interface (e.g., web browser) that is designed to reflect how sophisticated professional problem solvers organize a problem and perform value and tradeoff judgments about alternative solutions. The Solver Computing System can be driven not just by predetermined solving preferences but also by those of any individual or expert problem solver.

The Solver Computing System offers a way to unlock rich sources of untapped judgment and expertise, and to engineer those preferences into a personalized problem solving tool. As a result, both individuals and entire organizations can significantly enhance their ability to solve tougher, more complex problems, and opportunities, faster, better, and cheaper.

The Solver Computing System implementation we have developed is a combination of problem solving framework and technology that have been blended together to provide a disciplined mental process embodied in a software engine that can: • provide an organized view of an entire community's work;

• offer universal ways of organizing that work and the resources necessary to complete it by problem(s) and/or type of solution; • assist that work with intelligent tools to solve, rather than just search; and

• personalize/customize the operation and look and feel for the solver so that he or she feels comfortable with an individual style. Our Solver Computing System implementation provides a software- enabled problem solving system and tool that enhances human problem solving capability ~ irrespective of the specific field, profession, industry or specialty of the problem solver. The interface does this through a unique combination of a frame of reference and judgmental factors that constitute a thought system and a technical system that applies the thought system in manipulating knowledge objects available on the World Wide Web and elsewhere. The Solver Computing System provides existing and novel technologies, in modular form, assembled in an original fashion into a unique portal architecture that does not currently exist. Any future module that enables problem solving via the web can be incorporated into the interface architecture.

Problem solvers can view, in one place, the entire spectrum of problem solving activity, from several points of view, that is occurring in their professional community with a unique portal knowledge architecture. Problem solving is enhanced by a Solver Computing System with the ability to "solve" for partial solutions to problems using preference sets engineered by problem solvers that sort, rank and filter large numbers of knowledge objects in the context of a specific problem situation and make them available for purchase. Problem solving models and individual preference sets can be engineered, displayed, mixed and purchased ~ creating a broad market for problem solving expertise that is completely unique. Collaborative work is enhanced and streamlined through a virtual "portfolio" that allows the collection of any digital objects with a simple "drag and drop" approach and corresponds to a particular problem. The portfolio is tagged accordingly, allowing it to be stored and retrieved as a problem ~ not simply a knowledge object.

Used in concert, these tools allow a problem solver to access more valuable knowledge objects much faster over a computer network, and to manipulate them in an easier, more useful manner. As a result, problem solvers increase their ability to get results, improve their overall capability and gain enhanced control and influence over complex problem solving processes at very large scales.

Introduction To Example Detailed Implementation

In more detail, our preferred Solver Computing System 50 (i.e. Solver Portal) implementation shown in Figure 1H can include components including:

• Solution System 100;

• Solver Engine 200; and

• Solution Well 300.

These components, when integrated with software collaborative toolkit(s) 400 (i.e., Solving toolkits), portal infrastructure 600 (i.e., Solving Portal infrastructure) and content resources 500 (e.g., prequalified content) provide a comprehensive web-based solving tool. In the subsections below, each component of an example Solver Computing System 50 implementation is described, along with its respective operational elements and some of its advantageous features. An Example Solution System 100

An example solution system systematizes the methods and approaches to human problem solving (mindware) and, in fact, to any existing problem solving frame of reference. This is done in such a way as to make them possible to integrate in a systematic way with computer-based decision theories/tools (software) and to link those problem-solving and decision-making tools to knowledge objects for purposes of processing and manipulation. The solution system of one preferred exemplary implementation includes (See Figure 2): • a context matrix 102;

• a preference model 104;

• individual preference sets 105; and

• solver personalities 108.

Generally, the solution system 100 of the example embodiment performs the following functions:

• Map any problem solving system into a context matrix, including problem solving phases and judgmental decision-making modes.

• Use multi-attribute utility theory as the backbone of a computational mechanism • Build a preference model (objective function) and map its attributes to the information/data attributes

• Collect preferences of human experts and store in preference sets that can be invoked by solver engine in ranking and filtering partial solutions (i.e., knowledge objects) • Use end user preference for guidance

• Describe information/data with common set of attributes An Example Context Matrix 102

The context matrix 102 concept advances systematization of mindware by providing a problem solving framework that incorporates judgmental decision-making modes and time phasing. This allows building a goal hierarchy and observable measures using software tool(s) organized according to a decision theoretic approach (e.g. multi-attribute utility theory (MAUT)) that can in turn assist decision making throughout the entire context of solving a problem. Features of the context matrix 102 include:

• Generalized form of any problem solving method • Creation of problem solving frame of reference based on modes and phases

• Formulation of general goal hierarchy based on modes, principles and derived from matrix

An Example Preference Model 104 The preference model 104 advances systematization of mindware.

Using software, the human problem solver engineers goals and measures defined in a preference model, defining a personal set of utilities and tradeoffs, for a general or specific problem. Features include:

• Creation of specific goal hierarchy that represents problem solving method by mode and/or phase based on context matrix

• Application of decision theory/tools (e.g., MAUT) to problem solving (preference model)

• Mapping of preference attributes to knowledge object attributes

Example Individual Preference Sets 105 Individual preference sets 105 are instantiations of the problem solving preference model 104 that embody the judgments of an individual person either in general or in relation to a specific problem or set of problems. These decision-making preferences are used by the solver engine 200 to filter and rank. Features include:

• Elicitation of individual decision-making and problem solving preferences • Formulation of computerized preference set based on preference model that applies preferences

• Storing and invoking of individual or multiple preference sets

Example Solver Personalities 108

The Solver Personalities component 108 integrates mindware and software. It allows development of nearly infinite variety of customized and generally useful "personalities" that embody the problem solving preferences involved in a particular job, a problem set, or simply an individual with unique style or approach in a widely-applicable preference set. (See figure 7) It also allows these to be mixed and matched and purchased/sold. Using profiling and personalities, any individual can engineer their expertise or take advantage of someone else's "pre- engineered" expertise for the purposes of computer-assisted problem solving. Features include:

• Personalities based on problem solving preferences • Development of personalities types (process and architecture)

• Choosing and filtering personalities for use in driving a solver engine

• Mixing and matching personalities

• Buying and selling personalities

An Example Solver Engine 200

The Solver Engine 200 provides an intelligent problem solving mechanism that delivers a certain number of "best" partial solutions (e.g., knowledge objects) in context for the problem that the user is trying to solve. These objects or information may come from any automated data source (e.g., any combination of the Web, an organization's internal databases, pre-approved third party data, or other sources.) Unlike popular search engines in use in the consumer market which typically turn up large numbers of irrelevant "matches" to queries based upon word choice out-of-context, the example Solver Engine 200 ensures that knowledge vetted and displayed for the problem solver will be highly relevant to his or her work product. The example Solver Engine 200 "solves" rather than "searches" by sorting returns from search engines and ranking/ordering them based on preferences of human problem solvers in context.

The Solver Engine 200 of one exemplary preferred implementation includes an integration of mindware, software & knowledgeware providing aggregate algorithms that synthesize utility functions and tradeoffs in an overall set of relative values used to rank returns input by any data retrieval mechanism (e.g., search engine). Ranking is preferably performed according to overall utility as computed by application of a decision theoretic approach (e.g., MAUT) that allows the expression of a multi- attribute utility function. Sorting is by type and utility of partial solutions and includes a wide variety of use types. Features include:

• Using utility computed from preferences to rank results

• Combination of Solver Engine with at least one data retrieval mechanism (e.g., any search engine) • Integrating Solver Engine in an existing user interface and computing platform (e.g., www browser and portal) The Solver Engine 200 implementation includes a solutions array that integrates mindware, software and knowledgeware. Sorted returns are organized/presented by the engine in both use type and mode, top rankings, easily available for review, assessment and purchase. Features include:

• Use types for classifying knowledge objects by the type of partial solution • Arranging use types by decision-making mode

• Rank or organize information/data being returned by searches

• Search results dynamically displayed by mode and use type

Briefly, one preferred example solver engine 200:

• is driven by preferences and profiles (of both problem solver and the problem itself) from solution system;

• operates on knowledge objects aligned with and served up by data retrieval mechanism (e.g., search engine) from solution well;

• is initiated by solving query — question or decision about any aspect of the problem or opportunity being solved; • returns ranked knowledge objects, organized by mode and use type, that represent full range of possible partial solutions to the problem; and

• includes wide range of solving functionality from simple queries operating off of general preferences to customized profiles and specialized, designed or composite preference sets tuned to phase and mode of problem solving activity.

An Example Solution Well 300

In an example implementation, a Solution Well 300 provides a completely flexible means of storing and organizing any piece of digital information around a problem-solving effort. The portfolio component of the solution well 300 represents all data and problem solving analysis related to a team or individual project, organized around the problem being solved. Upon completion of the problem solving effort, the portfolios may be meta-tagged, centrally archived as a group and made available for future search, retrieval and purchase by approved problem solvers facing similar problems. Multiple sets or sequences of solver queries can also be tagged, and made available for retrieval and purchase.

Generally, example solution well 300 serves as the data, knowledge and information repository for a solver portal or other system. The solution well 300 searches, gathers, associates, integrates, and distributes data and knowledge from multiple, heterogeneous sources such as for example: • open sources such as the WWW and Internet; and

• reliable sources such as Gartner Group, Doculabs, etc. The example solution well 300 enables content management, knowledge curation, knowledge sharing and interchange via solving toolkit collaboration tools and economic mechanisms. It provides decision-making support using the Solver Engine 200. It may also provide active notification of key events; message and discussion groups; and knowledge auctions using domain expertise certificates. The example Solution Well also serves to foster the creation and dissemination of value-added strategic knowledge to solvers using the solver portal. The Solution Well system of one preferred exemplary implementation includes:

• data schema

• solutions well data structure

• portfolio system

Example Data Schema

Data schema are provided that advance organization of knowledgeware. This involves identification of "use types" (i.e. types of ^■ partial solutions used in solving problems), construction of metatag extensions to a base schema model, definition of data mappings (transformations) from metatags to preference attributes, and physical tagging (metatagging) of knowledge objects. Features include: • Schema extension that links to decision theory/tools

• Typology of use types by problem solving modes

• A data representation of the observable attributes of the knowledge objects that correspond to the terminal elements of the goal hierarchy, (sometimes there is no real (or obvious) "correspondence"; rather, the correspondence must be computed.)

• A type of metatagging scheme by which data retrieval mechanisms (e.g. search engines) can automatically recognize the attributes of knowledge objects and solver engines can rank the knowledge objects according to the values of the metatags.

• A data mapping (transformation) between the preference attributes and metatags.

Example Solution Well Data Structure

An example Solution Well 300 data structure (which integrates mindware with knowledgeware) creates wider, more varied source of partial solutions ~ not just knowledge objects ~ that can be sorted through for a solver to help him or her solve a problem more efficiently. Data is tagged according to schema. The well is manageable using knowledge management tools and techniques. Content is prequalified, marked and tracked in unique ways. Features include:

• Organization of knowledge by classes of problems

• Schema based on goal hierarchy • knowledge objects organized by solution and/or problem attributes

• Inheritance hierarchy of knowledge objects

The example solutions well 300 is a repository (actual or virtual) of data from heterogeneous sources including for example multimedia documents, reports, web-links to reliable sources, imagery, technical drawings, and discussion group messages, to name a few. It provides a storehouse of knowledge in the form of heuristics, rule bases for expert systems, video clips from experts, best-practice case studies, etc. A meta- data index provides both syntactic and semantic links to data and knowledge resources. A rich knowledge meta-model is used to handle the diverse types of information. This rich meta-model is based on the coexistence of several models and tools via a federated schema that uses an object meta-model.

Example Portfolio System

An example portfolio system integrates knowledgeware with software to link knowledge objects in the specific context of a problem or opportunity or set of problems and opportunities. It allows a library to be built of problems-solved and associations between data usage and problem solving preferences to be utilized in autogenerating problem solving knowledge. Features include:

• Packaging of knowledge, or query sets, by problem or opportunity

• Tagging and organizing of knowledge packages • Autogeneration of problem solving knowledge packages

• Presentation of problem portfolio and packaged solutions

A contributor publishing system comprising a template system and a portfolio creation system can be used by application/content developers to prepare specialized content, index, organized and packaged into portfolios. Design Templates provide the structure and operations for content creation, editing, content tagging, linking, integration, publication, and dissemination. Portfolios provide a metaphor for aggregating, organizing and indexing diverse data types, formats, and content. Contributors use these tools to provide domain-specific content for course modules, for specialized Knowledge Forums, etc. Multimedia such as streaming audio/video plays an important role in conveying knowledge such as best practices, case studies and interviews with certified experts.

A Fully Integrated Solution

Together, these elements - the Solution System 100, the Solver Engine 200 and the Solution Well 300, combined with pre-qualified content 500 and a widely accessible computing platform 600 (e.g., portal infrastructure) ~ constitute a complete computerized problem solving interface 50 (e.g. web-based portal). It provides the next generation of interfaces that will mediate between human problem solvers and computers in the Web age and beyond ~ offering professionals a way to work more effectively and efficiently with the people, tools and knowledge assets available via the Web, to solve more challenging problems. With a fully open architecture, this interface is powerful and flexible enough to incorporate any existing computer-based tool (e.g., Web-based tool) or application on the marketplace today.

An example Solver Computing System 50 (i.e. Solver Portal) implementation provides a one-stop workplace for any professional community that is solving problems. While many professionals work in relative isolation from their organizational stovepipe or industry, the Solver Computing System 50 will help them to discover similar problems on which others are working. For example, client applications can include communities within and across large enterprises (e.g., CIOs and CFOs) to collaborative workgroups across the value chain (e.g., the many participants in business financing and deal making).

The Solver Computing System 50 enables collaborative problem solving among multidisciplinary teams by enabling the exchange of tools and information that these teams are sharing, and by providing information to individual problem solvers regarding the status of their team members' problem solving efforts.

The Solver Computing System 50 also includes online learning and professional development modules. The infrastructure provides a portal foundation incorporating portal management, security, knowledge wells, e- cornmerce, database and server services. Other example features of a preferred Solver Computing System 50 implementation include:

• Multidimensional selection of multiple menu items and immediate navigation to that space

• Directed focus for not just information but decisions, transactions and crises

• Automatic charting of position in problem life cycle, not just place in infospace • Able to dynamically telescope information for different platforms and channels

• Meaning, not just form ~ Able to integrate speech recognition interface with mindware vocabulary schemas

• Smart and helpful, not just fast ~ Embedded artificial intelligence agents combined through assisted intelligence to guide/assistant quality of problem solving assistance

An example solver portal may include the following:

• General Portal Components Search

Link management Data repositories Content management Information security and privacy features Portal management Directory services Online Learning Courseware, authorware Virtual instruction

Collaborative Tools (e.g. groupware, officeware) Scheduling Expert Finding Personalize, customize Communities Content Information organization tools

Example Solver Computing System Process Flow

Figures 1J-1 and 1 J-2 are together an overall process flow diagram of an example process performed by solver computing system 50. This process flow diagram shows horizontal process activity zones performed by the following example participants:

• community/organization

• experts • human solver

• computer software

• knowledge objects • content providers.

For example, the operations shown within the "computer software" horizontal zone are performed by computer software (and/or hardware) components of system 50. The tasks within the "human solver" horizontal zone are performed by the person using solver system 50 to solve his or her problem. The tasks shown within the "experts" horizontal zone are performed by someone with problem solving expertise (e.g., an expert problem solver, someone eliciting information from an expert problem solver, or in some cases, the human solver himself or herself). The tasks shown within the "community organization" horizontal zone are performed by participants within a community and/or organization (e.g., a company, taskforce, etc.). The tasks shown within the "knowledge objects" horizontal zone are performed automatically and/or manually on partial solutions such as knowledge objects within the solution well. The tasks shown within the "content providers" horizontal zone are performed by one or more content providers providing partial solutions and/or other content 500 for delivery via solution system 100.

The "solve" process flow for the human solver as shown in this diagram is a similar but more detailed version of the steps shown in Figures IB and II discussed above. This flow is indicated by the "solve" paths of the process flow diagram.

The human solver starts to solve a problem (block 1002) by first choosing the problem (block 1004) and then logging in to the user interface provided by solver system 50 (block 1006). In response, solver system 50 displays a "home screen" to the human problem solver (block 1008). The human solver may orient himself or herself within a particular solver community and/or create profiles by selecting various items from the home screen (block 1006). The human solver may then choose an appropriate preference set (block 1010) which solver system 50 delivers on demand (block 1012). A purchasing transaction may be involved, i.e., the human solver may need to pay a fee to "buy" and/or have the right to use the chosen preference set (block 1010). The human problem solver may then formulate a solver query and specify mode and phase of a problem solving progression (block 1014). In response to the query, the mode and phase information and the selected preference set, the solver engine 200 computes utility, and activates a search engine 202 to access the solution well data repository (block 1016). Knowledge objects in the data repository may be linked to original content databases via the Internet or other computer network or through some other mechanism (block 1017). The solver engine 200 may then compute utility of the information returned by the search engine; rank the returned items based on utility; and present an array of partial solutions to the human problem solver (block 1018). The human problem solver can select one or more of the partial solutions presented and may be charged a fee to review or purchase some of these items (block 1020). A conventional purchasing/transaction module (e.g. e-commerce) may process purchase transactions (block 1022). The human problem solver may collaborate with colleagues (block

1024), using a "solving tool kit" 400 that the solver computing system 50 makes available to the human solver (block 1026). The solver toolkit 400 may make interactive applications such as online learning or the like available from a third party application server (block 1027). The human solver may iterate (path 1028) until the problem is resolved and an implementation is defined (block 1030).

The "build" paths of the process flow diagram generate the cognitive, software and data structures used to support the "solve" paths described above. For example, the designers of system 100 initially design a portal infrastructure (block 1032), and also create a model of problem solving progression and decision modes (block 1034). One or more preference sets are engineered (block 1036) based on the model, and these preference sets are loaded into the solver engine 200 (block 1038) so they can be delivered to the human solver at block 1012. The problem solving model may be based on an identification of the problem solving method(s) used by the particular community organization (block 1040). The solution well 300 may be constructed by identifying prequalified content (block 1042), tagging the objects with appropriate metadata and organizing them by use types (block 1044), and inputting the objects into a data repository (block 1046).

Through this entire process, system 50 uses conventional machine learning techniques (block 1050) to learn about the processes being performed. Machine learning (block 1060) coupled with explicit system feedback from the human problem solver (block 1052) may be used to optimize the process for later problem solving sessions - making the solving process more efficient and allowing the problem solving patterns of prior users to benefit later users.

Detailed Architectural/Operational Description Example Solution System 100

Figure 2 is an overall block diagram of solution system 100. As shown in Figure 2, an example solution system 100 includes the following elements:

• Context Matrix 102 (Create context matrix as frame of reference for problem solving)

• Judgmental Modes 103 A (Create judgmental decision-making modes that serve as basis for goal hierarchy) • Goal hierarchy 103B (Create goal hierarchy as basis for generating preference model)

• Preference Model 104 (Preference attribute domain capture, Mapping preference attributes to knowledge object attributes.) • Preference Set 105 (partial solution utility capture, capture of problem solver tradeoff weights)

• Solver Profiling 106 (Profiling of problem solver and problem solving activity)

• Problem solving personalities 108 (Creation of discrete solving personalities)

Briefly, context matrix 102 provides a representation of a how a human problem solver solves problems, in terms of problem-solving phases and modes. The preferred embodiment solution system 100 uses the judgmental decision-making modes 103 A that are part of the context matrix 102 to develop a goal hierarchy 103 A that sets out a systematic structure for valuing partial solutions at key decision points in a problem solving process. The goal hierarchy 103A is used to build a more specific preference model 104 - which in the example embodiment is a hierarchical goal structure defining preference attributes at an observable level. Solver profiling 106 allows a problem solver to define certain attributes of himself, as well as provide descriptions of the problem he is trying to solve, that are useful in presenting useful preference set options and in semantically bounding data retrieval (e.g. organization, professional specialty, level of experience). Problem solving personalities 108 provide a choice of concentration on a certain type of specialized decision focus in case a problem solver wishes to move beyond a specific preference set to a more generic one that focuses on a particular class of problems, a mode of decision-making, a particular domain or a type of job. Using a preference elicitation process, the preferences of human problem solvers are then engineered, using the structure of preference model 104 and personalities 108 into a specific preference set 105 that can be used by the Solver Engine and brought to bear to allow a user to solve a particular problem.

Example of Problem Solving Context Matrix

In this example, the context matrix 102 provides a problem solving frame of reference that guides determination of a goal hierarchy and observable measures using software decision-support tool(s) organized according to a decision theoretic approach (e.g. multi-attribute utility theory (MAUT)). Context matrix 102 allows formulation of a goal hierarchy based on decision-making modes derived from the matrix.

In this example, a context matrix 102 is based on the integration and interrelation of decision-making modes (i.e. where certain types of choices are optimized for) and phases of a problem solving effort over time. There can be any number or type of phases in a context matrix - which can be uniquely constructed according to a particular problem solving method to emphasize its specialized characteristics. For example, a context matrix for the Marine Corps, operationalizing their OODA loop (Observe, Orient, Decide and Act) may be shorter, more action-oriented and provide different phases than a context matrix for a large company following a more lengthy method. In the latter situation, the context matrix would likely have more phases and be more deliberative and analytical.

Similarly, any number of decision-making can be used A context matrix could be made up of just a few, six or even more in a very complex decision-making process. In fact, we envision the possibility of context matrices that go beyond mere two-dimensional form and add third, fourth or even more dimensions. For instance, a third dimension might include sub-problems, which are prioritized on an ongoing basis according to triage criteria. Such a three-dimensional matrix would set the context for the type of decision-making exhibited by an emergency room physician tackling dozens of problems at the same time with limited resources.

Figure 3 A shows one example problem context matrix 102 (in two dimensions) including six columns and six rows. The columns define problem solving "phases ," and the rows define problem solving "modes." The Figure 3 A problem solving matrix 102 includes the following exemplary rows corresponding to problem solving and decision-making modes: • ideas

• knowledge

• relationships

• problems

• capabilities • results

The example problem solving matrix 102 of Figure 3 A includes the following exemplary columns corresponding to problem solving "phases":

• orientation

• selection • initiation

• execution

• completion

• evaluation

One concept behind problem solving matrix 102 is that a human problem solver will, at any particular time in the course of solving a problem, be operating at an intersection of the context matrix (e.g. a particular problem solving mode and in a particular problem solving phase). Each such intersection, or in this example, combination of phase and mode, drives preferences for certain types of partial solutions. The process of solving a problem can thus be reduced to the pattern of intersections chosen by a particular problem solver as he moves through the modes and phases of a problem. Figure 3A also shows an example problem life cycle that traces a path through the 36 cells of matrix 102. Patterns of movement through a problem solving effort are universal and applicable to any type of problem, where problem or opportunity is defined as any intentional change of state in a system. The patterns range from simple and linear for straightforward problems (for example, applying for a job) to complex and non-linear for larger scale, more difficult problems (e.g. starting a new company).

As an example, a human problem solver may require some knowledge about a particular area before he or she has enough information to define it well enough to proceed toward resolution. This typically happens at the beginning of an effort - in the orientation phase. The human problem solver will thus be operating in the "knowledge" mode of problem solving matrix 102 (see Figure 3 A node Bl within the "knowledge/orientation cell).

Continuing on, the problem solver may need to tap into his personal network of relationships to help bring people to bear that will help initiate a resolution of a problem. In this case, he will be operating in "relationship" mode of problem solving matrix 102 (See figure 3 A, node C3 within the "relationship/initiation" cell.)

Eventually, once a capability has been marshaled and people are brought onto a team, the problem solver will begin executing a resolution of the problem, following protocols or tactics that will help him move through the execution phase into completion. In this case, he will be operating in "results" mode of problem solving matrix 102 (See figure 3 A, node F4 within the "results/execution" cell) Finally, once the problem has been solved and the problem solver is concentrating on evaluating his results and building knowledge to apply to the next problem or opportunity, he will begin accumulating knowledge and learning that comes from an "after-action" problem assessment. In this case, he will be back in "knowledge mode" in problem solving matrix 102 (See figure 3 A, node B6 within the "knowledge/evaluation" cell)

The human problem solver may freely move around in the matrix 102 in the course of solving a problem, switching rapidly from mode to mode and phase to phase as he or she attacks different aspects of the problem.

The following describes each of the six phases of the example context matrix 102 shown in Figure 3 A:

Orientation The goal of the orientation phase is clarity and transparency of any problems or opportunities in the solver's environment - - especially if that environment is new and/or dangerous. In this phase, problems are merely identified and opportunities uncovered. No selection is made of what to work on when.

Selection The goal of the selection phase is specificity and selectivity as to which priority problems and opportunities are likely to yield the highest risk-adjusted returns and possibilities for the solver in the future. Similarly, this phase involves prevention activity that is necessary to prevent problems from occurring at all.

Initiation The goal of the initiation phase is to define the problem more specifically and begin to choose/fashion potential solutions, as well as begin the process of defining just what success in solving the problem will mean. This is also the phase where resources begin to be marshaled, including in-depth knowledge.

Execution The goal of the execution phase is to implement the plan and approach designed in initiation, and to exceed expectations if at all possible. It is highly iterative, with considerable energy and attention going into both building initial momentum and "breaking the back" of a problem early. Significant adaptability and resourcefulness is required here should goals or conditions change. Completion The goal of the closure phase is to resolve the problem successfully, gain agreement that it has been solved, ensure that elements of a solution do not unravel and are likely to endure for the time anticipated. This phase involves considerable attention to proving and measuring resolution as well as contrasting it with previous starting points or milestones. Closure also involves some attention to the problem after next and laying the foundation for future efforts.

Evaluation The goal of the evaluation phase is to extract maximum knowledge and learning from the problem solving effort for use by others or in the future. It includes identifying lessons learned, best and worst practices, reusable solution components, reflection on implications of what has been done for other interested parties and formal evaluation of the risk-adjusted returns on investment/expenditure for the problem effort as a whole.

In this example embodiment, the six modes of context matrix 102 represent optimization choices — metachoices that heavily influence decision making about partial solutions. They are modular but work as an integrated system. They apply throughout the problem phases and are valid for small and large problems (i.e., they are fully scalable). They are associated with roles & responsibilities, and they operate at all levels — corporate to individual. The following briefly describes each of the six modes of example context matrix 102 of Figure 3 A:

Ideas Creating teams and organizations that are innovative and decisive. Optimizes for ideas - goal is creativity and innovation. Emphasis is placed on creative formulations, play, ideas and concepts, paradigms, and humor. Time, any type of goal, structured knowledge, completed solutions and practical resources are less important.

Knowledge Acquiring and absorbing strategic knowledge on a continuous basis. Optimizes for learning - goal is acquiring and absorbing strategic knowledge for the right person in the right form at the right time on a continuous basis. Emphasis is placed on completeness, depth and accuracy of knowledge, curiosity and connections between things. Play, concrete actions, and decisions are less important.

Relationships Building trust and loyalty by delivering value. Optimizes for human relationships - goal is building trust and loyalty. Emphasis is placed on people, organizations, feelings, communication, language and group identity. Problems, strategy, complete solutions are less important.

Problems Working on the right projects, for the right reasons at the right time. Optimizes for focus on a particular problem or opportunity. Goal is working on the right problems, for the right reasons at the right time. Emphasizes goals, strategies, routes and paths, planning and scenarios, sensitivities. Relationships, play and ideas are less important. Capability Designing end-to-end, complete and well-supported solutions. Optimizes for a solution with maximum capability to solve multiple problems - goal is designing end-to-end, complete, widely- applicable and well supported solutions. Emphasizes technology, integration, design, modularity, resources and specifications. Relationships, general knowledge, narrowly defined problems are less important. Results Implementing solutions effectively in complex, competitive situations. Optimizes for results to be achieved in resolving a particular problem situation - goal is implementing solutions effectively in complex, competitive situations. Emphasizes time, practical steps of execution, protocols, barriers, troubleshooting, adaptation, risk management. Concepts, ideas, play and more in-depth knowledge is less important.

Here are some general observations about the way a human problem solver typically moves through example context matrix 102 in solving a problem:

• Solver moves from current state to end state

• Solver is always in only one phase and mode at any one time but may repeat or skip phases as solutions move to scale

• Phases can be repeated or skipped as often as necessary, as judged by the solver (e.g., especially in scaling a solution)

• Each mode/phase pair represents a change in the state of both the problem solver and the problem situation; they are not action steps

• Each phase can be broken down into sub-phases that ultimately correspond to action steps

• Any existing multi-step problem solving process can be mapped against these phases and modes.

• The solver can enter the process at any point, but has proportionally less influence on outcomes the later it is • Decisions are required to make choices about partial solutions in order to progress toward goal state

• Hundreds, even thousands of decisions are required to move from begin to end states, for each journey

• Any problem journey can be represented by some or all units of the matrix

• Preference model can be built for choices made by phases or by mode or both. Figure 3B defines some of the various activities a human problem solver may be involved in as characterized by different cells (i.e. intersections) of the example context matrix 102. As an example, when the human problem solver is operating in the "orientation/knowledge" cell, he or she may be trying to spot patterns and/or assessing what people or organizations ought to be involved. Moving to the "initiation/knowledge" cell may involve recognizing key obstacles and getting to the heart of the problem. As another example, the "initiation/problems" cell may involve setting goals, defining success, and developing a strategy and path. The "results/completion" cell may involve mop up, organizing, and setting perimeters.

Example Goal Hierarchy Definition and Construction of Preference Model and Preference Set Using Problem Solving Context Matrix

In this example, solution system 100 creates a goal structure that is specifically designed to enhance decision-making in solving problems. Figure 4 shows an example process 110 for creating a problem-solving preference model 104 based on a decision theoretic approach (e.g., MAUT) and the judgmental modes 103A from context matrix 102.

Process 110 includes three steps and produces a goal hierarchy 103B and a preference model 104 with design input from personalities 108:

• create or define a goal structure (112),

• define preference attributes to make goals and subgoals within the goal structure observable (114),

• define units (i.e., domains and ranges) allowing attribute mapping (116),

Process 120 includes two steps and produces an individual preference set 105 based on preference model 104:

• assess utility of each preference attribute (122), and • perform tradeoffs to determine relative weights among preference attributes (124). In this example, the "create goal structure" step 112 creates a problem-solving goal structure that will serve as a basis for a preference model. In this example, this goal structure is hierarchical, and reflects a particular problem solving methodology. Step 114 makes sub-goals within the goal structure "observable" by defining measurable preference attributes for the sub-goals. The preference attributes defined by step 114 - which may be represented by "leaves" in the goal hierarchy "tree" ~ are used to measure the satisfaction of goals and sub-goals in the goal hierarchy. Step 116 defines units (domain and range) of each of the preference attributes. Figure 4A shows an example architecture for solution system 100 in creating a preference set. In this example, a block 100-1 defines a goal hierarchy 103B, and an input block 100-2 receives input from a user as to personalities, tradeoffs and preferences. A modeler block 100-3 creates a preference set 104 using multi-attribute utility theory. An output block/data converter block 100-4 converts the preference setl04 into a data file (e.g. flat ASCII file) or other form so it can be directly used by solver engine 200. Step 122 assesses the single-measure utility function for each of the various preference attributes to determine the "worth" of the preference attribute values. Step 124 performs tradeoffs to determine relative weighting among preference attributes.

Both the goal structure/hierarchy 103B and the preference set 105 may be constructed, for example, based on eliciting information from an expert in a particular field. For example, experts in solving particular problems may be interviewed according to a consistent methodology (e.g., based on a questionnaire), and this information can be used to help define the goals in the goal structure hierarchy (and/or the preference attributes used to measure the satisfaction of those goals and/or and the relative weighting among those preference attributes).

In terms of some practical consideration, preference elicitation may generally be facilitated by using a bias sensitive, cooperative facilitator. It may help to have agreement on overall goal structure, measure definitions, and the model. One approach is to simplify and organize the elicitation process in "tell a story" and/or "Q & A" approach.

The elicitation process typically achieves highest yield when applied to measurably "best" or "representative" individuals. The process generally works best one-on-one; it is less effective in eliciting preferences from teams of people. The resulting preference model can be used to evaluate solutions on either a one-shot or a reuse basis; it can be used to implement policy and quality assurance by providing a methodology for consistent capture and consistent application; and it promotes an understanding of the risk/reward profile of the decisionmaker.

Create Goal Structure/Hierarchy 103B

Step 112 creates a goal structure or hierarchy 103B having a goal of "excellent problem solving." A particular problem solving framework represented by a context matrix 102 is used to define sub-goals within that goal hierarchy 103B.

Figure 5 shows an example partial simplified goal hierarchy 103B that reflects the modes and phases of problem solving matrix 102. The Figure 5 example goal hierarchy defines the "excellent problem solving" goal 132 in terms of six sub-goals corresponding to the six problem solving "modes" of matrix 102:

• best ideas 134(a),

• best knowledge 134(b), • best relationships 134(c),

• best direction 134(d),

• best capability 134(e), and

• best results (execution) 134(f). In the example embodiment, each of these goals 134 corresponds to a mode within context matrix 102, but other examples and implementations can define different relationships and structures for the goal hierarchy and its relationship to the context matrix.

The Figure 5 example goal hierarchy 103B provides sub-goals for each of the six problem solving sub-goals 134.

As an example of a general problem solving goal hierarchy developed at least in part through such elicitation, the "best ideas" goal 134(a) of Figure 5 may have the following subgoals 134(a)(l)-(a)(5):

• right impact 134(a)(1) • right perspective 134(a)(2)

• right presentation 134(a)(3)

• right limits 134(a)(4)

• right tradeoffs 134(a)(5).

The overall impact of these subgoals 134(a)(l)-(a)(5) is to provide optimization for ideas, with a goal of creativity and innovation. Emphasis is placed on creative formulations, play, ideas and concepts, paradigms, and humor. Time, any type of goal, structured knowledge, completed solutions and practical resources might be considered to be less important.

As another example, the "best knowledge" goal 134(b) may have the following associated sub-goals 134(b)(l)-134(b)(4):

• right subject 134(b)(1),

• right scope 134(b)(2)

• right level 134(b)(3), • right form 134(b)(4).

These subgoals 134(b)(l)-(b)(4) provide for optimized learning, with a goal of acquiring and absorbing strategic knowledge for the right person in the right form at the right time on a continuous basis. Emphasis may be placed on completeness, depth and accuracy of knowledge, curiosity and connections between things. Play, concrete actions and decisions might be less important.

The "best relationships" goal 134(c) may have the following subgoals 134(c)(l)-(7) as one example: • right guidance 134(c)(1),

• right style 134(c)(2),

• right values and character 134(c)(3),

• right number 134(c)(4),

• right mix 134(c)(5), and • right commitments 134(c)(6).

These subgoals 134(c)(l)-(6) may optimize human relationships by building trust and loyalty. Emphasis may be placed on people, organizations, feelings, communication, language and group identity. Problems, strategy, and complete solutions might be considered to be less important in this example.

The "best direction" goal 134(d) may have the following example associated subgoals 134(d)(l)-(6):

• right problems 134(d)(1),

• right endgame 134(d)(2), • right goal 134(d)(3),

• right paths 134(d)(4),

• right strategy 134(d)(5),

• right complexity 134(d)(6). These subgoals 134(d)(l)-(6) may optimize for focus on a particular problem or opportunity by working on the right problems, for the right reasons, at the right time. Emphasis may be on goals, strategies, routes and paths, planning and scenarios, and sensitivities. Relationships, play and ideas may be considered less important.

The "best capability" goal 134(e) may have the following example associated subgoals 134(e)(l)-(6): right assets 134(e)(1), right people 134(e)(2), right organizations 134(e)(3), right efficiency 134(e)(4), right accessibility 134(e)(5), and right agility and stability 134(e)(6). These subgoals 134(e)(l)-(6) may optimize for a solution with maximize capability to solve multiple problems by desigmng end-to-end, complete, widely- applicable and well supported solutions. Emphasis may be on technology, integration, design, modularity, resources and specifications. Relationships, general knowledge, and narrowly defined problems may be considered less important. The "best execution" goal 134(f) may have the following example associated subgoals 134(f)(l)-(6): right control 134(f)(1), right schedule 134(f)(2), increase options 134(f)(3), appropriate risk 134(f)(4), manage resources 134(f)(5), and proper tactics 134(f)(6). These subgoals 134(f)(l)-(6) may optimize for results to be achieved in resolving a particular problem situation, by implementing solutions effectively in complex competitive situations. Emphasis may be on time, practical steps of execution, protocols, barriers, troubleshooting, adaptation, and risk management. Concepts, ideas, play, and more in-depth knowledge may be considered to be less important to achieve these subgoals. Each of the goals within the Figure 5 example goal hierarchy 103B is articulated in an extendible way through any number of intermediate sub- goal levels. The Figure 5 goal hierarchy 103B can have any number of levels.

Figure 6 shows the Figure 5 example goal hierarchy 103B extended to have additional levels in the way of sub-sub-goals, sub-sub-sub goals, etc. For example, Figure 6 shows the "proper scope" sub-goal 134(b)(2) having the following associated sub-sub-goals:

• coverage 134(b)(2)(l),

• proper depth 134(b)(2)(2), • right geography 134(b)(2)(3),

• appropriate age 134(b)(2)(4), and

• proper quantity 134(b)(2)(5).

Goal hierarchy 103B is expandable and scaleable depending on the application.

Define Preference Attributes to Construct Preference Model

The resulting hierarchical goal tree structure (which can have any number of levels) terminates in "leaves" comprising measurable preference attributes 136. Each level in the goal structure hierarchy can consist of any number of goals in combination with any number of preference attributes 136. Preference attributes 136 can be elicited from an expert in solving a particular problem, or they may be obtained in other ways (e.g., through survey results or other processes). The preference attributes 136 provide a mechanism for measuring ("making observable") whether the associated goal (sub-goal, sub-sub-goal, etc.) has been satisfied.

In this example, the "coverage" sub-goal 134(b)(2)(l) has two associated preference attributes: • breadth attribute 136(1), and

• temporal coverage attribute 136(2).

Define Units Allowing Attribute Mapping for Preference Model

Step 116 of Figure 4 defines units (domain and range) of each of the preference attributes 136 within goal hierarchy 103B. The first step in the example implementation is to capture the domain for each preference attribute, and uniquely identify each preference attribute. In this example, the unique identification of preference attributes is accomplished by assigning a unique positive integer to each preference attribute. The integer can act as an index into a vector of objects representing the preference attributes. For purposes of efficiency, the integers may be sequenced beginning at zero. The integer index can be shared across different files of data used to identify the attribute values that are the result of a later- performed "attribute mapping" step (explained below).

In the example implementation, the preference attributes 136 can be discrete valued or continuous valued. For discrete (symbolic) valued (e.g., HIGH. MEDIUM or LOW) attributes, the utility for each valid value in the domain of that attribute is converted to a linear single-measure utility function. For continuous valued attributes (e.g., 0 to 256), the utility is defined by the exponential single-measure utility function. In one example implementation, an ASCII file is created to contain the attribute domain information. Each line in the file represents the domain for a single attribute. The format for the lines in the file can be different for discrete valued attributes and for continuous valued attributes. For example, continuous valued attributes may have a format:

Attribute name" < Integer Index >.

(For example, "Size" 28) The format for discrete valued attributes is more complex, for example:

"Attribute name" <Integer Index> <Symbolic Value 1>...<Symbolic Value N>

(For example, "Semantic Density" 24 HIGH MEDIUM LOW) The symbolic values form an "enumerated type" for the attribute and, during computation of the utility for the attribute (see below), are converted to integers. As an example, the "semantic density" value described above would convert "HIGH" to the value one, "MEDIUM" to the value of two, and so on.

Example Mapping of Preference Attributes to Knowledge Object Attributes

Figure 8 shows how preference attributes 136 within preference model 104 can be used to map to attributes of objects using attribute and metadata object mapping in the example implementation. This Figure 8 shows preference attributes 136 (i.e., the "leaf" nodes in the goal hierarchy "tree") in the left-hand column, attribute values in the center column, and object attributes in the right-hand column. The preference attributes 136 in the left-hand column define attribute functions of the preference model 104 which, when they are evaluated by solver engine 200, yield corresponding attribute values in the center column. In this example, solver engine 200 maps attributes of objects (i.e. partial solutions) that will be ranked to these preference attributes. This mapping does not necessarily have a one-to-one cardinality. For example, some object attributes may be mapped to only one preference attribute, while other object attributes may be mapped to more than one preference attribute. In this example, the preference attribute values are generated by applying the corresponding attribute mapping function to the object attribute. This mapping will be described in more detail in connection with solver engine 200.

Assess Utilities of Preference Attributes to Create a Preference Set

Step 122 of Figure 4 assesses the single-measure utility function for each of the various preference attributes to determine the "worth" or "utility" of the preference attribute values. The example implementation uses a conventional multi- attribute utility theory modeling process to develop a preference model 130 from the problem-solving goal hierarchy 103B. The output of this MAUT modeling process is converted into a declarative form for automated use by solver engine 200. In this example, creating a declarative form of the MAUT model centers around the defined preference attributes 136 (i.e., the "leaves" of the goal structure "tree"). This modeling may be implemented using a standard conventional modeling tool such as, for example, Logical Decisions' LDW modeling tool for Windows. See also, for example, standard reference books concerning multi-attribute utility theory such as, for example, Keeney et al, Decisions with Multiple Objectives: Preferences and Value Tradeoffs (Cambridge Univ. Press 1993). This standard MAUT modeling process outputs utility functions and weights representing a preference model 130. Once the preference model 130 is converted into a declarative form, the solver engine 200 can apply it to a data item containing identified values for each of the preference attributes represented in the preference model. While MAUT provides certain advantages, other decision theoretic approaches may also be used to compute utility. Perform Tradeoffs to Determine Weighting Among Preference Attributes and Complete Creation of a Preference Set

Step 124 of Figure 4 performs tradeoffs to determine relative weighting among preference attributes. This involves assigning or deriving an overall weight (λ) value for each preference attribute 136. Figure 6 shows numerical values assigned to the various goals, sub-goals and preference attributes. These numerical values represent relative weights.

For example, among the sub-goals 134(b)(2)-(5) associated with "best knowledge" goal 134(b), the Figure 6 goal hierarchy values the "right level" sub-goal 134(b)(3) most highly with an example weight of 0.521. The

"accuracy" sub-goal 134(b)(5) is valued next most highly, with an example weight of 0.271. The "proper scope" sub- goal 134(b)(2) is valued next most highly, with an example weight of 0.146. The "right form" sub-goal 134(b)(4) is valued next most highly, with an example weight of 0.062. These relative weights are developed as part of the expert elicitation process to reflect and model how expert human problem solvers value these sub- goals relative to one another.

The preference attributes 136 are also weighted relative to one another. With respect to each goal from which they immediately depend, the preference attributes 136 should have weights that sum to 1.0. For example, referring to Figure 6, the "breadth" preference attribute 136(1) might be assigned a weight of 0.250, and the "temporal" coverage preference attribute 136(2) might be assigned a weight of 0.750. These preference attributes 136(1), 136(2) respectively measure coverage (sub- goal 134(b)(2)(l) in terms of overall breadth of the coverage and coverage over relative time periods. In this particular example, temporal coverage is considered three times more important than breadth (in other words, the timeliness of the information is considered to be more important than whether the information is comprehensive in this particular problem solving example). Breadth is assigned a weight of 0.250, and temporal coverage is assigned a value of 0.750. Such weights (along with the particular attributes and particular sub-goals) will depend on the particular problem being solved and the particular expert from which the goal hierarchy is being elicited.

Personalities 108

The developed preference model 104 for a particular problem may represent and embody a particular problem solving approach of a particular problem solving expert in the field, i.e., a "persona." (See figure 7) One feature provided by the present invention is to make such personalities available to non-experts ~ allowing them to receive the benefit of how a particular expert would filter, select and weigh information objects. By judging the utility of relevant knowledge objects in the same way that a world-renowned problem solving expert would value them, the example embodiment gives the user extremely valuable information and insight that should help the user to solve his or her problem more efficiently and effectively.

As one simple illustration, to solve the problem of what stocks to purchase to maximize long-term gain, one might interview an expert stock- picker to determine the goals (and associated preferences) he or she applies to pick stocks. The process 110 of Figure 4 thus can embody expert preferences into a preference model 104 that can be used to compute the relative utility of a collection of data items relevant to stock picking. Such a preference module 104 includes variables (i.e., attributes that can be measured); a concept of value/worth/utility (i.e., the "goodness" assigned to values of variables); and a concept of weight (i.e., the value of one variable compared to others). Furthermore, quantifying utility allows the preferred embodiment solver engine to employ a system of common units among attributes.

Figure 7 shows an example general classification of some different types of personalities that can be used in connection with the Figure 4 model generation process. This example personalities classification can be used to provide general categories or types of personalities to help to define preference model 104. Specific personalities (e.g., a famous and capable military commander, a well-known lawyer, a premier business executive, etc.) can alternatively be used to provide personalities that determine or influence the contents of preference model 104.

Example Solver Engine 200 Structure/Operation Overview of Example Solver Engine Operation

Solver engine 200 provides an efficient and useful tool to assist human problem solvers in their decision-making to solve problems. Figure 9 is a high-level schematic illustration of an example solver engine 200. In this example, solver engine 200 includes a data retrieval mechanism (e.g., search engine) 202 and a preference engine 204. A data repository (i.e. solution well) 300 is also connected to solver engine 200. Data repository 300 may be locally-connected mass storage and/or resources accessible via electronic communications networks, such as the Internet.

Solver engine 200, driving off of a solving query 206, uses data retrieval mechanism (e.g. search engine 202) to search and retrieve information from data repository 206. The preference engine 204 provides a mechanism for assigning a utility value, in other words a score, for the purposes of comparing competing or conflicting knowledge objects. In more detail, preference engine 204 invokes the problem-solving preference set 105 based on a problem solving goal hierarchy framework and decision- theoretic approach (e.g. multi-attribute utility theory, "MAUT") to produce a ranking 205 of the search results provided by data retrieval mechanism (e.g. search engine 202) (i.e. from "best" to "worst"), by valuing preference attributes according to the problem solving preference set invoked. Preference engine 204 is capable of evaluating and ranking alternatives according to decision-maker values and tradeoffs —embodied in preference set and derived from a problem solving system, judgmental modes, goal hierarchy and preference model.

Figure 10 shows an overall flowchart of an example process 210 performed to define and operate the solver engine 200. The Figure 10 process 210 includes the following steps:

• a human problem solver initiates a solving objective, question and decision (block 212);

• the solver engine performs object retrieval (and mapping) from solution well 300 (block 214); • the solver engine invokes preference set(s) and profile(s) (block

216);

• the solver engine performs a dynamic utility computation (block 218);

• the solver engine performs ranking/sorting of partial solutions (block 220);

• the solver engine maps retrieved objects to use types (block 222) and filters the top-ranked partial solutions (block 224); and

• the solver engine displays top-ranked solutions by use type (block 226). Figure 11 is a block diagram of an example architecture for solver engine 200. In this example, a user specifies a problem solving search objective by inputting a search request via an input block 230. Data retrieval mechanism (e.g. search engine 202) searches data repository 300 using conventional object searching techniques, and retrieves N objects and provides them in an object buffer 231. A suitable, conventional data retrieval mechanism (e.g. search engine 202) can be used to search data repository 206 by content across diverse types of objects including, documents, messages, web-based open sources, metadata associated with imagery, etc.

Data integration techniques (e.g. search integration techniques) can be used that make extensive use of metadata and meta-tags and/or use domain taxonomies (ontologies) to allow users to define their preferred terms for search and retrieval. A user-augmentable taxonomy based on feedback, syntactic and semantic analysis based on rules and a repertory grid decision model may be employed.

The data retrieval mechanism (e.g. search engine 202) can be tailored to the needs of the Solution Well 300 by having it search for content metatags. Since no single currently existing search engine provides the requisite functionality across all classes of knowledge objects, it is desirable to consult multiple search engines. Example of currently available candidate search engines that can be used and/or expanded to provide such functionality include Copernic, Inktomi, and/or Open Text's Live Link.

An attribute mapper 233 combines attribute values of each object retrieved by the search engine 202 in object buffer 231 with user defined domain data provided by a solver profile block 232 to compute a new set of attribute values (with attributes based on the preference model 104) for each object. In one example, the mapping is performed based on a problem solving activity profile inputted in the form of a user-defined domain. A utility computation block 234 computes the utility of each of the N objects in object buffer 231 based on the attribute mapper 233 and the MAUT model 130 to provide a ranked list of the N objects sorted or ranked from "best" to "worst" based on their relative utility to solving the particular problem the user is trying to solve. These sorted or ranked N objects are provided via an output (buffer) block 236. These N ranked objects may be further filtered before being displayed or otherwise outputted to the user by a filter/display block 238. For example, the user may request that only the top or "best" "k" objects are presented; block 238 may organize these top k objects into y categories based on use types (i.e. types of partial solutions)

Example Utility Function Dynamic Analysis and Computation to Provide Object Ranking Figure 12 shows an example architecture for utility computation block 234. In this example, utility computation block 234 includes a problem solving preference set utility computer 240; an intermediate buffer 242; and a sorter 244. Utility computer 240 assesses the relative utilities of each of the n objects within the object buffer 231, to dynamically calculate functional values. The resulting values are stored in an attribute data structure that is used by attribute mapper 233.

Figures 13 and 14 show an example resulting attribute entry for both a continuous-valued attribute (size) and a discrete-valued attribute (semantic density). Figure 15 shows an example object attribute data structure 250 the example embodiment uses to represent an attribute and its associated characteristics and to store dynamically calculated attribute values. Attribute data structure 250 in this example includes a mapping function 252, utility functions and weights 254, and a utility value 256. In this example, the utility functions and weights 254 are used to compute the utility value 256 which is then dynamically assigned.

Example Utility Computation

As mentioned above, utility computation block 234 dynamically computes utility of each object within object buffer 231 based on the preference set 105. The overall utility of an object is defined by the following:

U_ar ∑ t u. ( ttr Λ , here

• attrj is the value for the ¹ attribute of object obj, and • ui is the utility function for the 1^th attribute.

As described above, the preference set 105 represents preference attribute utility functions in two ways, depending upon whether the preference attribute has discrete values or continuous values. For discrete values, a linear function is used; for continuous values, an exponential function is used: u_t (x) = a + bx (discrete) u . (x) — a + be ^_cc (continuous)

Values a, b, and c come from the single-measure utility function. The solution system 100 provides this data. To construct the exponential utility function for continuous valued attributes, utility computation block 234 may use the preference set 105 directly and insert the a, b, and c values into the exponential form of the utility function above. Even though the linear function for a discrete attribute is simpler than the exponential function, the case for discrete valued attributes is a bit more complex. In this particular example, the a, b, and c values come from the same part of the model as for the continuous valued attribute case. In order to advance the goal of representing the utility function ui for each attribute in a manner that will make the computation of u_; simple and uniform, the value c can be used to differentiate between discrete valued attributes and continuous valued attributes. Strictly by examining the value for c, block 234 can determine whether or not the attribute value is discrete or continuous. If c = 0, the attribute is discrete; if c ≠ 0, the attribute is continuous. In the case of discrete values, one may define a utility for each discrete value of the attribute, and the symbolic attribute values are then enumerated (i.e., represented by an integral value). Two symbolic values that are next to each other can have successive integral values. For example, if the symbolic values for a particular attribute are HIGH,

MEDIUM and LOW, the enumerated integral values might be 1, 2 and 3. Two successive integral values will be used as the end points of a linear segment to define the utility of the attribute for the given values. The resulting utility function is "stepwise" in the sense that the utility is not necessarily linear over a range of three or more attribute values. Consider the following utility values for our HIGH, MEDIUM and LOW example: HIGH = 0.2 MEDIUM = 0.6 LOW = 0.4 The utility values for this example can be plotted as shown in Figure

16, where the horizontal axis represents the symbolic discrete values (1, 2 and 3), and the vertical axis represents the utility values (0.2, 0.4, and 0.6). This demonstrates the "stepwise" utility function. Notice that three discrete values yield two line segments. There will be n-1 such linear functions given n discrete values for the attribute. The linear function defined by f (x) = a + bx (where a and b are obtained from the attribute data line as described above, recalling that c=0 implies a linear utility function) defines the utility for the attribute with the discrete value x. This implies that the second enumerated value for one data line, when applied to its linear function, will equal the first enumerated value for the next data line when applied to its linear function. This last observation can be seen by looking at the inflection point in Figure 16. By using the linear functions defined by a and b with their associated enumerated values representing the symbolic discrete values, it is possible to compute the utility for each symbolic discrete value.

There is no simple way to represent the n-1 different linear functions required for the discrete valued attributes. It is possible to compute the utility for each discrete valued attribute using the n-1 linear function as described above. Indeed, a "sanity check" can be made by computing the utility for each "interior" discrete value (i.e., inflection points in the "stepwise" linear function as represented in Figure 16). The utility value for the second discrete value applied to one data line should equal the utility value for the first discrete value applied to the next data line. As discussed below, the approach taken (as shown in Figure 16) is to "demultiplex" the resulting values during the mapping process.

Referring back to Figures 2 and 3 A, example goal hierarchy 103B (and thus the preference model 104 and preference set 105) is defined in terms of context matrix 102 in the example embodiment. As discussed in detail above, a human problem solver in one example may operate within only one mode and one phase of context matrix 102 at a time. The solver engine 200 takes this into account by, in one embodiment, "turning off all but one mode and/or phase the human solver is "in" ~ after receiving input from the human solver about their phase and/or mode in the problem solving process. This "turning off function can be provided by introducing high-level weighting factors into the preference set 105. For example, referring to Figure 6, assume that the human problem solver is in the "best knowledge" mode corresponding to goal 134(b). The weight of this goal 134(b) can be set to 1.0, and the weight of each of the other goals on this level of the hierarchy can be set to 0.0. Such settings cause the utility calculations to reflect only those preference attributes 136 within the preference set 105 that are associated with the "best knowledge" mode. When the human problem solver is operating in a different problem solving mode, another one of sub-goals would be given a weight of 1.0. It is possible in certain other implementations that plural sub-goals 134 could have non-zero weights to provide "mixed mode" problem solving. In this example arrangement, the utility computation may be independent of other dimension (e.g., phase) of the context matrix 102. The resulting ranked objects may be sorted and/or presented based on use type relative to the phase. Other alternatives are possible.

Attribute Mapper 233

Figure 17 is a block diagram of an example attribute mapper 233 and its use of meta-data. In this example, the N objects 231(1)...231(n) returned from the search block 202 in object buffer 231 each comprise two parts:

• a data part 231 -d, and

• a meta data part 231 -md. Data part 231-d may, for example, comprise the content associated with the object 231(n) (e.g., text, graphics, audio, video, software, or any other information that may be represented in digital or analog form). Metadata part 231-md preferably comprises one or more metatags used to label or otherwise categorize object 231(n) and/or the data part 231-d. Referring once again to the example attribute mapper 233 shown in

Figure 17, the approach taken with regard to discrete valued attributes is to "demultiplex" the discrete valued attributes to create a corresponding binary vector (see Figure 16). This process may be performed (block 276) by replacing a discrete valued attribute with k pseudo attributes, where k is the number of discrete values that the attribute takes on. For example, if an attribute has three discrete values (HIGH, MEDIUM, and LOW), that attribute is demultiplexed and replaced by three pseudo attributes with binary values. Thus, if the attribute's original value is HIGH, it is replaced by three pseudo binary attributes with values (1, 0, 0). If the attribute's original value is MEDIUM, it is replaced by the three pseudo attributes with values (0, 1, 0). Similarly, if the attribute's value is LOW, that attribute is replaced by three pseudo attributes with values (0, 0, 1). This implies that k functions will be required — one for each of the new pseudo attributes u_i . The new functions maintain the original linear function definition a + bx. Let a = 0 and b = < the utility value > for the k^th discrete value (which is computed as described above). Then: u_ik (x) = u. (attr_ik)

where attr^ is the k discrete value for attri.

With this approach, it is possible to create vectors A, B, C, k and attr, such that the computation of U_0bj is straightforward. Recall that each of these vectors contains the expanded pseudo attributes: U(obj) = ∑. u^attr^λ_; Figure 18 shows a flowchart of example steps that may be used to implement this utility computation. The process assumes that:

• the total number of attributes and pseudo attributes n is known,

• A, B, C and λ values are known (having been obtained as described above), and • all attri are known (including the expanded pseudo attribute values). Referring to Figure 18, the variables "utility" and "counter" are both initialized to zero (block 270) and "counter" is tested to determine whether it exceeds n (i.e., the total number of attributes and pseudo attributes) (block 272). If "counter" exceeds n, this means that all attributes have been processed and the Figure 16 process returns the calculated utility (block 274). Otherwise, the value of C [counter] (the counter"¹ element in vector C) is checked to determine whether it is equal to zero (block 276). As mentioned above, if c = 0, then the attribute is discrete and the linear form of the utility function calculation is used ("yes" exit to decision block 278).

If c ≠ 0, the attribute is continuous and the exponential form of the calculation is used ("no" exit to decision block 280). For discrete values, utility can be calculated as:

Utility = Utility + B[counter] * attr[counter]

(block 278). Using the exponential form for continuous values yields the following utility calculation (block 280): Utility = Utility + A[counter] +

B[counter] * e-^c*^attr[counter In either case, the value of counter is incremented (block 282) and control returns to decision block 272.

Referring again to Figure 17, attribute value calculation (block 270) calculates a value based on an attribute mapping function and a user- inputted domain (problem solving activity profile) 232. Profile 232 describes particulars about the current problem solving task (e.g., "I need to install oil pipelines in Turkey.") This information can be elicited from the user through a declaration and/or questionnaire, and/or it may be inferred (e.g., based on the user's current and recent activities). Attribute value calculation 270 in this example provides a discrete (e.g., symbolic) value 272 or a continuous value 274 to accommodate the two types of preference attributes. Continuous value 274 represents a continuous range of values that can be used in a straightforward manner in a multi-attribute utility theory utility computation. However, not all attributes provide continuous values. Other attributes may have more complex but nevertheless discrete output values. In this example, a demultiplexer (transformer) 276 expands the discrete value into a binary vector which may then be used as attribute value. See Figure 16.

Example Filter and Display of Knowledge Objects

Figure 19 shows the filter and display block 238. In this example, filter and display block 238 operates by identifying use types for each object in the collection of n sorted objects outputted by utility computation block 234. Filter and display block 238 then determines a set of y unique use types (block 292) by taking the intersection of all the use types identified in block 290, and filters k of the n objects into the y use types (block 294). In this example, the value k may be specified by the user (e.g., "display your k top hits" where k may be any positive integer value such as 20, 30, 50, etc.). Finally, block 296 displays the k objects in y use types on a display.

The use types may be defined for each object based on the metatag information discussed above or some mapping of metatag information and/or content.

Example Solution Well Operation

Figure 20 is a block diagram showing an example architecture for solution well 300. In this example, solution well 300 provides a completely flexible means of storing and organizing any piece of digital information around a problem solving effort. In the example embodiment, solution well 300 includes three major components:

• data schema;

• solutions well data structure; and • portfolio system. Example Data Schema

Example data schema involve identification of use types and construction of metatag extensions to base schema model and organize as schema; and identification of data sets and physically tagging data accordingly. Figures 21A-21D show an example list of base schema metatags that can be used to associate attributes of any object with attributes in the preference model. Indentation is used to indicate nesting within the tags. The third column lists the possible applicability of the tag to one or more of the attributes of the example preference model. These metatags can be pre-applied to the content by the content providers and/or content can be processed (e.g., by a content management system 302) to apply appropriate metatags to the content. There are many available metadata schemas in common use, including for example:

• the Learning Object Meta Data document created by the IEEE Learning Technology Standards Committee; (including the

Dublin Core Meta Data Base Schema)

• the Meta Data Interchange Format standard defined by the Meta Data Coalition (MDC) (see e.g., www.mdcinfo.com/index.html);

• the Text Encoding Initiative (TEI) sponsored jointly by the Association for Computers and the Humanities, the Association for Computational Linguistics, and the Association for Literary and Linguistic Computing (see e.g.,www.uic.edu/orgs/tei/). Our solution oriented metadata schema provides a way to identify and extend any standard base schema and thus is highly adaptable to any existing meta-tagging schema. Example Solutions Well Data Structure/Knowledge Repository 316

The role of the knowledge repository 316 is to organize knowledge and classify it into use types. Repository 316 provides an organization scheme for knowledge objects by problem class, mode and use type. As shown in Figure 20, content can come from a number of different sources including for example:

• exclusive content providers 304;

• corporate and educational sponsors 306;

• public internet 308; • contributors 310;

• content managers 312;

• application developers 314; and

• other content providers.

This various content forms what is shown in Figure 20 as content base (knowledge repository 316). The example solution well repository 316 (actual or virtual) of data from heterogeneous sources including for example documents, reports, web-links to reliable sources, imagery, technical drawings, and discussion group messages, to name a few. It provides a storehouse of knowledge in the form of heuristics, rule bases for expert systems, video clips from experts, best-practice case studies, etc.

A meta-data index provides both syntactic and semantic links to data and knowledge resources. A rich knowledge meta-model could be used to handle the diverse types of information.

Figures 22A-22F show example use types of partial solutions associated with the various "modes" of the problem solving matrix 102. These non-exhaustive, exemplary use types are provided as illustrative examples to show different use types (as well as example objects corresponding to those use types) corresponding to each of the "modes" of problem solving matrix 102.

Example Portfolio System

An example portfolio system organizes and packages knowledge objects. It allows a library to be built of problems-solved, sets and sequences of solving queries. Associations between data usage and problem solving preferences to be utilized in autogenerating problem solving knowledge. The portfolio functionality may be built from a contributor publishing system 316 comprising a template system 318 and a portfolio creation system (not shown). These components can be used by application/content developers to prepare specialized content, index, organized and packaged into portfolios.

Design Templates generated by design template system 318 provide the structure and operations for content creation, editing, content tagging, linking, integration, publication, and dissemination. Portfolios provide a metaphor for aggregating, organizing and indexing diverse data types, formats, and content. They can also themselves act as knowledge objects. Contributors use these tools to provide domain-specific content for course modules, for specialized Knowledge Forums, etc. Multimedia such as streaming audio/video (see block 320) can play an important role in conveying knowledge such as best practices, case studies and interviews with certified experts.

Example Solver Computing System (i.e. Solver Portal)

Figure 23 shows an example Solver Computing System 50 implemented in an existing generic computing system (e.g. internet/www architecture). In this example, Solver Computing System 50 may be integrated into a channel of a computing system (e.g. portal infrastructure 600) using a source of prequalified content 500. Input device with user interface 602 provides a graphical or other user interface (e.g. voice) allowing a user to communicate with solver engine 200 and/or portal infrastructure 600 via an optional gateway 605, for example communications can be provided via the Internet, an Intranet, a local area network, or some other digital communications network or other means 604 using a mark-up language such as HTML or XML, and a protocol such as http for example. Portal infrastructure 600 may have appropriate web server(s) 606 and associated data storage 608. A user authenticator (not shown) may be used to authenticate the user to ensure the user is authorized to access solver engine 200 and/or portal infrastructure 600, and may grant different access rights to different users. Portal infrastructure 600 provides a variety of capabilities and support services useful to a busy decisionmaker, for example: • collaboration (e.g., chat, email, groupware, etc)

• help information

• publishing information

• human resource locator (e.g., people locator, expert locator, phone book) • e-commerce services (e.g., order management, shipping, pricing, payment, taxation, suggestion, third party distribution management, and transaction processing)

• document management

• news groups • advertising

• certificate authority '

• streaming audio/video support

• directory services • online learning (e.g., course management, distribution, author ware, course content)

• account management and billing

• portal activity reporting Portal infrastructure 600 can integrate these various capabilities, services and resources with the services Solver Computing System 50 provides.

Figure 24A is a more detailed schematic illustration of an example Solver Computing System 50 integrated into a generic computing system such as e.g. Portal infrastructure 600. This example implementation shows the Solver Computing System 50 residing on a web server 606 that draws data from a data repository 608. The Solver Computing System 50 is available to a problem solver via a computing network (e.g. the Internet 604), an access point to that network (e.g. gateway 605) and an input device 602.

In more detail, Figure 24A shows an overall example architecture for portal infrastructure 600. In this example, a solution system 100 (i.e. solver 672), solution well system 300 along with a search engine 202 and a preference engine 204 as described above are included within an overall portal architecture 600 including additional components, such as for example:

• e-commerce services 650,

• server services 652,

• online learning system 654, • database services 656,

• operating system 658,

• portal management system 660, and

• firewall 662. The bottom portion of Figure 24A may be implemented, for example, on a conventional web server having an appropriate back end. The top portion of Figure 24A may be implemented within a conventional web browser providing a user interface. The server may communicate with the browser over a decentralized or centralized computer network such as for example the Internet, a corporate intranet, a local area network, a wide area network, or other digital communications network. In one example embodiment, the server communicates with a web browser client over the Internet using standard http or other protocols and by communicating web pages in a markup language such as html, XML, etc. Conventional scripting such as Java Script and/or applets such as Java Applets can be used to provide functionality at the client side. Conventional security protocols such as SSL may be used to provide secure communications. The e-commerce services 650 provided by portal architecture 600 may provide the following example module/functionality:

• order management module 650a,

• payment module 650b,

• shipping module 650c,

• tax module 650d, • pricing module 650e,

• suggestion module 650f,

• third party distribution management system 650g,

• transaction processing engine 650h.

E-commerce services 650 may be used in a conventional fashion to buy and sell "products" generated by solution well 300. Such products may include, for example, a collection of knowledge objects representing a partial solution to a problem and/or one or more preference sets or models developed described above. E-commerce services 650 may be used to buy or sell such items in real time over the Internet or other computer network by, for example, soliciting credit card information, verifying credit card accounts, and performing other financial transaction functions as is typical for current web-based or other financial electronic transactions. E- commerce services 650 may use a third part distribution management system and/or interface 650g to deliver the requested information, and/or it may download information to the user's computer on demand after payment is received.

Server services 652 may include a number of different example servers such as:

IM server 652a, my toolbar 652b, document management systems 652c, personalization manager 652d, • community system 652e, directory services server 652f, chat server 652g, collaboration server 652h, streaming audio/video server 652i, • certificate authority server 652j, advertising server 652k, newsgroup server 6521, portfolio system server 652m; and other server(s). Online learning system 654 may include a course management module 654a, a distribution box 654b, an authorware module 654c, and a course content module 654d. Online learning system 654 may perform online learning and/or course functionality in a conventional manner - with the courses and course content selected based on the operation of preference engine 204 described above.

Portal management system 660 may perform conventional functionality such as portal activity reporting 660a, billing 660b, and/or account management 660c. Database services 656 and operating system 658 may comprise conventional software modules in programs as is well known.

The top portion of the Figure 24A diagram shows some example features of a user interface/functionality presented to an end user. Such example functionality includes for example:

• a directory system/authentication service 664,

• user interface 666,

• portfolio system 668,

• catalog system 670, • solver interface 672,

• publishing information system 674,

• help system 676,

• collaboration system 678.

The user interface 666 may include the following components for example:

• online learning interface 666a,

• advertising presentation interface 666b,

• state manager interface 666c,

• graphical user interface builder 666d, • personalization agents interface 666e.

Portfolio system 668 may include the following components for example:

• presentation 668a, • editing 668b,

• off-line services 668c,

• on-line services 668d,

• applications manager 668e. Catalog system 670 may include the following components for example:

• content listing 670a,

• browse 670b,

• purchase 670c. The Solution System (i.e. solver interface) 672 may include the following example components for example:

• problem definition 672a,

• presentation 672b,

• personal definition 672c, • solve 672d, and

• advanced solve 672e.

The publishing system 674 may include the following components for example:

• drag and drop publish 674a, • search 674b, and

• browse 674c.

Help system 676 may include the following example components:

• site navigator 676a,

• definition 676b, • call center 676c,

• call center tracking 676d.

Collaboration system 678 may include the following components for example: • newsgroups 678a,

• instant messaging 678b,

• workflow 678c,

• VTC 678d, • chat 678e,

• application collaboration 678f.

The directory system/authentication service 664 may include the following example components:

• account setup and management 664a, • authentication 664b,

• people locator 664c,

• expert locator 664d,

• phone book 664e.

Example User Interface Figures 1C-1G which were briefly described above, show example web-based browser user interface displays provided by an example portal infrastructure 600 as described above. Figure IC shows an overall example interface 690 including a toolbar 692, various functional selection blocks 694, and a "search and solve" input block 696. A toolbar 692 (see also Figure IE) may include various functions as described above including:

• e-mail,

• calendar,

• portfolio,

• workflow, • people,

• communities,

• chat, • document sharing,

• video conferencing,

• acquisitions.

The functional blocks 694 present the user with various lists (which may be based upon prior work the user has done using portal infrastructure 600). For example, such functional blocks may include:

• a communities block 694a specifying groups of people relevant to problem solving or aspects thereof,

• a "knowledge" functional block 694b listing relevant knowledge objects or classifications of knowledge objects,

• a "problems" functional block 694c providing a listing of problem or categories of problems, and

• a "links to learning sources" functional block 694d providing

^• hypertext or other links to sources of learning information (e.g., universities, seminars, etc.).

The "search and solve" element of interface 690 allows the user to input a search criteria into an input field 696a to perform a knowledge search and then apply problem solving criteria to filter or rank the search results using the solver engine 200 and other elements as described above. Figure ID shows an example further user interface provided in connection with the "search and solve" functionality shown in Figure IC. This example "solver controls" interface or dialog box 700 provides sort of a "dashboard" to solver engine 200. Interface 700 may, for example, include a "mode" display 702 and a "phase" display 704 indicating the mode and phase of problem solving within the context matrix 102 as described above. A "solver personalities" input block 706 allows the end user to select from a number of different solver personalities (e.g., software developer, military strategist, stock picker, lawyer, etc.) and/or to create and edit a new personality. A solver personality filter section 708 allows the user to select what type of filter is to be applied to the end results based upon any one or combination of solver personality filter criteria and/or specialization by content. A "preference model" control section 710 allows the user to select a preference model to use, edit, cut and/or paste.

Figure IE shows an example display 720 of the results of a problem solving search launched using the "search and solve" input block shown in Figure IC. In this example display, the load and phase "dashboard" indication 702, 704 are displayed at the top of the screen along with the "search and solve" function 696. Screen 720 may also include a "display mode" selector 722 that allows the user to select between different modes of context matrix 102 for displaying objects (see also Figure IF). In this example, the user may select all modes, any one of the modes, or any combination of the modes. The remainder of screen 720 provides display of results by use type in connection with the particular mode or modes that have been selected. Such display may categorize different retrieved, ranked and filtered knowledge objects by different use types (e.g., briefing, measures, articles, sources, models, concepts, etc.). Within each use type categorization, the various objects are display in ranked order. Display 720 may include an ability of the user to select objects (e.g., by clicking on hypertext links) in order to call up the objects for further display, perusal, etc. In some cases, if the object costs money, the user may be presented with a selection field 724 and a price indication. The user marks the selection field, the user's account may be charged and the object selected may be delivered to the user either electronically, through standard delivery mechanisms, or by making a reservation for the user (e.g., in the case of a conference, a seminar, or class).

While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be linked to the disclosed embodiment. For example, although the current instantiation of the invention is using a web-based browser, portal and a desktop computer, future instantiations could include, for example, voice-activated interfaces, operation via personal digital assistants or via wireless networks, with a variety of means of presenting and processing the data and decision making rules. The invention is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims.

Claims

WHAT IS CLAIMED IS: 1. A data processing system for assisting a user engaged in a problem solving activity, comprising: a computer-based modeler that develops a computer-based preference set reflecting a problem-solving progression; an object handler that accesses a set of information objects; and a solver engine operatively coupled to at least the object handler, the solver engine using the preference set to compute utilities of objects within the object set, the solver engine filtering and/or sorting the object set based at least in part on the computed utilities and presenting the filtered and/or sorted objects to the user.

2. The data processing system of claim 1 further including: a data server hosting the solver engine (e.g. a web-based server), the data server being coupled to a decentralized computer network, and an interface (e.g. browser) coupled to data servers via the decentralized computer network, wherein the solver engine presents the filtered and/or sorted object set to the user by transmitting messages to the browser via the decentralized computer network.

3. The data processing system of claim 2 wherein the solver engine includes a network interface that uses a distributing computing protocol (e.g. http) to communicate the filtered and/or sorted objects in a form usable by a person (e.g. web pages).

4. The data processing system of claim 1 wherein the solver engine presents the filtered and/or sorted object to the user in such as way as to directly assist the user in achieving a changed goal state.

5. The data processing system of claim 1 wherein the solver engine further includes a characterizer that characterizes the user relative to the problem-solving progression.

6. The data processing system of claim 5 wherein the characterizer comprises eliciting a problem solving mode and/or phase from the user, and the solver engine computes object utilities based on the user's problem solving mode and/or phase.

7. The data processing system of claim 6 wherein the problem solving progression comprises plural problem solving modes and plural problem solving phases, and the characterizer includes a user interface that requests the user to identify one of the plural problem solving phases and/or problem solving modes.

8. The data processing system of claim 1 wherein the solver engine presents a partial solution to the user's problem solving activity.

9. The data processing system of claim 1 wherein the computer modeler applies a decision theoretic process.

10. The data processing system of claim 9 wherein the decision theoretic approach comprises multi-attribute utility theory.

11. The data processing system of claim 1 wherein the problem-solving progression comprises a multi-dimensional problem solving space defining plural problem solving modes and plural problem solving phases within each of the modes.

12. The data processing system of claim 1 wherein the problem-solving progression comprises a problem-solving context matrix defining plural problem solving modes along a first dimension and plural problem solving phases along a second dimension different from the first dimension.

13. The data processing system of claim 1 wherein the computer- based modeler elicits problem solving preference information from at least one person, and uses the elicited preference information to develop at least one preference set from the preference model, the solver engine being responsive to the preference set.

14. The data processing system of claim 13 wherein the person consists of the user.

15. The data processing system claim 13 wherein the person is different from the user.

16. The data processing system of claim 13 wherein the person is an expert problem solver (e.g., consultants).

17. The data processing system of claim 1 wherein the solver engine maps object attributes to the preference set

18. The data processing system of claim 17 wherein the object attributes are encoded within object metatags associated with the objects, and the solver engine maps the object metatags to the preference set.

19. The data processing system of claim 1 wherein the solver engine ranks the objects based on computed utility.

20. The data processing system of claim 1 wherein the solver engine presents a ranked subset of the objects in an interactive digital form usable by a person (e.g. web page).

21. The data processing system of claim 1 wherein the solver engine presents the objects to the user based on use type.

22. The data processing system of claim 1 wherein the computer- based modeler produces a static stored data structure, and the solver engine operates dynamically based on the static stored data structure and user inputs.

23. The data processing system of claim 1 wherein the computer based modeler develops at least some continuous utility functions, and the solver engine dynamically evaluates the continuous utility functions for particular objects within the object set.

24. The data processing system of claim 1 wherein the object handler includes a data retrieval mechanism (e.g. search engine) coupled to at least one object repository, the data retrieval mechanism searching an object repository to provide search results, the solver engine operating to select a subset of the search results based at least in part on the search results.

25. The data processing system of claim 1 wherein the object handler retrieves at least some of the objects over a communications system (e.g. the Internet).

26. The data processing system of claim 1 wherein the solver engine maps objects to the preference set based on metatags associated with the objects.

27. The data processing system of claim 1 wherein the solver engine presents the filtered and/or sorted objects on a web portal.

28. The data processing system of claim 1 wherein the computer- based modeler develops plural specialized preference sets corresponding to different personalities, and the solver engine employs a preference set.

29. The data processing system of claim 28 further including a transaction engine that electronically charges the user a fee for a partial solution and/or access to a preference set based on the preference model.

30. A computer-assisted method for assisting a user engaged in a problem solving activity, comprising: (1) developing a computer-based preference model of a problem-solving goal hierarchy reflecting a problem-solving progression; (2) accessing a set of objects; (3) using the preference model to generate an individual preference set (4) using the preference set to compute utilities of objects within the object set; (5) filtering and/or sorting the object set based at least in part on the computed utilities; and (6) presenting the filtered and/or sorted objects to the user.

31. The computer-assisted method of claim 30 wherein the presenting step includes presenting the filtered and/or sorted object to the user in such as way as to directly assist the user in achieving a changed goal state.

32. The computer-assisted method of claim 30 further including characterizing the user relative to the problem-solving progression, and performing at least one of steps (2)-(5) based at least in part on results of the characterizing step.

33. The computer-assisted method of claim 32 wherein the characterizing step comprises eliciting a problem solving mode and/or phase from the user, and step (3) comprises computing object utilities based on the user's problem solving mode.

34. The computer-assisted method of claim 30 wherein the problem solving progression comprises plural problem solving modes and plural problem solving phases, and the characterizing step includes requesting the user to identify one of the plural problem solving phases and/or problem solving modes.

35. The computer-assisted method of claim 30 wherein step (5) presents a partial solution to the user's problem solving activity.

36. The computer-assisted method of claim 30 wherein the developing step includes applying a decision theoretic process.

37. The computer-assisted method of claim 36 wherein the decision theoretic approach includes applying multi- attribute utility theory to develop a preference set computer model.

38. The computer-assisted method of claim 30 wherein the problem-solving progression comprises a multi-dimensional problem solving space defining plural problem solving modes and plural problem solving phases within each of the modes.

39. The computer-assisted method of claim 30 wherein the problem-solving progression comprises a problem-solving context matrix defining plural problem solving modes along a first dimension and plural problem solving phases along a second dimension different from the first dimension.

40. The computer-assisted method of claim 30 wherein the developing step includes eliciting problem solving preference information from at least one person, and using the elicited preference information to develop the model.

41. The computer assisted method of claim 40 wherein the person consists of the user.

42. The computer assisted method of claim 40 wherein the person is different from the user.

43. The computer assisted method of claim 40 wherein the person is an expert problem solver (e.g. consultants).

44. The computer-assisted method of claim 30 wherein the using step includes mapping object attributes to the model.

45. The computer-assisted method of claim 44 wherein the object attributes are encoded within object metatags associated with the objects, and the mapping step comprises mapping the object metatags to the model.

46. The computer-assisted method of claim 30 wherein the filtering and/or solving step includes ranking the objects.

47. The computer-assisted method of claim 30 further including presenting a ranked subset of the objects on an interactive web page.

48. The computer-assisted method of claim 30 wherein the method further includes associating at least one use type with each of the objects, and presenting the objects to the user based on use type.

49. The computer-assisted method of claim 30 wherein the developing step produces a static stored data structure, and the using step is performed dynamically based on the static stored data structure.

50. The computer-assisted method of claim 30 wherein the developing step includes developing at least some continuous utility functions, and the evaluating step includes dynamically evaluating the continuous utility functions for particular objects within the object set.

51. The computer-assisted method of claim 30 wherein the accessing step includes searching a database to provide search results, and the filtering and/or sorting step comprises selecting a subset of the search results.

52. The computer-assisted method of claim 30 wherein the accessing step includes retrieving at least some of the objects over the Internet.

53. The computer-assisted method of claim 30 wherein the using step includes mapping objects to the model based on metatags associated with the objects.

54. The computer-assisted method of claim 30 wherein the presenting step includes presenting the filtered and/or sorted objects via a digital user interface (e.g. web portal).

55. The computer-assisted method of claim 30 wherein the developing step (3) includes developing plural specialized preference sets corresponding to different personalities, and the method further includes selecting one of the plural preference sets for use by the using step (3).

56. The computer-assisted method of claim 55 further including providing the selected preference set to the user for a fee.

57. The computer-assisted method of claim 30 wherein the presenting step (5) includes charging the user a fee for a partial solution.

58. In a computer system that filters objects to help a user solve problems, a storage arrangement storing a data structure representing a problem solving goal hierarchy corresponding based on a time-independent sequence of problem solving modes the goal hierarchy including plural terminal nodes each defining decision theoretic parameters and additional information for, in use, correlating the node with a metatag associated with at least one object to be filtered/ranked.

59. The storage arrangement as in claim 58 wherein the decision theoretic parameters comprise multi-attribute utility theory parameters.

60. The storage arrangement of claim 58 wherein the multi-dimensional problem solving space is defined by a context matrix having first and second dimensions, the first dimension defining problem solving modes, the second dimension defining problem solving phases.

61. A computing system (e.g. web portal coupled to the Internet), comprising: a storage arrangement storing at least one preference model representing a problem solving goal hierarchy that models human problem solving and/or one preference set representing human problem solving preferences; a search engine arranged to retrieve a set of objects; a utility computation engine that computes the utility of retrieved objects based on the selected preference set; a ranker that ranks the retrieved objects based on computed utility; and a user interface that presents the ranked objects to at least one user via the Internet.

62. The web portal of claim 61 wherein the search engine retrieves objects having identifying data associated therewith and the engine includes an object mapper that maps the retrieved objects to a preference set based at least in part on the identifying data.

63. The web portal of claim 61 wherein the user interface transmits the ranked objects encapsulated in digital messages (e.g. http protocol messages).

64. The web portal of claim 61 further including a filter coupled to the ranker, the filter categorizing the retrieved object based on use type.

65. A computer-assisted method for helping at least one user solve a problem, comprising: (a) distilling at least one person's problem solving decision preferences into an operational model (b) characterizing the user's problem solving activity as a progression of plural problem solving modes ; (c) selecting at least one mode within the mode progression in response to input signals and/or stored information; (d) invoking a preference set that embodies human problem preferences (e.g. a subject matter expert) (e) performing a comparison between attributes of an object set and the preference set, based at least in part on the selected mode; and (f) selecting and presenting a subset of the object set to the user based at least in part on the comparison.

66. The method of claim 65 wherein the selecting and presenting step includes arranging the subset of objects on a display based on use type.

67. A computer-assisted method for helping at least one user solve a problem, comprising: (a) developing a preference set; (b) characterizing the user's problem solving activity as a progression of plural problem solving modes; (c) selecting at least one mode within the mode progression in response to input signals and/or stored information; (d) selecting a subset of objects from an object repository based at least in part on the selected mode and the preference set; and (e) delivering the subset of objects to the user.

68. The method of claim 67 further including delivering the preference set to the user.

69. The method of claim 67 wherein the object subset or the preference set is delivered to the user for compensation.

70. The method of claim 67 wherein the object selecting step (d) includes the step of mapping data schema associated with the objects to preference attributes within a preference set.

71. The method of claim 70 wherein the data schema comprises metatags associated with the objects.

72. In a data processing system including a preference engine (i.e. solver engine) that develops object utility values based at least in part on a progression of problem solving modes and/or phases, a solution well comprising: an object repository providing a set of prequalified objects each having associated data schemata (i.e. use types) facilitating mapping of the objects to the progression of problem solving modes and/or phases; a content management system that manages the objects within the object repository and presents objects to the preference engine; and a portfolio that organizes and packages objects based on content.

73. The solution well of claim 72 wherein at least some of the objects comprise streaming objects.

74. The solution well of claim 72 wherein at least some of the data schemata comprise metatags.

75. The solution well of claim 72 wherein the portfolio packages objects based on problem and/or problem-solving mode and or problem solving phase.

1 76. The solution well of claim 72 wherein the portfolio is tagged

2 and electronically organizes knowledge objects by problem-solving mode,

3 solution type problem and/or problem-solving phase.

1 77. The solution well of claim 72 wherein the portfolio includes a

2 template system and a portfolio creation system.

1 78. The solution well of claim 72 wherein the portfolio aggregates,

2 organizes and indexes diverse data types to provide domain-specific

3 content.

1 79. A method of buying or selling partial solutions comprising:

2 (a) using a preference set based at least in part on a problem solving

3 progression to organize and rank a set of knowledge objects that may be

4 used to directly assist a user in solving a particular problem; and

5 (b) performing an electronic, computer-based transaction that

6 exchanges the knowledge object set for value.

1 80. A method of buying or selling preference sets comprising:

2 applying domain specific parameters to a generalized preference

3 model based on a problem solving progression to provide at least one

4 preference set; and

5 performing an electronic, computer-based transaction that exchanges

6 the preference set for value.

1 81. A method of creating specialized preference sets according to

2 different personalities comprising:

3 (a) developing a generalized preference model based on a

4 progression of problem solving modes and/or phases;

5 (b) eliciting judgment information from a human being; and 6_. (c) applying the elicited judgment information as weighting

7 information to the generalized preference model to produce at least one

8 preference set.

82. The method of claim 81 wherein the human being comprises an expert problem solver, and the elicited judgment information provides expert problem solving values.

83. A data processing system for assisting a human problem solver engaged in a problem solving activity, comprising: a computer-based modeler that develops a computer-based preference model reflecting a problem-solving progression represented by a multidimensional correlation between plural problem solving modes and plural problem solving phases; a context determiner that determines a context of the human problem solver within said multidimensional correlation; and a solver engine operatively coupled to at least the context determiner, the solver engine using the preference set and the determined context to present partial solutions to the human problem solver.

84. A computer-based method for assisting a human problem solver comprising: developing a computer-based preference model reflecting a problem-solving progression represented by a multidimensional correlation between plural problem solving modes and plural problem solving phases; determining a context of the human problem solver within the multidimensional correlation; and using the preference model and the determined context to present partial solutions to the human problem solver.

85. A storage device that stores a computer data structure including a preference set representing a weighted problem solving goal hierarchy.

86. The storage device of claim 85 wherein the preference set includes utility functions and weights defined at least in part in accordance with value judgments.

87. The storage device of claim 86 wherein the value judgments are elicited from an expert problem solver.

88. The storage device of claim 86 wherein the value judgments are elicited from an end user.

89. A storage device that stores a partial solution delivered at least in part based on a computer-based preference model reflecting a problem-solving progression represented by a multidimensional correlation between plural problem solving modes and plural problem solving phases.

90. The storage device of claim 89 wherein the partial solution comprises a knowledge object.

91. The storage device of claim 89 wherein the partial solution comprises a multimedia object.

92. The storage device of claim 89 wherein the partial solution comprises a video presentation.

93. The storage device of claim 89 wherein the partial solution comprises an interactive course.

94. The storage device of claim 89 wherein the partial solution is supplied to an end user on a fee basis.

95. A digital storage medium storing a data structure comprising an object including a metatag, the metatag defining at least one parameter facilitating mapping of the object to a preference attribute value defined based at least in part on a problem solving goal hierarchy, and further facilitating a utility computation based at least in part on a multidimensional correlation between modes and phases of a problem solving progression.

96. A method of packaging partial solutions comprising: generating, for a partial solution, a metatag that defines at least one parameter facilitating mapping to a preference attribute value defined based at least in part on a problem solving goal hierarchy, and further facilitating a utility computation based at least in part on a multidimensional correlation between modes and phases of a problem solving progression; and associating the metatag with the partial solution.

97. A method of profiling human problem solvers comprising: eliciting at least some information describing a human problem solver; and using the elicited information to a problem solving context describing the position of the human problem solver within a problem solving progression.

98. A computer-based method for interacting with a user comprising: presenting at least one computer-based array of information to the user, including a problem solving context interface indicating the position of the user within a problem solving progression; and using the indicated position to select partial solutions for presentation to the user.

99. The method of claim 98 wherein the display includes a mode indicator and a phase indicator.

100. The method of claim 98 further including the step of allowing the user to select a position within the problem solving progression by interacting with the display.

101. The method of claim 98 wherein the display includes a dashboard providing a mode display portion and a phase display portion.

102. A method of communicating with an end user's computer comprising: transmitting, to the end user's computer at least in part in a markup language, at least one page including a problem solving context representing a position within a multidimensional problem solving progression; and receiving at least one message from the end user's computer indicating that the user has interacted with the page.

103. The method of claim 102 wherein the receiving step includes the step of receiving indication that the end user has clicked on a portion of the page indicating a particular position within the multidimensional problem solving progression.

104. A computer-based method of presenting information to a user comprising: using a computer-based preference model reflecting a multidimensional correlation between plural problem solving modes and plural problem solving phases to select partial solutions to the human problem solver; and presenting the selected partial solutions to the user based on use type.

105. A machine learning method comprising: a computer-based problem solving system used by a person incorporating at least preference models reflecting a multidimensional correlation between plural problem solving modes and plural problem solving phases; preference sets allowing a first user to operationalize problem solving judgments and preferences; a preference engine that manipulates knowledge objects and presents them to a person; a solution well that organizes knowledge objects according to use type; a solution toolkit allowing the first user to interact with the system and a second or multiple users; gathering and storing information about the users interaction with the system, with components of the system, with other users and with the knowledge objects; and using that information to feedback and improve the efficiency and effectiveness of the computer-based problem solving system.

106. A method comprising: developing a preference set by applying judgment values to a problem solving preference model; using the preference set to compute utility of a set of partial solutions; developing a subset of set of partial solutions; and offering at least one of the preference model, the preference set and the subset of partial solutions for sale.

107. The method of claim 106 wherein the offering stpe includes using electronic commerce techniques.

108. A method comprising: associating use types with partial solutions; and employing the use types to compute utilities of the partial solutions based at least in part on a problem solving life cycle progression.

109. The method of claim 108 wherein the associating step comprises defining the use types in metatags associated with the partial solutions.

110. The method of claim 108 wherein at least some of the partial solutions comprise knowledge objects.

111. A computer-assisted method comprising: requesting and supplying partial solutions responsive to a query; and computing utility of the supplied partial solutions based at least in part on a problem solving life cycle progression.

112. A method of interfacing a solver engine with a data retrieval mechanism comprising: commanding the data retrieval mechanism to access partial solutions within a data repository; returning partial solutions to the solver engine in response to the command; and computing utility and generating a solutions array in response to the returned partial solutions.

113. A solution system comprising: a first input that receives a set of value judgments; a second input that receives a preference model; and a computation mechanism responsive to the first and second inputs, the computation mechanism generating a preference set based on the value judgments and the preference model.

114. A data representation of observable attributes of partial solutions, the data representation corresponding to terminal elements of a problem solving goal hierarchy, the data representation providing a topology of use types by problem solving modes within a problem solving progression.

115. A data representation as in claim 114 wherein the data representation includes metatags indicating attributes of partial solutions allowing a solver engine to rank the partial solutions according to the values of the metatags.

116. A portfolio system for integrating knowledgeware with software to link knowledge objects in a problem solving context, the portfolio system including: a user interface; and a mechanism for aggregating, organizing, and indexing diverse data types, formats and content to provide domain specific partial solutions for particular problem solving domains.