WO2009049353A1 - An apparatus for education and assessment rather than and apparatus for education and assessment - Google Patents

Abstract

The present invention relates generally to the field of education incorporating training and learning. In particular, the present invention relates to the testing of knowledge and skills. In one form, the present invention is suitable for use in online validation and assessment of training, in one embodiment disclosed herein there is provided an apparatus adapted to and / or method of educating a user, the comprising the steps of providing access for the user to a virtual training room comprising at least one page of a Graphical User interface wherein the virtual training room is operatively associated with an information repository; enabling the user to select at least one module of a field of knowledge from the information repository to be validated against; and in response to the user's selection generating a quiz for the user wherein the quiz comprises one or any combination of questions, answers and instructional content from the information repository; and presenting the quiz to the user as a sequence of dynamically created pages of a Graphical User Interface.

Description

THAN AND APPARATUS FOR EDUCATION AND ASSESSMENT

FIELD OF INVENTION

The present invention relates generally to the field of education incorporating training and learning. In particular, the present invention relates to the testing of knowledge and skills. In one form, the present invention is suitable for use in online validation and assessment of training and it will be convenient to hereinafter, describe the invention in relation to that use; however it should be appreciated that the present invention is not limited to that use, only. BACKGROUND ART

Throughout this specification the use of the word "inventor" in singular form may be taken as reference to one (singular) inventor or more than one (plural) inventor of the present invention. The discussion throughout this specification comes about due to the realisation of the inventor and/or the identification of certain related art problems by the inventor.

In general, the provision of education whether that be in the form of specific training for persons in need of particular skills or learning in the sense of gaining knowledge and/or skills generally may be problematic when attempting to do so for persons onsite, for example, in the home or at a schooling or work site that may or may not be remote from centres of education.

Once any form of education is provided it may also be necessary to undertake assessment of the learning or training that has purportedly taken place. By way of example oniy, with respect to the assessment or validation of work and industry training, the inventor has recognised the following in terms of related art, which may also be common to more structured schooling or educational institution environments.

Validation methodologies may be static in their nature and as such easily circumvented. Accordingly, these methodologies may require rigorous scrutiny and physical security to mitigate circumvention. Results of tests, examinations and/or assessments have been historically manually marked and collated and, with this comes a significant elapse of time before the person being tested becomes aware of their test results and/or feedback from their test. Moreover, at the completion of a question a user, hereinafter defined as a candidate, student or other person(s) requiring the benefit of an education facility or product, may benefit from a reinforcement of the relevancy of their answer. This is often not available at least not without an appreciable time delay which may diminish the effectiveness of the reinforcement of relevance in the first place. A lower level of literacy may often prevent users from participating themselves in a validation or assessment of their training and/or education. Validation and assessment techniques may be difficult to adapt for remote locations, which may be attributed to limited Internet bandwidth preventing interactive, and/or multimedia services from being delivered efficiently and cost effectively. Attempts to overcome this particular problem have been based upon a specific application being created, and delivered on a transportable medium such as a CD-ROM which comprises physical media required at the user's site. Users may then use the application provided by the CD-ROM in a textual manner, and results may be physically printed and manually collated on site. However, the CD-ROM media may have insufficient storage to allow multimedia applications to operate effectively.

Any discussion of documents, devices, acts or knowledge in this specification is included to explain the context of the invention. It should not be taken as an admission that any of the material forms a part of the prior art base or the common general knowledge in the relevant art in Australia or elsewhere on or before the priority date of the disclosure and claims herein. SUMMARY OF INVENTION

An object of the present invention is to alleviate at least one disadvantage associated with the related art or to at least provide a useful alternative to related art systems.

In a first aspect of embodiments described herein there is provided an apparatus adapted to and / or computer implemented method of educating a user, comprising the steps of providing access for the user to a virtual training room comprising at least one page of a Graphical User Interface wherein the virtual training room is operatively associated with an information repository; enabling the user to select at least one module of a field of knowledge from the information repository to be validated against; and in response to the user's selection generating a quiz for the user wherein the quiz comprises one or any combination of questions, answers and instructional content from the information repository; and presenting the quiz to the user as a sequence of dynamically created pages of a Graphical User Interface.

At least the step of presenting the quiz may comprise server to web browser communication in which a plurality of message transfers from a server are correlated with a single message transfer acknowledgement.

In another aspect of embodiments described herein there is provided apparatus adapted to and / or method for educating a user comprising an information repository for generating a Data Store comprising at least one user selectable module corresponding to a field of knowledge; an Education Interface Engine for compiling a quiz comprising at least one computer displayable page of questions relating to the user selected module; and web server means for asynchronously communicating the at least one computer displayable page to the user to enable the user to complete the quiz. The Education Interface Engine may be adapted to asynchronously communicate with the Data Store and may further comprise a Random Generator for randomising the questions of the quiz. Preferably, the Random Generator is adapted to perform a randomisation during one of compiling a quiz, and / or asynchronously communicating to the user. The apparatus may further comprise a client device for displaying the at least one computer displayable page to the user and wherein the client device is adapted to asynchronously communicate user messages to the Web server.

In one preferred form, the information repository, the Education Interface Engine, the Web server and the client device are adapted for asynchronously communicating with each other wherein the asynchronous communication comprises a plurality of message transfers correlated with a single acknowledgement.

The at least one computer displayable page and the user messages may comprise interactive multimedia content. The interactive multimedia content may be adapted for one or a combination of text; video; animation; closed captioning for hearing impaired users; audio comprising speech for low literacy skilled users. The information repository is preferably adapted to store one or a combination of questions, quizzes, and quiz results at the Data Store. Also, preferably, the Data Store comprises a relational database.

In yet another aspect of embodiments described herein there is provided a device for and / or computer implemented method of randomising a quiz suitable for use in testing a user's knowledge and skills, comprising the steps of determining a first set of questions associated with a user configuration relating to a field of knowledge; determining a second random subset of questions from the • first set of questions based on predetermined rules associated with the user configuration; for each question determining a random set of answers based on predetermined rules associated with the context of the question; generating the quiz such that each question of the second random subset is collated with its respective random answers.

The predetermined rules may be based on configurable criteria comprising one or any combination of: the quiz name and description; the total number of possible questions; any mandatory questions; the number of questions required per quiz; the number of question types; the number of compulsory question for a given quiz; the answers to be available for each question, and in the case of multiple choice questions, when the answers are to be shuffled, overriding criteria to position an answer in this ordering (e.g. "All of the above" to be put last)

In still another aspect of embodiments described herein there is provided apparatus for and / or method of randomising a quiz suitable for use in testing a user's knowledge and skills, comprising first determining means for determining a first set of questions associated with a user configuration relating to a field of knowledge; second determining means for determining a second random subset of questions from the first set of questions based on predetermined rules associated with the user configuration; third determining means for determining, for each question, a random set of answers based on predetermined rules associated with the context of the question; and quiz generating means for generating the quiz such that each question of the second random subset is collated with its respective random answers.

The apparatus may further comprise storage means for storing the generated quiz in a quiz repository. A relational database may also be used for storing questions and quizzes.

In another aspect of embodiments described herein there is provided apparatus adapted to and / or method for educating a user comprising a client device operatively associated with a Graphical User Interface application for use by the user; a server means for providing the user access to a virtual training room comprising at least one page of a Graphical User Interface, wherein the server means comprises a Web server for presenting the user with pages for display on the Graphical User Interface, an information repository for storing information content, and an Education Interface Engine adapted for managing communication between the client device and the server means.

The Education Interface Engine is preferably adapted to generate a quiz for the user as a sequence of dynamically created pages of a Graphical User Interface and the dynamically created pages may comprise interactive multimedia content. In a preferred form, at least one of the Web server, the information repository, the Education Interface Engine and the client device is adapted to provide asynchronous communication.

The dynamically created pages may be provided by asynchronous communication between the client device and the server means. Preferably, the asynchronous communication comprises Asynchronous JavaScript and XML (AJAX).

In this preferred form of apparatus, the server means is adapted to provide validation for a user in which the validation comprises one or any combination of: closed captioned subtitles to video for hearing impaired users; audio playback to text-based content for users with low literacy levels; audio feedback for user interface controls; multimedia reviews of quiz questions and answers to assist users with learning difficulties; video and animation reviews after questions and answers to provide relevance reinforcement.

Further, the Education Interface Engine may comprise a Message Broker adapted to combine data and/or functions from a plurality of sources and scripting libraries for facilitating asynchronous communication between one or more of: the client device; the Web server; the information repository

The Message Broker may be further adapted to perform message aggregation wherein synchronous messages are intercepted-from one or more of the Web server and the information repository, aggregated and then transferred asynchronously to external libraries. The Message Broker may comprise one or a combination of:

Microsoft ASP .NET AJAX; The Prototype JavaScript Framework;

JSON (JavaScript Object Notation);

Microsoft .NET Framework 2.0.

In a preferred form the Education Interface Engine further comprises a dynamic data switch which comprises a class library adapted for referencing a .NET Framework data provider class library. The dynamic data switch may be adapted to communicate with the information repository by executing SQL queries.

In yet a further aspect of embodiments described herein there is provided a device adapted to and / or computer implemented method of educating a user, comprising generating a Data Store comprising at least one user selectable module corresponding to a field of knowledge; compiling a quiz comprising at least one computer displayable page of questions relating to the user selected module; and asynchronously communicating the at least one computer displayable page to the user to enable the user to complete the quiz. The step of compiling may comprise asynchronously communicating with the Data Store. The preferred method may further comprise the step of randomising the questions of the quiz, wherein, in one form, the step of randomising is performed during one of the steps of: compiling a quiz, and; asynchronously communicating to the user.

The preferred method may further comprise the step of: asynchronously communicating user messages from a user's client device to a server adapted to perform at least one of the steps disclosed hereinabove.

Preferably, the steps of asynchronously communicating comprise a plurality of message transfers correlated with a single acknowledgement.

It is also preferred that the at least one computer displayable page and the user messages comprise interactive multimedia content and the interactive multimedia content may be adapted for one or a combination of: text; video; animation; closed captioning for hearing impaired users; audio comprising speech for low literacy skilled users.

The preferred method may further comprise the step of storing one or a combination of questions, quizzes, and quiz results in the Data Store.

In preferred embodiments there is provided apparatus adapted to and / or method of educating a user, comprising processor means adapted to operate in accordance with a predetermined instruction set, said apparatus, in conjunction with said instruction set, being adapted to perform the method steps as disclosed herein.

In further preferred embodiments there is also provided a computer program product comprising a computer usable medium having computer readable program code and computer readable system code embodied on said medium for educating a user within a data processing system, said computer program product comprising computer readable code within said computer usable medium for performing the method steps as disclosed herein.

Other aspects and preferred forms are disclosed in the specification and/or defined in the appended claims, forming a part of the description of the invention.

In essence, embodiments of the present invention stem from the realization that asynchronous communication techniques allow for an enriched and effective learning and validation experience for a user when education and training are required at remote locations or in an online environment. Further, a randomisation of questions and answers in accordance with configurable rules as disclosed herein enhances the veracity of an online education, learning and validation tool. Advantages provided by the embodiments of the present invention disclosed herein comprise the following:

• Dynamic validation of a user's knowledge and/or skills may be delivered reducing circumvention opportunities.

• Immediate results may be provided to a user. • Where no immediate re-enforcement of relevancy is available in related art systems, at the completion of a question, regardless of accuracy, re-enforcement of the relevance is provided.

• Validation questions may be provided in both written and spoken mediums and allows users to respond in an interactive manner. • The same level of validation may be provided regardless of physical location.

• ' Provides 24 hours a day 7 days a week access from anywhere including regional and remote areas. This makes training and validation easier for businesses. • Randomisation of questions and answers with video and animation reviews allows for more effective training. It is therefore very hard to cheat and the review process means that trainees learn as they go.

• Interactive and video based questions and answers allows training to be more effective. Thus high impact and engaging content that is easy to understand for people who have low levels of written literacy may be provided.

• Closed Captioning for the hearing impaired allows hearing impaired people to use the system.

• Spoken questions, answers and review allows for people with low literacy skills to use the system.

• Fully hosted results accessible 24 hours a day 7 days a week. This means the system stores all data and can be accessed at anytime which, In turn, means the system can be used from home or on the road or anywhere in the world.

• Reporting and validation of trainees results is available meaning organisations can prove training has occurred and identify strengths and weaknesses in their workforce.

• Can be used on wireless and mobile networks. This means it can be used in remote and regional areas on the road.

• Job Evaluation and Aptitude testing & validation of employees/prospective employees/contractors for all industries including the Labour Hire Industry. This may improve efficiency and quality of the industry, for example, Job evaluation and Aptitude testing & validation for health and safety for Contractors may improve the health and safety of Contractors and reduce health and safety costs. The system also helps with selecting potential job applicants and therefore improves standards such as for example, health and safety standards.

• Being a Global system means multi-national businesses and organisations can use the system.

• As a fully hosted system IT. costs may be reduced for global, national and local businesses and organisations and provide access anywhere anytime.

Further scope of applicability of embodiments of the present invention will become apparent from the detailed description given hereinafter. However, it should be understood that the detailed description and specific examples, while indicating preferred embodiments of the invention, are given by way of illustration only, since various changes and modifications within the spirit and scope of the disclosure herein will become apparent to those skilled in the art from this detailed description. BRIEF DESCRIPTION OF THE DRAWINGS Further disclosure, objects, advantages and aspects of preferred and other embodiments of the present application may be better understood by those skilled in the relevant art by reference to the following description of embodiments taken in conjunction with the accompanying drawings, which are given by way of illustration only, and thus are not limitative of the disclosure herein, and in which:

Figure 1 is a system block diagram of an exemplary architecture for a web- based interactive training facility in accordance with a preferred embodiment; Figure 2 illustrates the logical interaction between elements of a web- based system in accordance with a preferred embodiment;

Figure 3 is a logical flow diagram of a method of generating quiz questions in accordance with another embodiment. DETAILED DESCRIPTION Preferred aspects of embodiments disclosed herein are applicable to an online or Web based interactive training facility for staff and employees of businesses as well as other organisations and may also be applicable to contractors and interviewees for positions in a given field of activity. In certain embodiments, it is envisaged that preferred aspects as disclosed herein may be applicable to remote education and schooling such as for example in facilitating the schooling of children in Outback" Australia and other remote regions of the world where it may be difficult to provide robust assessment of a person's education with limited facilities onsite.

A preferred embodiment provides an online testing and validation service that is interactive, intuitive and user friendly and may be accessible 24hrs a day, 7 days a week from any location where Internet access is available including wireless Internet access. With reference to figure 1 , the preferred educational, training and validation facility comprises a server application, generally indicated at 10, that is typically located in an Internet data centre and a multimedia enabled web browser located or residing on a client device (not shown), and which client device may comprise a computer, a hand held digital assistant or equivalent device. The server 10 may be loaded with a HTTP server 11 , an information repository 12 such as a relational database or an equivalent Data Store, and an Education Interface Engine 13. The client (not shown) may be loaded with a standard web browser such as Microsoft™ Internet Explorer™ or Firefox™, preferably with Adobe™ Flash player plug-in loaded.

As a general overview, the preferred interactive education, training and validation process is accomplished by way of: • Using customised multimedia content that's relevant to the user and his/her circumstance;

• Using Asynchronous JavaScript and XML (AJAX) techniques to provide communications asynchronously between the building blocks of the server 10. By leveraging an asynchronous process such as the AJAX communication process, the user experience may be greatly enhanced. The asynchronous nature of the AJAX calls allows for the dynamic rendering of the HTML web page, without the need for a traditional page reload. This may be achieved in a preferred form by placing a background web request to the server and receiving a partial web page render response from the server. This response is then dynamically applied to the currently loaded HTML web page displayed in the browser.

• Leveraging server side scripting to provide access to and manipulation of the Data Store 12 and operating system functions. • Using a browser embedded media player to deliver content and the validation interaction.

A validation User Interface has been designed to provide a user with accessibility features, such as:

• Closed captioned subtitles to all relevant video for hearing impaired users;

• Audio playback to text-based content for users with low literacy levels;

• Audio feedback for User Interface controls to enhance the user experience; • Multimedia reviews to assist users with learning difficulties;

• Video and animation reviews after questions and answers enforce the trainee to learn as they go providing relevance reinforcement.

With general reference to figure 2, in operation the user directs their browser to the website 21 enabling the education interface where a log in page may be presented 22. The user authenticates themselves to the system. In one form this may be achieved by the user logging into the site with a specific username and password. Upon successful authentication, the user is then presented with a list of "Modules" or "examinations" that need to be completed, preferably in the form of quizzes. The user then selects the module that they are to be validated against and the Quiz/Examination is dynamically created by the Education Interface Engine 13 located on the server 10. A page is then dynamically created and the content pushed by the server 10 via the HTTP server 11 to the browser. As the quiz progresses, the browser is dynamically updated with new questions and content without the need for a web page reload. At the conclusion of the Quiz/Examination the results are collated and reported. This preferred facility is accessible from anywhere in the world where Internet connectivity is available. This enables businesses or organisations to operate their training and validation requirements across regional boundaries and time zones. It reduces the investment involved with infrastructure and building costs to maintain a physical "training room". Using the virtual online "Training Room" of the preferred embodiment, users can be validated against training modules at anytime at any location with results securely delivered to a training manager.

In an example field of learning and training such as occupational health & safety learning, it is considered that such learning is most effective through interactive multimedia. Accordingly, all validation services of the preferred embodiment are designed interactively to maximise the user experience involved in the delivery of the training content. Further, by providing customised randomisation of the content and multimedia reviews, described below, the preferred embodiment encourages and enables learning much more effectively.

Content accessibility is a major feature of the preferred validation service. It has capacity to deliver closed captioned subtitles to video for the hearing impaired, while audio playback of content further enhances the user experience and provides users with low literacy skills to participate in safety learning. Accordingly, the system does not discriminate users with accessibility issues, but encourages the participation of all users. At the conclusion of Examinations/Validations results are collated automatically and reported back to a Training manager in a consistent manner.

With more detailed reference to figure 1 , the online validation service is controlled by the Education Interface Engine 13, which manages the communication and content services between the HTTP server 11 and the client web browser (not shown) for the website. The Education Interface Engine 13 provides messaging logic between server-side and client-side scripts to control and determine the sequence of display of dynamic multimedia content. The Education Interface Engine 13 employs a number of external libraries 14, comprising at least one software component being a system element offering a predefined service and able to communicate with other components, to assist in providing optimal performance and scalability.

The preferred external libraries utilized by the system are:

• Apache™ Iog4net¹ • Microsoft™ ASP.NET AJAX²

• Microsoft™ .NET Framework 2.0³

The above noted external libraries are Microsoft™ Windows™ based in the preferred embodiment. However, it is envisaged that corresponding or equivalent functioning software components as would be recognised by the person skilled in the art may be applied in other computing platforms and operating systems such as, for example, Apple Macintosh™, Unix™ or Linux™.

Apache™ Iog4net is used to provide logging services for debugging the engine's 13 class libraries and the web application. The engine 13 references the Iog4net library of tools via the .NET Global Assembly Cache (GAC) to output log statements to various output targets. Output targets may be, for example, a file or files on a disk, a database, a TCP/IP socket or port, an event log, an email or any other equivalent destination as would be understood by the person skilled in the art.

The preferred- website facility currently takes advantage of the .NET Framework and its ASP.NET AJAX libraries to create Rich Internet Applications⁴ (RIA). However, it is to be noted that the facility is not to be restricted or limited to a .NET Framework solution, and accordingly the Education Interface Engine 13 may be applied and/or ported to other programming languages, such as for example, "LAMP" (Linux, Apache, Mysql, PHP) or Ruby on Rails.

¹ http://en.wikipedia.org/wiki/Log4net

² http://en.wikiρedia.org/wild/ASP.NET_AJAX * http_//en.wikipedia.org/wiki/.NET_Framework 4 http://en.wikipedia.org/wiki/Rich_Intemet_Apρlioation The Education Interface Engine 13 in its preferred form is written in the Microsoft™ C# programming code specifically adapted for the .NET environment. It is noted that a direct port or a recompile of the C# source to a Visual C++ runtime would achieve a similar result. Alternatively, porting the code base to Java is envisaged and in theory achievable, but may be different and/or restricted to how the Java runtime operates. Or alternatively again, a direct port to other Common Language Infrastructure (CLI)⁵ standards and variations such as Mono may be more achievable.

The Message Broker 15 located within the Education Interface Engine 13 as shown in figure 1 comprises a set of server-side and client-side frameworks and scripting libraries that facilitate the asynchronous communication process between the HTTP server 11 and the client web browser (not shown). The Message Broker 15 thus provides a communications pathway between the browser and the HTTP server. In a preferred form, the Message Broker 15 manipulates the traditional operational mode of the browser to enable it to mimic a locally installed computer program. The Message Broker 15 acts as the glue and mediator between the engine 13 and the user, the engine 13 and the database 12 and maintains the application state for the validation process. The application state comprises a state machine function that may be stored in a relational database that may form the Data Store 12. The Message Broker 15 may dynamically monitor the state and where appropriate alter the state value based upon defined events. An example of a defined event may be a user commencing a session. The Message Broker 15 does not simply mediate between the different client/server scripting libraries; it functions to combine data and functions that are gathered from multiple sources and different scripting libraries that function together coherently and seamlessly to build the framework upon which the messages between the browser and the Education Interface Engine 13 can transfer. One preferred functional purpose of the Message Broker 15 is to provide a user experience that is intuitive and interactive while remaining invisible to the user. This architecture and implementation of the Message Broker

^! httρ://eπ.wikipedia.org/wiki/Common_Latiguage_In£rastructure 15 allows the Education Interface Engine 13 to not require page reloads between interactions occurring on the currently loaded web page.

The Message Broker 15 comprises a range of web technologies, such as a combination of one or more of: • Microsoft ASP.NET AJAX

• Prototype - a JavaScript Framework

• JSON (JavaScript Object Notation)

• Microsoft .NET Framework 2.0

Some of the functions and advantages of the Education Interface Engine 13 comprise:

• Richer User Interface (hereafter referred to as Ul)- a user interface that provides intuitive behaviours and allows the user to interact in a friendly and simple way. In the present web environment, and the way in which we interact with the web and the internet in general may form the benchmark to which all new applications must meet or exceed. A Richer Ul may therefore be an Internet application that is finding new ways of using web technologies to provide content to a user that provides a more involved sensory experience for the user compared with present benchmarks. The preferred application accordingly provides a broad spectrum of ways of delivering content to users. The inclusion of multimedia content, Video, Audio, subtitles, and interactive graphical animations combine to deliver a Richer Ul.

• Responsive Ul - a user interface that provides indicators and animations of the client/server communications state. In present Client / Server environments, delays associated with server communication may be overt or clearly evident to users. For example, poor network communications and processor intensive server side applications can result in delays. Resultant delays brings rise to user frustration, lost productivity and a bad User Interaction (Ul). By alleviating the need for page reloads, the user never actually experiences the transfer time between the browser client and the server; the Message Broker 15 transfers only the required portions of the page in the background and this transfer is performed outside of the usual browser page loading mechanism. 15. In addition, the Adobe Flash application content is downloaded to the user's computer cache. The content is then locally retrieved from the cache as and when it is required under control of the HMTL page loaded in the active browser window.

• Asynchronous communication - the engine 13 communicates with the server 11 asynchronously, and removes the responsibility of the user to initiate or perform an action to retrieve new content for the page. It works in the background seamlessly without the user moving a finger. For example, and by way of explanation, in a normal HTTP server to browser interaction, each message transfer is required to be acknowledged by the browser. With Asynchronous transfer, multiple messages can be transferred and the acknowledgements can be correlated. In some instances where a higher or lower level protocol such as TCP/IP is performing a communication guarantee function, then there is no longer a need to duplicate the acknowledge functions.

• Network efficiency - the network traffic is reduced and managed in a more intelligent way by using an engine 13 that controls when and what to communicate with the server. The network traffic is preferably reduced by message aggregation that may be performed by the Message Broker. In a preferred form, as the Message Broker 15 is intercepting synchronous messages from the HTTP server 11 and the Data Store 12, these messages are aggregated and transferred asynchronously to the external libraries 14.

A Dynamic Data Switch 16 located within the Education Interface Engine 13 comprises a class library of data manipulation logic and data methods used by the engine 13, to provide communications between the database and the engine. The class library is written in C#, and references the .NET Frameworks' data provider class library. It communicates to the Data Store 12 by executing SQL queries against the database 12, and returning relevant data tables and/or data objects. The preferred database used is Microsoft™ SQL Server 2005, and specifically the class library executes Stored Procedures, which are precompiled SQL queries that reside in the database 12, and may be, for example, specifically a Microsoft™ technology used by an SQL Server. A Stored Procedure is a function that has been defined in the database that performs multiple queries of the database and any associated calculations to produce an output that is refined. The purpose of the Stored procedures is to reduce the query load on the database. An alternative relational database may be used, with corresponding additional changes to the Dynamic Data Switch class library and its data providers. A .NET data provider is the layer between the application and the database that takes care of the essential work of database interaction. The SQL Server data provider is currently used, and is the most efficient way to connect to an SQL Server database. An alternative relational database may require a different data provider that uses different network protocols for connecting to the database. This may inturn provide a faster data transaction throughput. For example, possible alternatives may be Oracle or MySQL. An Oracle .NET data provider is currently available as a solution, while the MySQL database will require something different, such as an ADO .NET data provider or a direct socket level interaction to the database.

The class library is referenced by the Message Broker 15 to provide relevant content to the user while also maintaining consistent application state during a validation session. Based upon the Broker Action that is passed to the Message Broker, the content is determined, processed and sent to the user. The Application state is stored in the database. Depending upon the broker action that is requested, the Message Broker 15 will process and retrieve the current state of the user from the database. For example, when a session is started it will have a session state of 1 and a session state of 2 is applied once the session has progressed to the next defined stage. The session state is continually updated as the session passes though the system until the session is completed.

A Random Generator 17 uses the data switching class library differently from the Message Broker 15. The data the Random Generator 17 requires is that of quizzes, questions, and answers - as opposed to users, passwords, and screen layouts, which are handled by the Message Broker. Hence its interactions with the data switching library concern the retrieval and modification of these types of data. The data switching class library provides an abstracted, generic interface to the application's Data Stores, which is available to all other parts of the application. The Message Broker 15 comprises a range of server-side scripts and client-side scripts. The server-side scripts are a set of C# classes that inherit from the following ASP.NET libraries:

• System.Web • System.Web.Security

• System.Web.Extensions

The System.Web libraries are used by all web pages developed in ASP.NET and provide the HTTPRequest and HTTPResponse objects to manage the HTTP messaging and are also used to generate custom JavaScript for the client side. The custom JavaScript library is a set of function calls that provide the interactivity of the HTML page and controls a Flash component. A Flash Player container provides the Ul and multimedia content for each quiz. The client-side JavaScript enables the retrieval and display of a SWF file on the HTML page, and also injects JSON objects into the Flash component's ActionScript functions. The Flash component uses an External Interface⁶ API library by Macromedia, lnc to enable cross-page communication between the HTML page and the SWF file. Externallnterface is similar in functionality to the fscommand(), CallFrame() and CallLabelO methods, but is more flexible and more generally applicable. Use of Externallnterface is recommended for JavaScript-ActionScript communication. The System.Web.Security libraries provide user authentication and identity checks against various user roles. The System.Web.Extensions libraries are the AJAX libraries, used to provide asynchronous HTTP communications without refreshing the web page. It contains a range of AJAX Web Ul controls to manage data, various JavaScript converters and serializers, and enables messaging with JSON response objects. The Message Broker 15 is dependant on this library to control the messaging between the server-side scripts and the client-side scripts. With reference to figure 2, the following steps are performed in operation for a validation process for each quiz:

• User navigates to the quiz listing page 23 after user authentication 22 at the home page 21.

htφ:/Λivedocs.adobe.cora/flash/8/main/wwhdp/wwhimp)/common/html/wwhelρ.htm?context=LiveDocs_P arts&file=00002200.html • User selects a quiz to start the validation session, ie start quiz 28. Query Broker 24 returns a listing of the Quiz's that are to be completed by the user.

• Server 11 initiates the "Generate Quiz" action 29. • Message Broker 211 begins generating the random quiz.

• Random Generator 17 212 is called and processes the generation of the Random Quiz.

• A random quiz is generated and updates the Repository 27 of quiz questions. • Server 11 sends a URL redirection to the client browser.

• User is redirected to the Quiz page 213.

• At this point Quiz 213, Server 11 checks user permissions against database .

• At 213 Server 11 checks that the quiz settings (as shown in figure 3 at step Get Settings 31 ) are valid

• Server 11 generates user key for the Quiz, again at 213.

• Server 11 generates new client script with relevant settings for the quiz. Quiz 213

• Quiz page begins downloading client-side JavaScript (js) library and CSS. Quiz 213

• Client-side js library begin various function timers. Quiz 213

• Client-side js timers are loaded and generates dynamic HTML. Quiz 213

• DHTML is generated for Flash Player container. Quiz 213 • js library creates proxy to Flash Player. Quiz 213

• Flash Player registers the Extemallnterface API. Quiz 213

• Flash Player ActionScript library registers JavaScript callbacks. Quiz 213

• js library calls ActionScript function to initialize, via proxy callback function. Quiz 213

• ActionScript returns and confirms communication flow. Quiz 213 • js libraries initiates WebService call back to server to retrieve Quiz objects. Quiz 213

• JSON objects returned to js library. Quiz 213

• js library validates data returned and injects JSON into Flash proxy. Quiz 213

• ActionScript receives Quiz object and validates data objects. Quiz 213

• ActionScript calls js library to initiate "Broker Action" 214 START__QUIZ 28. ^' • js library calls WebService to START_QUIZ 28.

• ActionScript begins the Quiz by changing the Ul display at 214

• User answers question and updates ActionScript data at 216.

• ActionScript calls js library to initiate "Broker Action" FINISH_QUESTION at 214 • js library calls WebService to FINISH_QUESTION. See 216

• Server processes "Broker Action" FINISH_QUESTION see 214

• js library receives action status from Server, see 214

• User continues Quiz (repeat Step 30 to 34) 213.

• User completes Quiz 213 , • ActionScript calls js library to initiate "Broker Action" 214 FINISH_QUIZ. See 214

• js library calls WebService to FINISH_QUIZ. See 216

• Server processes "Broker Action" FINISH_QUIZ 218

• Server redirects user to Results page 219. The Training Room is a conceptual area or virtual construct of the preferred facility website, specifically designed for trainees to participate in an online validation Quiz. With further reference to figure 2 a "Quiz" is a general term used to describe a set of questions and answers. Other terms may also be used to describe the online test or validation process. The "Training Room" sub-site or validation area of the website comprises a number of web pages providing relevant information as follows. The User Home 21 page allows only authenticated users into the sub-site. All users must be authenticated 22, via a login page. The Listing 23 page displays a listing of uncompleted or unstarted quizzes that have been assigned to the user. Once a user chooses to begin the validation process or initiates to start a quiz, the "Message Broker" 211 is used to generate a new random Quiz for the user. The Quiz 213 page is the actual validation page that is used to validate a user's knowledge. It comprises various "Broker Actions" 214, which is a term used to describe a range of commands or actions that are required by the Quiz to maintain session state and to manage the communications flow between the user and the server 11.

The following shows a sample of some basic commands or "Broker Actions":

Broker Action Description

LOAD_QUIZ Get random quiz that is generated

START_QUIZ Start the quiz and update the quiz state

FINISH_QUZ Update the quiz state to finished and associated data GET_QUIZ Get the specific quiz data (i.e. number of questions)

START_QUESTION Update the question state for the quiz

FINISH-QUESTiON Update the question state to finished

The Message Broker 15 sends and receives various actions during the process of the Quiz. The messaging sequence or communications flow is determined by the state machine which resides inside the Education Interface

Engine. The state machine is maintained by the Message Broker 15 and the status is an integer value that is stored in the database. The Results 219 page shows the immediate results acquired after completing a validation Quiz. It displays a summary of completed validation quizzes and their performance results.

In conducting a quiz, the Random Generator 17 shown in figure 1 is directed to the process of generating a set of random questions and answers, based on a series of configuration settings. With reference to figure 3 the process comprises the following steps: Step 31 "Get Settings" - obtain relevant settings from the site configuration.

Step 34 "Get Questions" - obtain all relevant questions from the "Questions Repository" 36 based on settings 33. Settings may be configurable rules that are to be applied to the particular session. An example of a configurable rule is whether or not a particular question is mandatory or not. Further description of settings is below.

Step 38 "Randomize Questions" - a set of random questions is created from the list of questions available, based on rules provided in the settings 33. Step 311 "Randomize Answers" - a set of random answers per question is created based on a set of rules relevant to the question type. Step 313 "Generate Quiz" - a new random quiz is generated by collating the random questions and their respective random answers. Step 316 "Update Repository" - once a random quiz is generated, it is saved to the "Quiz Repository" 317 for access.

The "Questions Repository" 36 is a conceptual term used to describe where all the original questions are stored, while the "Quiz Repository" 317 is a subset, and only stores data (questions and answers) relevant for the Quiz. In the preferred embodiment the "Questions Repository" 36 and "Quiz

Repository" 317 comprise a relational database, or Data Store, which contains a series of tables. The "Repository" or Data Store 12 is not limited to a relational database.

There are a range of settings which are used to determine the rate of distribution of questions and answers for each Quiz. Some of these settings may comprise:

• Name and code of the Quiz

• Description of the Quiz

• Culture (language) of the Quiz • Topics within the Quiz

• Number of required questions per Quiz

• Number of question types (i.e. Multiple Choice, True / False etc)

• Number of compulsory questions per Quiz quiz name Filters and settings are used to query the "Questions Repository" 36 for relevant questions. An example may comprise questions which only have a particular category as its property / attribute, such as "Green", as shown in the sample table below: Attribute Question Question Type

Green Name three primary colours. Pick a Box

Green What are the 3 primary colours? Multiple Choice

Green Is Blue a primary colour? True / False

Red Is Green a primary colour? True / False

The pseudo-code on the following pages is a summary of an example randomization process that is applied by the Random Generator 17.

Procedure to generate a random quiz

ω c

CD CO

m

CO

I m m

Ul

C m

73 O

:>

C

ω c

CO ω

m ω

I m m N3 05

Ta ι- m σ>

Ti O

:>

Annex 1 provides the syntax of the above pseudo code for the procedure to generate a random quiz.

While this invention has been described in connection with specific embodiments thereof, it will be understood that it is capable of further modification(s). This application is intended to cover any variations uses or adaptations of the invention following in general, the principles of the invention and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains and as may be applied to the essential features hereinbefore set forth. For example, an alternative to the web based solution may comprise a

CDROM or DVD based content delivery system. This requires the physical media to be delivered to the client. The difference between the CDROM and DVD disc media formats is the physical data storage capacities of the media, and does not affect the application. Both methods, CDROM and DVD₁ require an application to be installed on the client PC. These applications are platform dependent and may contain dependencies on the host operating system. The Web based system is substantially platform independent and does not require the installation of a proprietary education interface application onto the client PC. It would consist of an alternative engine, without the Message Broker 15, because the Education Interface Engine 13 is specifically designed to communicate, via the Message Broker 15, using internet protocols and technologies. The engine 13 may maintain the same Random Generator 17, white an alternative Dynamic Data Switch 16 may need to communicate to a different type of Data Store. An online Data Store 12 can still be used in this scenario, but user authentication to the data may need to be considered in an alternate fashion.

Another alternative is to provide the content via a PDA device or Mobile Telephone. This method may need changes to the methods of content delivery and user interface design to compensate for the technology constraints as would be recognised by the person skilled in the art. A modification to the Education Interface Engine 13 may also be required to accommodate for the changes in messaging and communications protocols that are required for portable devices and these would be recognised by the person skilled in the art. As the present invention may be embodied in several forms without departing from the spirit of the essential characteristics of the invention, it should be understood that the above described embodiments are not to limit the present invention unless otherwise specified, but rather should be construed broadly within the spirit and scope of the invention as defined in the appended claims. The described embodiments are to be considered in all respects as illustrative only and not restrictive.

Various modifications and equivalent arrangements are intended to be included within the spirit and scope of the invention and appended claims. Therefore, the specific embodiments are to be understood to be illustrative of the many ways in which the principles of the present invention may be practiced. In the following claims, means-plus-function clauses are intended to cover structures as performing the defined function and not only structural equivalents, but also equivalent structures. For example, although a nail and a screw may not be structural equivalents in that a nail employs a cylindrical surface to secure wooden parts together, whereas a screw employs a helical surface to secure wooden parts together, in the environment of fastening wooden parts, a nail and a screw are equivalent structures.

It should be noted that where the terms "server", "secure server" or similar terms are used herein, a communication device is described that may be used in a communication system, unless the context otherwise requires, and should not be construed to limit the present invention to any particular communication device type. Thus, a communication device may Include, without limitation, a bridge, router, bridge-router (router), switch, node, or other communication device, which may or may not be secure.

It should also be noted that where a flowchart or its logical equivalent is used herein to demonstrate various aspects of the invention, it should not be construed to limit the present invention to any particular logic flow or logic implementation. The described logic may be partitioned into different logic blocks (e.g., programs, modules, functions, or subroutines) without changing the overall results or otherwise departing from the true scope of the invention. Often, logic elements may be added, modified, omitted, performed in a different order, or implemented using different logic constructs (e.g., logic gates, looping primitives, conditional logic, and other logic constructs) without changing the overall results or otherwise departing from the true scope of the invention.

Various embodiments of the invention may be embodied in many different forms, comprising computer program logic for use with a processor (e.g., a microprocessor, microcontroller, digital signal processor, or general purpose computer), programmable logic for use with a programmable logic device (e.g., a Field Programmable Gate Array (FPGA) or other PLD), discrete components, integrated circuitry (e.g., an Application Specific Integrated Circuit (ASIC)), or any other means comprising any combination thereof. In an exemplary embodiment of the present invention, predominantly all of the communication between users and the server is implemented as a set of computer program instructions that is converted into a computer executable form, stored as such in a computer readable medium, and executed by a microprocessor under the control of an operating system. Computer program logic implementing all or part of the functionality where described herein may be embodied in various forms, comprising a source code form, a computer executable form, and various intermediate forms (e.g., forms generated by an assembler, compiler, linker, or locator). Source code may comprise a series of computer program instructions implemented in any of various programming languages (e.g., an object code, an assembly language, or a high-level language such as Fortran, C, C++, JAVA, or HTML) for use with various operating systems or operating environments. The source code may define and use various data structures and communication messages. The source code may be in a computer executable form (e.g., via an interpreter), or the source code may be converted (e.g., via a translator, assembler, or compiler) into a computer executable form.

The computer program may be fixed in any form (e.g., source code form, computer executable form, or an intermediate form) either permanently or transitorily in a tangible storage medium, such as a semiconductor memory device (e.g., a RAM, ROM, PROM, EEPROM, or Flash-Programmable RAM), a magnetic memory device (e.g., a^* diskette or fixed disk), an optical memory device (e.g., a CD-ROM or DVD-ROM), a PC card (e.g., PCMCIA card), or other memory device. The computer program may be fixed in any form in a signal that is transmittable to a computer using any of various communication technologies, comprising, but in no way limited to, analog technologies, digital technologies, optical technologies, wireless technologies (e.g., Bluetooth), networking technologies, and inter-networking technologies. The computer program may be distributed in any form as a removable storage medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the communication system (e.g., the Internet or World Wide Web). Hardware logic (comprising programmable logic for use with a programmable logic device) implementing all or part of the functionality where described herein may be designed using traditional manual methods, or may be designed, captured, simulated, or documented electronically using various tools, such as Computer Aided Design (CAD), a hardware description language (e.g., VHDL or AHDL), or a PLD programming language (e.g., PALASM, ABEL, or CUPL). Programmable logic may be fixed either permanently or transitorily in a tangible storage medium, such as a semiconductor memory device (e.g., a RAM, ROM, PROM, EEPROM, or Flash-Programmable RAM), a magnetic memory device (e.g., a diskette or fixed disk), an optical memory device (e.g., a CD-ROM or DVD-ROM), or other memory device. The programmable logic may be fixed in a signal that is transmittable to a computer using any of various communication technologies, comprising, but in no way limited to, analog technologies, digital technologies, optical technologies, wireless technologies (e.g., Bluetooth), networking technologies, and internetworking technologies. The programmable logic may be distributed as a removable storage medium with accompanying printed or electronic documentation (e.g., shrink wrapped software), preloaded with a computer system (e.g., on system ROM or fixed disk), or distributed from a server or electronic bulletin board over the communication system (e.g., the Internet or World Wide Web). 'Comprises/comprising" when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof." Thus, unless the context clearly requires otherwise, throughout the description and the claims, the words 'comprise', 'comprising', and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to".

Annex 1 procedure generate_random_quiz() set query_settings = get configuration settings set num_required = get value from query_settings set questions = get all questions from repository set raπdom_set = create empty collection from num_required do while total questions of random_set < num_required set random_index = get random index from questions pop question from questions based on random index push question into random_set

loop

set special_words = get value from query_settings

for each question in random_set pop answers from question set shuffled_answers = execute random_shuffle passing answers

for each answer in shuffled_aπswers if answer contains special_words

pop answer into last_answer end if loop push last_answer into shuffled_answers into last position push shuffled_answers into question loop return random_quiz end procedure

procedure random_shuffle set listing = create collection of answers do while counter < total number of items in listing

set randomjndex = get random index from listing

set tempjtem = get item from listing[counter] set listingfcouπter] = listing[raπdom_index] set listing[randomjndex] = temp_item;

loop return listing end procedure

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS:

1. A computer implemented method of educating a user, the method comprising the steps of: providing access for the user to a virtual training room comprising at least one page of a Graphical User Interface wherein the virtual training room is operatively associated with an information repository; enabling the user to select at least one module of a field of knowledge from the information repository to be validated against; in response to the user's selection generating a quiz for the user wherein the quiz comprises one or any combination of questions, answers and instructional content from the information repository; presenting the quiz to the user as a sequence of dynamically created pages of a Graphical User Interface.

2. A method as claimed in claim 1 wherein at least the step of presenting the quiz comprises server to web browser communication in which a plurality of message transfers from a server are correlated with a single message transfer acknowledgement. ^■

3. Apparatus for educating a user comprising; a client device operatively associated with a Graphical User Interface application for use by the user; a server means for providing the user access to a virtual training room comprising at least one page of a Graphical User Interface, wherein the server means comprises: a Web server for presenting the user with pages for display on the Graphical User Interface, an information repository for storing information content, and an Education Interface Engine adapted for managing communication between the client device and the server means.

4. Apparatus as claimed in claim 3 wherein the Education Interface Engine is adapted to generate a quiz for the user as a sequence of dynamically created pages of a Graphical User interface.

5. Apparatus as claimed in claim 4 wherein the dynamically created pages comprise interactive multimedia content.

6. Apparatus as claimed in any one of claims 3 to 5 wherein at least one of the Web server, the information repository, the Education Interface Engine and the client device is adapted to provide asynchronous communication.

7. Apparatus as claimed in claim 6 wherein the dynamically created pages are provided by asynchronous communication between the client device and the server means.

8. Apparatus as claimed in claim 6 or 7 wherein the asynchronous communication comprises Asynchronous JavaScript and XML (AJAX).

9. Apparatus as claimed in any one of claims 3 to 8 wherein the server means is adapted to provide validation for a user in which the validation comprises one or any combination of: closed captioned subtitles to video for hearing impaired users; audio playback to text-based content for users with low literacy levels; audio feedback for user interface controls; multimedia reviews of quiz questions and answers to assist users with learning difficulties; video and animation reviews after questions and answers to provide relevance reinforcement.

10. Apparatus as claimed in any one of claims 3 to 9 wherein the Education Interface Engine comprises a Message Broker adapted to combine data and/or functions from a plurality of sources and scripting libraries for facilitating asynchronous communication between one or more of: the client device; the Web server; the information repository

11. Apparatus as claimed in claim 10 where the Message Broker is further adapted to perform message aggregation wherein synchronous messages are intercepted from one or more of the Web server and the information repository, aggregated and then transferred asynchronously to external libraries.

12. Apparatus as claimed in claim 10 or 11 wherein the Message Broker comprises one or any combination of:

Microsoft ASP .NET AJAX;

The Prototype JavaScript Framework;

JSON (JavaScript Object Notation); Microsoft .NET Framework 2.0.

13. Apparatus as claimed in any one of claims 3 to 12 wherein the Education Interface Engine further comprises a dynamic data switch which comprises a class library adapted for referencing a .NET Framework data provider class library.

14. Apparatus as claimed in claim 13 wherein the dynamic data switch is adapted to communicate with the information repository by executing SQL queries.

15. A computer implemented method of educating a user, the method comprising the steps of: generating a Data Store comprising at least one user selectable module corresponding to a field of knowledge; compiling a quiz comprising at least one computer displayable page of questions relating to the user selected module; asynchronously communicating the at least one computer displayable page to the user to enable the user to complete the quiz.

16. A method as claimed in claim 15 wherein the step of compiling comprises asynchronousiy communicating with the Data Store.

17. A method as claimed in claim 3 or 16 further comprising the step of: randomising the questions of the quiz.

18. A method as claimed in claim 17 wherein the step of randomising is performed during one of the steps of: compiling a quiz, and; asynchronousiy communicating to the user.

19. A method as claimed in any one of claims 3 to 18 further comprising the step of: asynchronousiy communicating user messages from a user's client device to a server adapted to perform at least one of the steps of any one of claims 3 to 18.

20. A method as claimed in any one of claims 3 to 19 wherein the steps of asynchronousiy communicating comprise a plurality of message transfers correlated with a single acknowledgement.

21. A method as claimed in claim 19 or 20 wherein the at least one computer displayable page and the user messages comprise interactive multimedia content.

22. A method as claimed in claim 21 wherein the interactive multimedia content is adapted for one or any combination of: text; video; animation; closed captioning for hearing impaired users; audio comprising speech for low literacy skilled users.

23. A method as claimed in any one of claims 3 to 22 further comprising the step of storing one or any combination of questions, quizzes, and quiz results at the Data Store.

24. A method as claimed in any one of claims 3 to 23 wherein the Data Store comprises a relational database.

25. Apparatus for educating a user comprising: an information repository for generating a Data Store comprising at least one user selectable module corresponding to a field of knowledge; an Education Interface Engine for compiling a quiz comprising at least one computer displayable page of questions relating to the user selected module; web server means for asynchronously communicating the at least one computer displayable page to the user to enable the user to complete the quiz.

26. Apparatus as claimed in claim 25 wherein the Education Interface Engine is adapted to asynchronously communicate with the Data Store.

27. Apparatus as claimed in claim 25 or 26 wherein the Education Interface Engine further comprises a Random Generator for randomising the questions of the quiz.

28. Apparatus as claimed in claim 27 wherein the Random Generator is adapted to perform a randomisation during one of: compiling a quiz, and; asynchronously communicating to the user.

29. Apparatus as claimed in any one of claims 25 to 28 further comprising a client device for displaying the at least one computer displayable page to the user x and wherein the client device is adapted to asynchronously communicate user messages to the Web server.

30. Apparatus as claimed in any one of claims 25 to 29 wherein information repository, the Education Interface Engine, the Web server and the client device are adapted for asynchronously communicating with each other wherein the asynchronous communication comprises a plurality of message transfers correlated with a single acknowledgement.

31. Apparatus as claimed in claim 29 or 30 wherein the at least one computer displayable page and the user messages comprise interactive multimedia content.

32. Apparatus as claimed in claim 31 wherein the interactive multimedia content is adapted for one or any combination of: text; video; animation; closed captioning for hearing impaired users; audio comprising speech for low literacy skilled users.

33. Apparatus as claimed in any one of claims 25 to 32 wherein the information repository is adapted to store one or a combination of questions, quizzes, and quiz results at the Data Store.

34. Apparatus as claimed in any one of claims 25 to 33 wherein the Data Store comprises a relational database.

35. A system for educating a user comprising apparatus as claimed in any one of claims 3 to 14 or 25 to 34.

36. A computer implemented method of randomising a quiz suitable for use in testing a user's knowledge and skills, the method comprising the steps of: determining a first set of questions associated with a user configuration relating to a field of knowledge; determining a second random subset of questions from the first set of questions based on predetermined rules associated with the user configuration; for each question determining a random set of answers based on predetermined rules associated with the context of the question; generating the quiz such that each question of the second random subset is collated with its respective random answers.

37. A method as claimed in claim 36 wherein the predetermined rules are based on configurable criteria comprising one or any combination of: the quiz name and description; the total number of possible questions; any mandatory questions; the number of questions required per quiz; the number of question types; the number of compulsory question for a given quiz; the answers to be available for each question, and in the case of multiple choice questions, when the answers are to be shuffled, overriding criteria to position an answer in this ordering (e.g. "All of the above" to be put last)

38. A method as claimed in claim 36 or 37 further comprising the step of: storing the generated quiz in a quiz repository.

39. A method as claimed in any one of claims 36 to 38 wherein questions and quizzes are stored in a relational database.

40. Apparatus for randomising a quiz suitable for use in testing a user's knowledge and skills, the apparatus comprising: first determining means for determining a first set of questions associated with a user configuration relating to a field of knowledge; second determining means for determining a second random subset of questions from the first set of questions based on predetermined rules associated with the user configuration; third determining means for determining, for each question, a random set of answers based on predetermined rules associated with the context of the question; quiz generating means for generating the quiz such that each question of the second random subset is collated with its respective random answers.

41. Apparatus as claimed in claim 40 wherein the predetermined rules are based on configurable criteria comprising one or any combination of: the quiz name and description; the number of questions; any mandatory questions; the number of questions required per quiz; the number of question types; the number of compulsory question for a given quiz; the number of properties/attributes used to filter out relevant questions; the answers to be available for each question and, in the case of multiple choice questions, when the answers are to be shuffled, overriding criteria to position an answer in this ordering (e.g. "All of the above" to be put last)

42. Apparatus as claimed in claim 40 or 41 further comprising: storage means for storing the generated quiz in a quiz repository.

43. Apparatus as claimed in any one of claims 40 to 42 further comprising a relational database for storing questions and quizzes.

44. Apparatus adapted to educate a user, said apparatus comprising: processor means adapted to operate in accordance with a predetermined instruction set, said apparatus, in conjunction with said instruction set, being adapted to perform the method as claimed in any one of claims 1, 2, 15 to 24 and 36 to 39.

45. A computer program product comprising: a computer usable medium having computer readable program code and computer readable system code embodied on said medium for educating a user within a data processing system, said computer program product comprising: computer readable code within said computer usable medium for performing the method steps of any one of claims 1 , 2, 15 to 24 and 36 to 39.

46. A method or protocol as herein disclosed.

47. An apparatus, system and/or device as herein disclosed.