WO2001088878A2 - Arrangement for teaching - Google Patents

Abstract

A teaching aid is provided, which includes a question and answer source (102), means for asking groups of questions on a plurality of occasions, means (110) for determining whether an answer is correct, means (100) for increasing the interval between successive cycles of groups of questions asked in case of correct answer (110), means for reducing at least one interval between cycles of a group of questions asked in incorrect response (110). The invention also relates to a teaching method.

Description

ARRANGEMENT FOR TEACHING

The present invention relates to an arrangement for teaching. The invention also relates to a corresponding method.

The aim of most teaching techniques is the transfer of information to long term memory. When something is learned, call it an item, it does not normally pass straight into the long term memory. The processes which occur are still the subject of research but, in simplistic terms, it can be said that after it has first been learned the item passes into short term memory, which may last at most a few minutes. Unless the item is actively recalled from memory the process of transferring the item to medium term memory (when the item can be successfully recalled for perhaps a few days) and to long term memory (when the item becomes permanently available for recall) will probably not take place. The item can be actively recalled from memory by testing.

It has been demonstrated that the process of recalling an item from memory (for example by testing) is much more effective at enabling the transfer of the item to long term memory than reading or listening to the item in a repetitive fashion without actively testing the memory. Regular repetition of recall for a number of times, however, does not effectively utilize the time and effort of a student as will be discussed hereinafter.

US Patent 4, 193, 210 describes a system which concentrates on those facts which the user is having greater difficulty with. The problem with the system disclosed therein is that, when a student makes an error, a fixed "priority presentations" routine is implemented. This treates all errors in the same manner and thus seriously overreacts to a momentary lapse in concentration by the user. Because of this, use of the system can be expected to lead to boredom on the part of the user and inefficient use of learning time. It is an object of the present invention to provide an improved arrangement and corresponding method for teaching.

According to a first aspect of the present invention, there is provided a teaching arrangement as defined in appended Claim 1.

According to a second aspect of the present invention, there is provided a teaching arrangement as defined in appended Claim 16.

According to a third aspect of the present invention, there is provided a method as defined in appended Claim 17.

In the context of the present invention, the term "teaching" includes the imparting of knowledge, assessment of individuals on the basis of what is learned and examination revision.

Further preferred features of the invention are set out in the appended dependent claims.

The present invention is based on a realisation that certain items in any group of items to be learned will be more readily remembered than other items.

The present invention will now be explained and described with reference to the accompanying drawings, in which:

Figure 1 shows a schematic curve of memory strength and a curve of probability with which a particular item can be recalled,

Figure 2 shows a schematic memory curve for an item that has been learned and then recalled,

Figure 3 shows a schematic memory curve as shown in Figure 2, showing the relative strength of memory after both initial learning and recall,

Figure 4 is a block schematic diagram of an arrangement in accordance with the present invention, Figure 5 is a flow chart for explaining the operation of the arrangement shown in

Figure 4,

Figure 6 shows three diagrams for explaining the operation of the present invention,

Figure 7 shows a table relating to another embodiment of the present invention, Figure 8 is flow chart for explaining the operation of this another embodiment, Figure 9 shows a student's profile table for use with this another embodiment, and

Figure 10 shows a diagram explaining the operation of another embodiment of the invention.

After a new item is learned the strength of the memory trace gradually diminishes. Even though the memory trace diminishes the probability that a test of the item will lead to successful recall may remain high, although the individual may have to search harder to remember the answer. After a while the memory trace will have diminished to the point where, no matter how hard the individual searches for the answer, it cannot be recalled. Figure 1 demonstrates the fall in strength of the memory trace and probability of successful recall but is intended to provide a schematic representation of the relevant probabilities.

As already noted, experimental evidence indicates that recalling an item from memory strengthens the human memory trace for the item.

Figure 2 illustrates how recall strengthens the memory trace. Improvement is obtained is recall occurs before the probability of recall (Figure 1) deteriorates to the point at which the student cannot actually remember the item. The number of times an item is recalled is not the only factor which affects whether an item will pass into long term memory. The length of time which passes between each recall is also important. Thus recalling an item once a minute for 10 consecutive minutes is not as effective in transferring the item into long term memory as recalling the item once an hour for 10 consecutive hours (provided the item can be recalled after these intervals). Using a pattern of expanding the length of time which passes between each recall - e.g. one hour, two hours, four hours, etc. - is more effective at transferring an item into long term memory than a pattern of recalling the item after a period of time which does not gradually increase. This effectively exploits the curve shown in Figure 3.

However, releaming an item after it has been forgotten (or at least after it can no longer be recalled) can take significantly longer than the time spent recalling an item just before it is forgotten. Hence, the time spent learning the item sufficiently well to transfer it to long term memory is minimised if the memory is tested after increasing periods of time but taking care to test the memory before the item is actually forgotten.

In the absence of direct means for measuring the memory traces shown in Figures 1, 2 and 3, the problem which arises is how to trigger recall reliably close to the point at which an item can no longer be recalled. The problem is exacerbated by the fact that certain items will be more readily learned than others.

When more than one item is being learned - in this example, when more than one item of foreign language vocabulary is being learned - the object is triggering of the testing of each individual item at the appropriate point. Thus, if 20 items of foreign language vocabulary are being learned, the aim is to monitor and trigger the testing of each of those 20 items of vocabulary at the appropriate point. There are thus conflicting demands to be satisfied. If the interval between testing of a particular question or class of questions is too great or too small then learning efficiency will deteriorate. A class of questions will comprise differing questions that test the same basic concept. Examples are 2 questions with the same answer or, in a mathematical context, a number of questions from the seven-times-table and so on.

When a number of items of vocabulary (for example) are being learned it is inevitable that some will be learned less well than others for various reasons. For example, there may have been a momentary lack of concentration when a particular item was being learned. If an attempt is made to ensure that testing takes place at a point when there is a 100% probability that every item will be successfully tested, this will result in items having to be tested within very short spans of time in order that the least well learned item can be recalled when tested. This approach may have the advantage of strengthening the memory trace of items which have been poorly learned but, overall, may be inefficient if other items are tested well before they are about to be forgotten. There is thus over-emphasis on those items which are weak in the student's memory.

The present inventor has realised that greater efficiency may be achieved if it is accepted that some items will fail the test and have to be relearned. In response to an incorrect answer, the item that the student is finding difficult is emphasised. Thus the point at which testing takes place may be set such that some items which have not been properly learned are filtered out by failure on the test (and relearned) whilst the other items are successfully tested just before they are forgotten (see next paragraph). This leads to the desired success rate of answers to questions. The 'success rate' is a parameter used in the following description and should not be confused with the ultimate ability of a student using the invention or with a proportion of students that will benefit therefrom.

One compromise which has been found to be successful is to test items at a point when the probability of successful testing (discussed further below) is, say, 80%. The 20%> of the most poorly remembered items which have been learned least well are filtered out by failure and relearned. Thus, the arrangement reverts to re-testing this poorly learned item at shorter intervals. In this example, 80% is the desired "success rate" but any value may be chosen; the best results having been achieved using values of between 70 and 90%). However, the values may vary widely for particularly difficult or easy topics, for those with learning difficulties and so on.

To say that an item is tested just before it is forgotten, then, is to say that testing takes place, for example, at a point when there is, given its membership of a class of items, an 80% probability that the item will be successfully recalled. The system will make the answers more difficult to remember for the student (by spacing the questions further apart) until approximately this success rate is achieved. The approximation is required since the questions and answers are integer entities and thus defy exact adjustment of probability on a question-to-question basis. Previously, the usual paradigm under which students operate is that a test is applied after learning and the test is then marked to reveal what the success rate was. In some cases there may be no opportunity to revise the items of the test which were failed much less to provide an immediate or almost immediate re-learning of the item or items that are causing particular difficulty. The present invention enables this relatively poor approach to be improved because capabilities are tested at such a point as enables a predetermined success rate to be approximately achieved.

The operation of the invention depends on feedback on whether tests are being passed or failed, in other words, a current success rate. The invention can then adjust the point at which tests are presented so that the success rate is substantially equal to the desired success rate.

The invention has a mechanism by which feedback from a test (i.e. whether it was answered correctly or incorrectly) is used in conjunction with the desired success rate to vary the point at which testing is applied. Multiple tests of the same piece of information are applied.

It is usually required to test an item several times in order that it can be remembered. Indeed the preceding discussion has been predicated on the hypothesis that an item needs to be recalled several times as part of the process of transferring it to long term memory (in spite of introductory comments which, in setting the background to the invention, indicated that once recalled an item might be remembered for several days). Four or five repetitions have been found to be suitable for many circumstances but more repetitions are likely to be required for particularly difficult material or for those with learning difficulties.

The invention allows the number of times an individual item is to be tested to be predetermined for a particular group of questions by the user.

For each item being learned and tested, the invention estimates the point at which the nth test of the item should be triggered (n < the number of tests of the item) in order that the probability of success is equal to the desired success rate. While a point in time is referred to, this may not be a strict application of a time delay. It could, for example, be determined in terms of a number of intervening questions and answers.

If, for example, the item is to be tested three times this can be done in several ways. It may simply be tested three times (even if incorrectly answered on each occasion), or the correct answer may have to be given three times, with perhaps some incorrect answers as well) or it may have to be answered correctly on three consecutive occasions (so that if an incorrect answer is given the score of correct answers for that item is reduced to nil and the sequence of testing begins afresh). Only a computer can do the computations faster than the user (not another person).

In the following description of a first embodiment of the invention, it is required that the item be successfully tested on n consecutive occasions.

Suppose that tests of a specific item are carried out after a period of time Ti and again after a further period of time T₂. Suppose that Ti is determined so that the probability of successful test is p and that T₂ is determined so that the probability of successful testing is also p. Then, in the case of human memory, experiments indicate that T₂ is greater than Tt.

Thus it would not be appropriate, in the case of human memory, to apply each test of the item after a fixed amount of time or activity since Ti and T₂ are not equal and the process would thus be inherently inefficient.

The probability of successful testing after a period of time or activity (p) may vary with time or activity. For example, as a period of learning activity proceeds the capacity of the human memory to retain information may diminish, i.e. the individual gets tired. This would lead to corresponding reductions in the achieved^ for a given delay interval Ti. Hence, ifp is to be maintained, then the length of Ti will need to be shortened as a period of learning progresses. (It may also be appropriate to interrupt the period of learning to give the user a break). In the present invention, as will be described, the delay is continuously updated. In other words the length of Ti used to trigger the testing of an item for the ith time. The selected success ratio is then achieved and the student learns effectively.

We now consider the calculation of the time which elapses before an item is tested.

Consider the example of a person learning 20 items of foreign language vocabulary, p may be set at, say, 80%. That is, having learned an item of vocabulary, it is desired that on each occasion of testing the probability of success is 80%. Once an item has been successfully tested on, say, four occasions it is considered to have been learned (although it may need to be revised after a further period of time, say one day). If an item fails it is relearned and testing begins afresh. The successful testing may be applied to consecutive or absolute counts of correct answers.

Ti(l < i < 4) needs to be determined such that the 80% success rate is achieved. However, Tj cannot be predicted in advance. The difficulty of the material being learned, the existing familiarity of the learner with the material, the innate capacity of the learner to retain information and the general well being of the learner are some factors which may influence the probability that the information will be retained after time Tj for the i'th test of the item.

Tj therefore needs to be continuously monitored and adjusted as the learning session proceeds. The adjustments to Tj will depend upon feedback from the results of each test - i.e. an increase in Tj when a test is successfully passed and a reduction when a test is failed (assuming that the probability of success decreases as Tj increases).

Of particular interest is an iterative approach to estimating T;. The amount by which T; is increased or reduced according to the desired success rate will depend on the amount of information known about the relationship between Tj and^?.

It is possible that in a particular situation the relationship is a step function, so thatp is either 100% or 0%; the unknown information being the length of T; at which/? changes from 100% to 0%. In another situation it may be known that/? is inversely proportional to T; the unknown information being the length of Tj at which p is equal to the desired success rate. In another situation, the relationship may not be known at all. The relationship between Tj and in animals or algorithms in machines which are designed to learn information may also be different to that, generally, for humans. This means that the relationship between ? and Tj cannot be described in all circumstances.

An iterative approach is therefore taken to identifying the length of T; which results in p. This involves increasing or decreasing the length of Tj according to whether the actual success rate is greater than or less than the desired success rate. The rate may take some time (in other words a number of questions) to be determined. A simple increase in the length of T; by one unit of time or activity (question and answer combination) following a successful test and a decrease of one unit following a failure would not take into account the desired success rate at all.

However, it is possible to construct a method for varying Tj which takes into account p. Suppose that the length of T; is increased by r when a test is successfully passed and reduced by w when a test is failed. If the desired success rate is p then the formula pr = (l-p)w could be used to link/?, r and w. Thus, once s and r have been determined w can be calculated.

The validity of the formula is seen by assuming that the estimate of Tj [E(T ] is, in fact, the correct length of Tj to give the desired success rate/?. Suppose that a question is successfully answered. The iterative approach means that the length of Tj will be increased by r and, subject to the comments below, if the proportion of times this happens is p, then the total increase in length of Tj will be pr. The proportion of incorrect answers, subject to the comments below, will be (1-p) and the reduction in length of Tj will be (l-p)w. If the reduction in length of Tj is to compensate the increases in length of Ti thenpr = (l-p)w.

Overall, if E(Tj) is the correct length to give the required success rate/? then the expected net movement, m, in length of Tj over time m is zero and is given by the formula m =pr-(l-p)w = 0. Suppose that E(Tj) is longer than necessary to give the required success rate/?. The probability of success is less than/? because too much time elapsed before a question is retested - i.e. (T;) is too long - and may be called/?'. Suppose that/?' =p - d. Then it follows that m'= (p-d)r - (l-p+d)w =pr - (l-p)w - d(r+w) = -d (r+w) < 0. Thus, if E(Tj) is longer than necessary to achieve the desired success rate/?, the expected change in Tj, m', is a reduction in length. Similarly, if E(Tj) is too short then d will be negative and the expected change in length of Tj, m, will be positive. Thus the link between/?, r and w results in an expected change in the length of Tj - E(T;) - towards the optimum length.

Furthermore, as the magnitude of d increases E(Tj) the expected compensation necessary to bring the length of Ti back to the appropriate length for the desired success rate p will also increase.

Thus, if E(Tj) is a highly inaccurate estimate of the appropriate length for T given a desired success rate ?, E(T;) is greater than that where E(Tj) is a reasonably accurate estimate of Tj.

The expected change in length of Ti is also dependent on the magnitude of r and w. When the length of T; is not known it might be appropriate to use a length for w which is large so that E(Tj) quickly changes to the most appropriate length. The disadvantage of using a large w is than when E(Tj) approximately equals Tj, correct and incorrect answers will result in large oscillations about Ti which will result in variability in the actual success rate which is achieved. Conversely, if w is small, then to start with, it will take longer for the iterative process to change the length of Tj to the appropriate length.

The following example gives the order in which a set of items may be learned and recalled. In other words selection of the next question. Let the items to be learned be labelled Li, L₂, L₃ and so on. Let the number of consecutive occasions on which the item must be successfully tested before testing of that item ceases be N.

Consider the testing of item Lj. The method proceeds in cycles, testing one or more items in each cycle. For example: in cycle zero item Li might be presented for learning in cycle one item L₂ might be presented for learning and item Li might be tested for the first time in cycle two item L₃ might be presented for learning and item L₂ might be tested for the first time in cycle three item L might be presented for learning and item L₃ might be tested for the first time and item Li might be tested for the second time in cycle four item L₅ might be presented for learning and item L might be tested for the first time and item L₂ might be tested for the second time

Let the number of cycles which pass between learning an item and being tested on it for the first time be Ti. Let the number of cycles which pass between being tested on an item for the nth time and the (n+l)th time be T_n+ι. In the previous example Ti is one and T₂ is two. However, the delay is not a strict temporal one because Ti takes as long as the time taken to present L₂ for learning and T₂ takes as long as the time taken to present L₃ and L and test L₂ and L₃.

Figure 4 shows a block schematic diagram of an embodiment of the present invention. A Central Processing Unit (CPU) 100 is provided with a number of ancillary devices. The operation of the CPU is described with reference to the flow chart shown in Figure 5. A question database 102 may comprise a list of questions and answers stored in any suitable format on semiconductor memory, magnetic or optical disc and so on. In the context of a teaching arrangement for mathematics it may comprise one or more formulae for posing the questions (the CPU can readily calculate the answers). A Read Only Memory (ROM) 104 stores software for operation of the CPU and a Random Access Memory (RAM) 106 stores parameters required during program operation as is well known to those skilled in the computing arts. A user input 108 may comprise a keyboard or other suitable input device and a display 110 may likewise comprise any suitable means of communicating the questions to the student. A user profile 112 is an optional feature which will be discussed further hereinafter. In brief, the user profile produced by the invention in a form which can be stored in any suitable format on semiconductor memory, magnetic or optical disc and so on. It is an indication of how well the student has performed in the past (on the current learning cycle or on previous learning cycles). This can be used by the arrangement to set initial parameters such as intervals between successive questions to more quickly achieve the desired probability of success, even for material which is new to the student. Alternatively, it may be printed out for reference purposes.

Test Results, Learn Results and Learn Statistics are further outputs from the invention. These may also be stored on any suitable format on semi-conductor memory, magnetic or optical disc and so on. These outputs are discussed further hereinafter.

Figure 5 shows a flowchart with explanatory annotations and indicates the response of the arrangement to correct and incorrect answers to questions. The operations performed are clear from the chart. A correct answer lengthens the interval (or gap) between successive posing of the same question (for several or all items currently being learned). The interval between successive posing of the same question (for all items currently being learned) can be shortened (step SI 18). The important feature of the present embodiment, however, is that the question which has been answered incorrectly is reinserted in the sequence of questions or is arranged to be asked with greater frequency in future. The question can be re-posed immediately, left for an interval to allow the student to recover from failure to answer or re-located at the end of a question queue. The example in the flow chart reduces the interval for all questions and re-inserts the question relating to the failed item of information in the question sequence immediately (steps SI 18 and S120). As a further option (not shown) an item that was answered incorrectly at the last attempt is shown to the student in a teaching (as opposed to testing) step prior to it being asked again.

There follows an annotated pseudocode explanation of the software for selecting the intervals and questions in order to obtain the desired success rate.

The Questions

Take nq questions and answers which are to be learned and/or tested.

Number these as q; 1 < I < nq Define 1 < I < nq as "not blank" Define φ I < 1 as "blank" Define qi I > nq as "blank"

Comments: blank questions will not be presented to be learned or tested.

Method of Tackling the Questions

L = 1 if, prior to testing, the question and answer is to be presented to the user

(learn) to be learned.

L = 0 if the questions are not to be presented to the user to be learned.

RL = 1 if, having answered a question incorrectly, it is to be relearned/tested at

(relearn) the next available opportunity.

RL = 0 if, having answered a question incorrectly, it is to be put to end of list of questions. C = a user-selected number used by "CFS" (see next).

CFS = 1 if the question must be correctly answered

(criterion C consecutive times before it stops being presented for success) = 2 if the question must be correctly answered

C times before it stops being presented = 3 if the question must be asked C times whether or not correctly answered.

Intervals (Gaps)

Gapi is the number of iterations of the "Routine" (see later) between the question being learned and the question being presented for the ith time for questioning, if the question is always correctly answered.

If the user is not given an opportunity to learn the question first gap; remains constantly at zero and qi will first be tested on the ith iteration of the routine. Gap; j > 1, is then the number of iterations of the routine between the questions being asked for the first time and the jth time. If, for example, gap₂ = 1 the question will be asked for the second time in the iteration immediately following the iteration in which the question was successfully asked for the first time.

At the start:

• Set Gap_j, = 0

Set Gap;, 2 <j < c at any positive value subject to the rule that If I_m = Gap_m rounded to nearest integer; then I_m must be < I_m+ι

• Set ca; = 0 0 < i < nq caj counts the number of times the question has been asked or answered (according to the criterion set by the value of cfs).

• Set k = l k counts the number of iterations of the routine.

Desired Success Rate

Set r_f 1 < I < c rj is the amount by which Gapi is increased if a question is correctly answered.

Set Sj 1 < I < c subject to 0 < Si < 1

Si is the desired proportion of successfully answered questions for Gapi.

Calculate Wi = rι_Si

Alternatively Set w; 1 < I < C Wi is the amount by which Gap, is reduced if a question is incorrectly answered.

Calculate r; = Wj (1 - Si)

Si

The Routine

Routine followed on the kth iteration 1. If k < nq and L = 1 then present the question to be learned by the user.

2. For j = 1 to c carry out the following sub-routine

2.1 let t = Gap; rounded to nearest integer

2.2 if Q_k-_t is not a blank question; and

If ca_k-_t <j; then Test Qk-t

2.2.1 If Qk-t is correctly answered; then a) let cak-t = ca^t + 1 b) if j = 1 and L = 1 ; then let gapi = gapi + n c) if j > 1 let gap,- = gap_j + r d) for m = 1 to c - 1 let Im = gap_m rounded to nearest integer ifl_m > l_m+ι; then let Gap_m+ι = Gap_m.

2.2.2 If Q_k-_t is wrongly answered; and RL = 1 ; then a) flag Qk-t as a blank question b) for all non-blank questions Q_p where p > k let Q_p+ι = Q_p c) let Q_k+i = Qk-t, flagged non-blank If Q_k-_t is wrongly answered and RL = o; a) flag Qk-t as a blank question b) let Q_nq+ i = Qk-t

c) let nq = nq + 1

2.2.3 If Q_k-_t is wrongly answered a) let gap_j = gapj - w unless, as a result, I_j would be I_j._! or gap_j would be < 0. Figure 6 shows a visual representation of the principles of a straightforward embodiment of the present invention. Figure 6(a) shows a QUESTION LINE having a number of questions from 1 to 14 and so on up to a desired number of questions (not shown). Time is assumed to progress from left to right, in other words the questions are asked for the first time in number order. Above the question line is a question selecting line having four question choosing branches Cl, C2, C3 and C4 and three variable length gaps Gl, G2 and G3. The position of the question selecting line indicates that a cycle of questions is being asked. Question 11 is being asked for the first time (Cl), question 7 is being asked for the second time(C2) and question3 is being asked for the third time (C3). No questions have yet been asked for the fourth (and final) time and this is indicated by choosing branch C4 being located at a time before question 1. The zig-zag sections of the question selecting line indicate that the gaps may vary in length. If the questions are answered correctly, the selection line will progress rightwards and the gaps G will increase slightly as described previously.

Figure 6(b), however indicates what would be expected to happen after a question is incorrectly answered. Question 7 is assumed to have been incorrectly answered on the second attempt. Question 7 is then marked so as not to be asked at its original location and is re-inserted into the question line before question 12 (in other words it will be re- asked almost immediately). In addition, the gaps G are reduced since the interval between questions is deemed to be too long for the student (since a mistake was made). However, this creates a slight problem which can be seen from a comparison between figure 6(a) and 6(b). If the branch Cl is moved to the new location of question 7 (figure 6(b)) then the second occurrence of question 8 (C2) and the third occurrence of questions 4 and 5 (C3) will be missed. This is avoided as shown in figure 6(c).

Figure 6(c) shows the question selecting line and the question line as they were in figure 6(b), in other words the location of the questions and the gaps are the same. However, the question selecting line has been moved to the left relative to the question line so that the earliest question (on any question cycle C2, C3 etc.) will be asked. Question 7 will not be asked as a result of the branch C2 because it has been marked as invalid. Question 7 will next be asked as a consequence of branch Cl selecting it from its new position. A separate note is made to ensure that questions 10 and 11 (and any others that have already been asked and answered correctly) will not be selected as a result of branch Cl (or any other branch).

Later in the cycles of the learning arrangement, the branch Cl will move beyond the last question (not shown) and so only questions on a second or subsequent presentation will be asked. Also not shown are questions asked as a consequence of the final branch C4 but the principle is analogous to that for the other branches. As many branches can be provided as desired. As described above, the learning session can end when the student has answered each question, correctly for a number of times, when each question has been posed for a given number of times, when the student wants to finish and so on.

A further embodiment of the invention is described with reference to figures 7,8 and 9. Figure 7 shows a table of questions stored, for example, in a spreadsheet-style format. Each question is provided with a number of parameters as follows:

1) Consecutive correct answers - when a student has answered a question correctly for a predetermined number of times, the question is no longer asked (or may be intended for asking the next day or such like). This attribute maintains a record of the number of times each question is correctly answered so that the question can be made inactive when the threshold is reached.

2) Time of Last Asking - is a real-time record of the time that the question was last put to the student.

3) Interval until next asking -is determined according to the speed of the student and how many times the question has been asked as described in Figure 8.

4) Time of next asking - is simply an addition of the parameter 3) to the parameter 2).

5) Time now - can simply be a clock, for example already provided in personal computers, it is shown in the table for descriptive purposes only. 6) If time now exceeds time of next asking then place a ' 1 ' in this column.

The program proceeds downwards along this column and asks the relevant first question flagged with a 1. The table is then updated according to the student's success and the advancement of time. The operation is described in greater detail in figure 8.

Figure 9 shows a table which the arrangement generates optionally for each particular student. The GAP number is the interval between successive askings of the same question (G in figure 6). The Time From Last Asking is the value of the gaps for a particular student. The success required is the probability and the Increase in gap are the values r previously discussed. This table may be stored to give an initial set of gap lengths for a particular student. If a standard set of gaps were used initially the student would likely make lots of errors (if the gaps were too long) or find the material too easy and so waste learning time (if the gaps were too short). By providing a custom 'profile' table as shown in figure 9, a student who has previously used the arrangement can benefit more quickly when using it subsequently.

In a further embodiment the invention produces a series of Subsets of questions drawn from a larger Set of questions. Each question within the larger Set of items will be presented on a plurality of occasions as a result of being included in more than one of the Subsets.

The situation in which each item is to be drawn only once from the Set is not dealt with. The solution to this is to place the items in an appropriate order and then to start with the first item in the set and continue with succeeding items until all items have been presented

As a result of taking the items specified by each Subset in turn, a sequence is produced that specifies the items to be presented and has the following characteristics.

The second item identified by the sequence is presented immediately after the first item has been presented. The third item identified by the sequence is presented immediately after the second identified item. The fourth item identified by the sequence is presented immediately after the third identified item and so on. In other words, no timer is employed to determine the time at which an item should be presented.

1. In general the number of other items presented between the first and second presentations of a particular item is less than the number of other items presented between the second and third presentations of an item and so on. Thus, an item frequently in the series once it first begins to feature in the series but its frequency of occurrence gradually diminishes after that.

2. If the presentation of items is to be assessed as either successful or unsuccessful, and a desired "success rate" is specified, the invention is capable of dynamically adjusting the sequence of items to be presented in order to try to achieve the desired Success Rate.

As well as successful or unsuccessful, assessments could be made that the presentation is right or wrong, or correct or incorrect, or 1 or 0 or some other binary assessment measure could be employed.

The "Success Rate" is defined as the proportion of successful presentations to total presentations, using successful and unsuccessful presentations as the measure. The success rate is a measure of the proportion of answers that the user is allowed to get right. The gaps between successive repeats of a question will be altered until the user makes the desired proportion of errors. Generally speaking, if a user gets a high proportion of answers right (i.e. higher than the dersired probability) the gaps will be extended. A higher probability value will generally result in smaller gaps because the user will forget more material (reducing the probability) as the gaps increase. A lower probability value will generally result in a "harder" test because the system is trying to obtain a higher failiure rate from the user by using larger gaps. The success rate is prefearbly set by the user.

Figure 10 provides an illustration of the invention. Items in the Set have been numbered with consecutive integers, and Figure 10a) shows these items placed on a number line extending from 31 to 54. Illustrations b), c), d) and e) show what is here referred to as a "Claw" that gradually moves along the number line, one integer at a time.

The Claw contains five "Hooks". The Claw may contain a different number of Hooks to 5. Illustration b) indicates that the first Hook, at the far right of the Claw, is pointing at Item 50, the second Hook is pointing at Item 49, the third at Item 46, the fourth at Item 42 and the fifth at Item 35.

Hook 1 has a characteristic, called Gapi, equal to zero. This indicates that the integer distance of the hook from the right hand side of the claw is zero. Hook 2 has characteristic Gap 2 equal to 1 because its integer distance from the right side of the Claw is 1. The Gap 3 characteristic for Hook 3 (relative to Hooke 1) is 4, the Gap 4 characteristic for Hook 4 is 8 and the Gap 5 characteristic for Hook 5 is 15. Whether the gap characteristics for the Hooks may be different to those indicated in this example is not key to the invention.

The position of the claw in Figure 10b) defines a subset of items, namely, {50, 49, 46, 42, 35}. Illustrations c), d) and e) define similar subsets. When items defined by these subsets are placed together, the following sequence is obtained: {50, 49, 46, 42, 35, 51, 50, 47, 43, 36, 52, 51, 48, 44, 37, 53, 52, 49, 45, 38}.

Examination of this sequence shows that Item 53 is chosen once, Items, 52, 51, 50 and 49 are chosen twice and items between 35 and 48 inclusive are chosen either once or not at all.

In general, the number of other items presented between the first and second presentations of a particular item are less than the number of other items presented between the second and third presentations of an item and so on.

When joining together Subsets of Items to generate a sequence, it is not vital that, within a particular Subset, the Item with the highest number is presented before an Item with a lower number, as illustrated above. Whatever the order in which items within a Subset are presented, the decreasing frequency of presentation of an item will be achieved so long as the Subset defined by Figure 1, Illustration b) is presented before the Subset defined by Figure 1, Illustration c) and so on (in this example).

Failure of an Item that is presented.

Suppose during its presentation an item is assessed in some way. For example, a fact that has been learned may be tested. If the test is failed it may be concluded that too many other items were presented in between the Item last being presented and the presentation in which the test is failed. The way to deal with this is to reduce the gap characteristic for the relevant Hook by one, moving the Hook closer to the right of the Claw by one integer. From that point, Items will be presented more quickly than they would otherwise have been so far as that particular Hook is concerned. Whether the gap is reduced by one or some other number is not key to the invention.

A related issue arises. Take Figure 1, Illustration a) as an example and consider Hook 3, which points at Item 46. Suppose Item 46 fails its test. Hook 3 is moved one integer to the right and then points at Item 47. However, the entire Claw is then moved one integer to the right so that Hook 3 selects Item 48 for the next Subset, and not Item 47. Item 47 will never be selected by Hook 3, which leads to an imbalance in the treatment of Item 47 compared to other Items. It might be assumed that the failure of Item 46 suggests that Item 47 is also at risk of being failed. One way to deal with this is to present Item 47 at the time that Hook 3 is moved to point to it, but before the Claw is moved one integer to the right. Whether this or some other way of dealing with the potential anomaly is chosen is not key to the invention.

A second related issue arises. If, in this example, Hook 3 is moved to the right then the number of other Items that are presented between a particular Item being presented for the third and fourth time will now increase. By altering only the position of the relevant Hook relative to Hook 1 the system tailors itself to the user's memory curve. Alternatively, Hook 4 and possibly Hook 5 might also be moved to the right when Hook 3 is moved. A third related issue arises. Consider Figure 10b). If Item 46 fails when selected by Hook 3 then it might be assumed that it will fail when selected by Hook 4 and, possibly, Hook 5. It might be acceptable to wait and see. Another possibility would be to move Items 51 et seq one integer to the right and to place Item 46 at position 51 on the number line. A further possibility is to move Item 46 to the very end of the number line so that it becomes the last Item in the Set.

Success of an item that is presented

If Items picked by a particular Hook are consistently successful it might be assumed that moving the Hook to the left might yield a lower but still acceptable Success Rate for Items selected by that Hook.

In the present invention therefore, a Hook is moved to the right if the item selected by it for a Subset is successfully tested and to the left if it is not.

The Desired Success Rate

The inventor has found that a success rate of 80% gives good results. This may be set with in the system or it may be set by the user.

A movement of the Hook to the right by one integer has been illustrated above but it was noted that a movement of one integer is not key to the invention. No amount for the movement to the left of a Hook on success has so far been given ..It is now discussed what the relationship might be between the size of movement to the right of a Hook (referred to as r), the size of movement to the left of a Hook (referred to as w) and the desired Success Rate (referred to as/?).

Suppose that the Hook is moved to the right by r when a test is successfully passed and to the left by w when a test is failed. The formula/?r = (l-p)w can be used to link ?, r and w. Thus, once ? and r have been determined w can be calculated. The validity of the formula is seen by assuming that the Hook is, in fact, correctly positioned on the claw to give the desired success rate/?. Suppose that a question is successfully answered. The iterative approach means that the Hook will be moved to the right by r and, subject to the comments below, if the proportion of times this happens is/?, then the expected movement to the right of the Hook will be pr. The proportion of times the Item is unsuccessfully tested, subject to the comments below, will be (1-/?) and the expected movement of the Hook to the left will be (l-p)w. If the Hook is correctly positioned on the Claw to produce the desired success rate ? then pr = (l-p)w. In other words the expected movement to the right must equal the expected movement to the left if the Hook is to stay at that position in the long term.

Suppose that the Hook is placed on the claw too far to the left to give the required success rate/?(i.e. the gap between successive repeats of the same question are too far apart and the user's memory has already failed to recall the item). The probability of success is less than/? because too many other items are presented before a particular Item is presented again. This lower (actual) probability of success may be called/?'. Suppose that/?' =/? - d. Then it follows that the expected movement of the Hook is m' = (p-d)r - (l-p+d)w =pr - (l-p)w - d(r+w) = -d (r+w) < 0. Thus, if the Hook is placed too far to the left on the Claw to achieve the desired success rate/?, the expected movement in the Hook, m', is negative. In other words, if the Hook is too far to the left then the Hook can be expected to move to the right on the Claw provided/?, r and w satisfy the formula pr = (l-p)w.

Similarly, if the Hook is placed too far to the right then d will be negative and the expected movement in the Hook, m, will be positive, ie to the left of its position on the Claw. Thus if/?, r and w satisfy the formula pr = (l-/?)w then the expected movement in the Hook is towards the point on the claw at which the desired Success Rate can be expected to be achieved.

Furthermore, as the magnitude of d increases so does the expected movement of the Hook, given above by - d(r+w). The expected movement of the Hook on the Claw is also dependent on the magnitude of r and w. When the appropriate position of the Hook on the Claw is not known it might be appropriate to use a length for w which is large so that the Hook quickly moves to the most appropriate position on the Claw. The disadvantage of using a large w is that when the Hook is at approximately the right place on the Claw, successful and unsuccessful presentations will result in large oscillations about the correct position on the Claw, which will result in variability in the actual success rate which is achieved. Conversely, if w is small, then it will take longer for the iterative process to move the Hook to the most appropriate position on the Claw.

By moving only the claw that has picked up the question that has not been answered correctly, the gaps between succesive questions tailors itself to the user's learning curve thus providing a high efficiency of learning.

Other aspects of the invention

The only Hook that cannot be moved around according to the formula given above is the first Hook. The first Hook defines the gap characteristics of the other Hooks and it would not be appropriate to vary its position from the right hand end of the Claw. In other words, this embodiment doesn't stop presenting new material to the user simply because he has got one question wrong. This leads to boredom and inefficiency of learning.

At the start, the first Hook will point at the first item on the number line. Other Hooks will not point at any Items and thus the first Subset will comprise just one Item. Gradually, as the Claw moves to the right along the number line, other Hooks will begin to point at items on the number line and those Items will begin to be included in Subsets and hence to be presented for second and subsequent times.

Towards the end, the first and subsequent Hooks will cease to point at Items placed on the Number line. The number of Items selected for any particular Subset will gradually diminish as the first Hook ceases to contribute Items to subsets and then the second Hook and so on. If an item is unsuccessfully presented - and it is decided in such cases that the item must again be selected by every Hook in turn - then it will need to be placed on the number line to the right of where the first Hook is positioned at that time.

In some cases an Item that has been unsuccessfully presented may be moved to a place on the number line to the right of its current position. This might, in turn, displace other items on the number line which might, in turn, move an Item that has already been selected by a particular Hook from a position to the left of the Hook to a position to the right of the Hook. The consequence of this is that that Hook will again select the Item. In such a case it might be desirable to note whether a particular Hook has selected an Item and, if the same Hook again selects it, to ignore that selection

An extension to the invention is to allow Items to be placed at non-integer positions on the number line. In such a case, if a Hook moves past those items then all the items that it moves past will be selected. In such a manner, several items may be grouped together and always asked within the same Subset, provided they are placed sufficiently close together on the number line. If one of those Items is unsuccessful in a presentation of it several different ways could be adopted to deal with that. For example, all Items are treated as having been unsuccessful and moved to another point on the number line. Precisely how a failure is to be dealt with is not key to the invention.

Of course, the number line is represented as lying from left to right but could, for example, go to the left (or any other direction) and the Claw be moved to the left with the first Hook being designated as that which is at the leftmost edge of the Claw. Such an arrangement is merely a mirror image of that described above and is not considered to be anything other than a different way of arranging the invention.

The present invention also acts as an assessment device.

The system continuously monitors the user's responses during a period of learning and adjusts the gap lengths that determine the cycle at which a question is asked accordingly. The user's response to a question will often take place within a matter of seconds and the system described is able to carry out the monitoring and adjusting exercise fast enough to keep pace with the user's requirements. At the end of a period of learning the system records a composite index (the learning and familiarity index) of the user's ability to learn and the user's familiarity with the material. This index was described as the User Profile in Figure 4.

The index comprises the gaps achieved at the end of a learning session in reverse order. Assume the following profile has been achieved:

gap (0) = l gap (l) = 3 gap (2) = 8 gap (3) = 15

The system records the learning and familiarity index in the following way: 15.8.3.1.

The index is a measure of the user's familiarity with the material. If the user is totally familiar with the material and therefore knows the answer to every question, there will be no occasion for the gaps to be reduced. In fact, with every correct answer, the relevant gap length will be increased. Thus the learning and familiarity index will, for each gap length, generate a higher number than that for someone who is less familiar with the material and therefore makes mistakes.

The index is a measure of the user's ability to learn material. If the user has a good ability to learn and remember material, he or she will make less mistakes than someone with a lower ability (when both users have previously seen the material for the same amount of time). Thus the learning and familiarity index will contain higher numbers for the person with good ability to learn and remember.

The learning and familiarity index is not an absolute measure since it depends on factors which includes:

1. The profile of gap lengths at the start of a period of learning; 2. The amount of material being learned;

3. The difficulty of the material;

4. The number of correct responses required before learning of a question is terminated.

However, when the factors listed above are held constant for a particular series of questions, the index represents a measure of current familiarity with the material and a benchmark for improvement. It also provides a focus for discussion with the user of how to improve ability to remember material in the first place.

In addition to the learning and familiarity index, the system records several other sets of information which the user would not be able to record whilst maintaining satisfactory concentration on the material being learned.

The system records the results of a test in a Test Results File. The description of each question that is tested is recorded in a file, stored in electronic format, together with whether that question was correctly or incorrectly answered, and the inputs to the dialogue box which precedes the test (for example, whether the questions were presented in random order or not).

The system records the results of a learning session in a Learning Results File. The description of each question that is tested is recorded in the Learning Results File, stored in electronic format, together with whether that question was correctly or incorrectly answered and the inputs to the dialogue box which precedes the learning session (for example, whether the answer is initially shown to the user to learn). This file can be analysed at a later time to identify the most problematic questions (i.e. those that appear the greatest number of times).

The system records more detailed results of a learning session in a Learning Statistics File. As each question has to be answered correctly several times in a row the particular gap length associated with the question on the occasion on which it is tested is also recorded, together with details of the other gap lengths. Thus, for example, if a question must be answered correctly four times consecutively before it stops being presented, then it will be shown in the file a minimum of four times. If it is incorrectly answered on the second time it is presented, for example, and correctly answered from then on, it will appear in the file six times. Each occasion on which the question description appears is accompanied by details of gap lengths. This file can be analysed at a later time to identify the manner in which the gap lengths vary over the course of the session. This information could be used to investigate several features. These might include:

1. What the appropriate gap profile is for an individual learning information for the first (or subsequent) times.

2. The extent to which the gap profile varies according to the type of information being learned. It could be hypothesised that gap lengths for difficult and almost meaningless material such as the square roots of numbers might be lower than those for general knowledge material which can be linked to knowledge already known by the user and may therefore be easier to remember.

3. The extent to which the gap profile varies as the same set of information is repeatedly tested (it could be hypothesised that the gap lengths would become increasingly greater).

4. The extent to which the gap profile varies as information is left for varying periods of time before next being revised.

5. The extent to which the gap profile varies following a period of instructions to the user on how to remember information. (It might be hypothesised that in two matched groups the group that had received instructions in how to improve their memory might show higher gap lengths than the other group.)

In addition to recording changes in gap lengths on a question by question basis, the Learning Results File also records the exact time at which a question was presented. The File can be analysed at a later date to determine how long was spent on a particular question to investigate several features. These might include: 1. The extent to which time spent on a particular question is correlated with whether it was subsequently answered correctly or incorrectly.

2. The extent to which a time spent on questions varies as a learning session progresses. This might ultimately indicate for a particular individual what the optimal period of study should be before taking a rest.

Claims

CLAIMS:

1. A teaching arrangement comprising: a source of questions and answers, means for asking groups of questions on a plurality of occasions, means for determining if an answer is correct, means for extending an interval between successive asking of groups of questions in response to a correct answer, and means for reducing at least an interval between successive asking of that group of questions in response to an incorrect answer.

2. A teaching arrangement as claimed in claim 1, wherein the means for reducing at least an interval between successive asking of that group of questions in response to an incorrect answer comprises means for re-timing that group of questions relative to the remaining questions.

3. A teaching arrangement as claimed in claim 1 or claim 2, further comprising means for indicating a question and answer to a student before the question is asked for a first time.

4. A teaching arrangement as claimed in claim 1, claim 2 or claim 3, further comprising means for indicating a question and answer to a student in response to an incorrect answer.

5. A teaching arrangement as claimed in any one of the preceding claims, further comprising means for reducing an interval between successive asking of a plurality of groups of questions in response to an incorrect answer.

6. A teaching arrangement as claimed in claim 5, wherein the means for reducing an interval between successive asking of a plurality of groups of questions in response to an incorrect answer reduces that interval by a greater margin than the means for extending an interval between successive asking of groups of questions in response to a correct answer.

7. A teaching arrangement as claimed in Claim 6, wherein the margin by which an interval between successive asking of a plurality of groups of questions in response to an incorrect answer is reduced is approximately four times greater than that margin by which an interval between successive asking of groups of questions is extended in response to a correct answer.

8. A teaching arrangement as claimed in Claim 6, wherein the margin by which an interval between successive asking of a plurality of groups of questions in response to an incorrect answer is reduced is between 3.5 and 4.5 times greater than that margin by which an interval between successive asking of groups of questions is extended in response to a correct answer.

9. A teaching arrangement as claimed in any one of the preceding claims, wherein a group of questions is suspended in response to a predetermined number of correct answers.

10. A teaching arrangement as claimed in Claim 9, wherein the predetermined number of correct answers are consecutive correct answers.

11. A teaching arrangement as claimed in any one of the claims 1 to 8, wherein a group of questions is suspended after they have been asked a predetermined number of times.

12. A teaching arrangement as claimed in any one of the preceding claims, wherein a group of questions comprises a single question.

13. A teaching arrangement as claimed in any one of the claims 1 to 12, wherein an interval between successive asking of groups of questions is altered in response to a success rate.

14. A teaching arrangement as claimed in claim 13, wherein the inrvals are altered using the equation pr = (l-p)w.

15 A teaching arranagement as claimed in any one of the claims 13 to 15, wherein the success rate is set by a user.

16. A teaching arrangement comprising means for providing a plurality of questions and corrresponding answers, means for counting cycles of questions, means for asking one of the plurality of questions, during a particular cycle, for the n* time and another of the plurality of questions for the (n+N)* time where N is an integer greater than zero, means for receiving an answer and determining if the answer is correct, means for increasing an interval between successive asking of a particular question in response to the correct answer, means for decreasing an interval between successive asking of a particular question in response to an incorrect answer, and means for updating the means for counting cycles of questions.

17. A method of teaching comprising: a source of questions and answers, asking groups of questions on a plurality of occasions, determining if an answer is correct, extending an interval between successive asking of groups of questions in response to a correct answer, and reducing at least an interval between successive asking of that group of questions in response to an incorrect answer.

18. An arrangeemnt for teaching substantially as herein described with reference to the accomapnying drawings.

19. A method of teaching substantially as herein described with reference to the accomapnying drawings.