GB2183880A

GB2183880A - Speech translator for the deaf

Info

Publication number: GB2183880A
Application number: GB08627625A
Authority: GB
Inventors: Billy A Montgomery
Original assignee: International Standard Electric Corp
Current assignee: International Standard Electric Corp
Priority date: 1985-12-05
Filing date: 1986-11-19
Publication date: 1987-06-10
Also published as: ES2004496A6; GB8627625D0

Abstract

A speech translator has a phoneme recogniser for detecting phonemes and a display for displaying them. An interface communicates between the recogniser and a telephone line. The phonemes recogniser may be provided in the user's home. Alternatively a plurality of phoneme recognisers may be provided in a central telephone office or in the office of an independent service provider. The user may be provided with a keypad, for reply, coupled to a pre-recorded announcement module and a text-to-speech module. <IMAGE>

Description

SPECIFICATION Speech translator for the deaf The invention relates to a communication device for the deaf.

Telecommunications Devices for the Deaf (TDDs) are known which provide the capability for someone at the other end of a telephone connection to type information on a keyboard.

This information is encoded and transmitted over the telephone connection by one of several means and arrives at the TDD belonging to the deaf person. There it is decoded and displayed or printed for the deaf person to read. The deaf person can respond in a similar manner.

For a deaf person to carry on a conversation with a party over the telephone network, that other party as well as the deaf person must have a TDD. TDDs are relatively expensive, and are not in common use. Thus, a deaf person cannot communicate with most of the population via the telephone.

The invention, incorporated into a device entitled the Speech Translator for the Deaf (STD), overcomes these limitations by translating the speech sounds coming in over the telephone connection to symbolic text which is displayed to the deaf user. The deaf user can then recognise this symbolic text as words. This symbolic text is a phonetic representation of the incoming sounds, thereby looking like the spoken words. For example: the spoken word "Boston"; might appear to the deaf user as "baahstun".

In order that the invention and its various other preferred features may be understood more easily, some embodiments thereof will now be described, by way of example only, with reference to the drawings in which: Figures 1 and 2 are basic schematic drawings which show use of the invention in a preferred embodiment, Figure 3 is a block schematic diagram showing an implementation of the invention, Figures 4 and 5 show schematically portable alternative embodiments constructed in accordance with the invention, and Figures 6, 7 and 8 are block schematic diagrams which show alternative embodiments formed as a unit located remotely from the user.

Figures 1 and 2 depict the STD in a deaf user's residence connected to the telephone network with a hearing party at the other end of the connection. A deaf user of the STD who has never heard speech may require some training to associate the symbolic text with words, but this would still be considerably easier than learning lip reading. Deaf users who at one time could hear should require little or no training in the use of the STD.

The deaf user has the capability, either user is also mute, the STD would optionally convert text which has been typed into the keyboard to mechanical speech which is then output to the hearing party. This would be done using an optional Text-to-Speech module.

Also, pre-recorded, canned phrases can be provided which are activated by simple commands from the keyboard.

The particular implementation described here interfaces directly to the telephone network, and is described with reference to Figure 3.

The Telephony Interface 1 interfaces to the telephone line. It provides notification of incoming ringing and an idle/seized status indication to an Output Controller 6. It also responds to commands from a Keypad Interface 9 to answer or seize the telephone line, by providing loop resistance for current to flow from the serving telephone office, and to generate address or dial pulses on the telephone line, by interrupting the loop current.

The Hybrid 2 is a 2-wire to 4-wire circuit to separate the incoming speech or tones from the outgoing speech or tones.

The Audio Monitor 3 is a speaker which can be switched in or out so the conversation from the far end can be monitored by someone who can hear at the location of the deaf user.

The Tone Detector 4 detects special tones coming from the telephone network such as ringback tone, busy tone, reorder (fast busy) tone, and dial tone. As an option to provide a lower level of communication in case a phoneme recogniser 5 fails, or an accent or speech impediment makes recognition difficult, DTMF (duai tone multi-frequency) tone detectors may also be provided. The outputs of the Tone Detector are sent to the Output Controller 6.

The Phoneme Recogniser 5 is programmed to recognise small parts of speech in the incoming words. From 40 to 150 of these small distinct parts or phonemes would, for example, be used. The output of the Phoneme Recogniser is passed to the Output Controller 6.

The Output Controller 6 controls a display 7. It takes the outputs from the Tone Detector 4 and the Telephony Interface 1 and displays, either on an alpha-numeric display or labelled indicator lamps, the status of the telephone connection along with any applicable messages having to do with operation of the device. It also takes the output of the Phoneme Recogniser 5 and converts this output to symbolic text which is displayed on the alpha-numeric display 7 for the deaf user to read and interpret.

The Display 7 consists of an alpha-numeric display on which the translated symbolic text and any applicable operation control messages are presented. Also associated with the Display there are individual indicator lamps associated with control of the device.

A Keypad 8 is the means by which the deaf user both controls the device itself and, in case the deaf user is also mute, communicates with the hearing party at the far end of the telephone connection. It consists of an alpha-numeric keypad with various special function keys and switches. The Keypad outputs are directed to the Keypad Interface 9.

The Keypad Interface 9 interprets commands from the Keyboard 8 and directs them to other blocks of this system such as a DTMF (or Touch-Tone) Generator 10 which can generate DTMF address signals for routing to the telephone network; a Pre-Recorded Announcements module 11 which can generate a prerecorded announcement for routing to the telephone network; a Text-to-Speech Module 12 which can convert the ASCII text from the keyboard to speech and route it onto the telephone network; and the Telephony Interface 1 which would seize or answer the telephone line or send out addressing dial pulses.

The DTMF Generator 10 generates DTMF address signals which are routed to the telephone network in response to commands from the Keypad 8.

The Pre-Recorded Announcement Module 11 generates pre-recorded announcements which are routed to the telephone network in response to commands from the Keypad 8. This module could be optional as some deaf users have no speaking problem and would not need this capability. This would allow standard phrases to be played in response to short commands from the Keypad. Examples of such phrases are as follows: "Hello, please speak slowiy and distinctly as you are speaking into a device to translate for a deaf per son "Please repeat that again".

"Yes", "No", "Okay", etc.

The Text-to-Speech Module 12 translates ASCII text from the Keypad into speech and routes it onto the telephone network. This module could be optional as some deaf users have no troubie speaking and would not need this capability.

A Microphone and Amplifier 13 are provided to allow someone to speak directly onto the telephone line. This also, in conjunction with a Test switch or relay 14, allows for test and verification that the system is functioning properly.

The Test switch or relay 14 provides for the transmit side of the system to be connected directly to the receive side for purposes of test and verification.

The invention may also be employed for other applications. By coupling the Telephone Interface 1 with a microphone connected to the receive circuitry and a speaker connected to the transmit circuitry, this device could be used to carry on a face-to-face conversation between a deaf user and another person who speaks into the microphone. The deaf user observes and interprets the symbolic text on the display in the same manner as with the other applications. This could be a portable unit. Figure 4 illustrates this implementation.

By replacing the Telephony Interface 1 with an acoustic coupler, this device could become a portable unit which could be carried about and used at pay telephones, hotels, etc. by placing the telephone handset in the acoustic coupler. The unit does not then have to be connected directly to the telephone line. Figure 5 illustrates this implementation.

The STD, or several of them, could be placed in a centrai location, such as a telephone office, and its use leased on a daily, weekly, or monthly basis. To accomplish this, the Display and Keypad resides in the deaf user's home and can be connected when needed to the main unit over a dial-up data communication with data modems. Several of these units can be supplied in the central location with switched access to telephone lines both on the hearing party side and on the deaf user side which would have the remote data capability via data modems. This may require some intelligence in the switches between the telephone lines and both sides of the STDs to accomplish the connection.

Figure 6 shows the centralised service provided in the telephone office with the data connection between the STD in the central office and the keyboard/display in the deaf user's residence accomplished using standard data modems. This may require the deaf user to have two telephone lines to his home.

Figure 7 shows the service still being provided in the telephone office, but here the connection to the deaf user's home is over a single voice/data line, for example, an LADT or an ISDN type of line.

Figure 8 shows the service being provided by an independent service provider who has several voice/data lines leased into his facility from the telephone office. This may require the served deaf user to have one normal telephone line and one voice/data type line to his home. This same service could be provided by replacing each voice/data line with two normal telephone lines and using standard modems to transmit data.

The STD presents to the user not only the symbols representing the voiced sounds, but also indications as to the status of the telephone line (incoming ringing, busy tone, dial tone, ringback tone, reorder tone, DTMF di gits, etc.). The STD also provides the capabil ity of generating speech from typed text or to generate a pre-designated speech message so that a deaf user who is also mute can com municate back to the party at the other end of the connection. The STD allows several methods of centralising this translation function at some common facility which could provide service to a number of users.

The phoneme recogniser of the exemplary system detects the phonemes, from a defined repertoire, that are present in the speech which is presented to its input and to present at its output some code, in real time, to the external circuitry indicating the presence of those phonemes.

A phoneme is a distinctive speech sound which, when coupled with other phonemes, form words. Words in the English language can be represented by as few as around 50 phonemes. For better clarity more than 100 can be used. A typical two-syllable word might consist of 6 to 12 phonemes depending upon the repertoire of defined phonemes.

A phoneme recogniser which may be employed in the STD is described, for example, in U.S. Patent 4,181,813 to John Marley, which employs delta modulation techniques to distinguish different phonemes. A system which embodies the invention does not require entire words to be recognised by the equipment-this is a function of the human brain putting the phonemes together.

Claims

1. A communication device, comprising a phoneme recogniser, for detecting phonemes present in speech, display means, coupled to the recogniser, for displaying the detected phonemes, means, coupled to the recogniser, to interface the recogniser to a telephone line, and means, coupled to the display means and to the phoneme recogniser for controlling the display means.

2. A communication device as claimed in claim 1, comprising a 2-wire to 4-wire hybrid circuit, coupled to the input of the phoneme recogniser, for separating incoming signals of the device from outgoing signals of the device.

3. A communication device as claimed in claim 2, comprising a test switch, coupled to the hybrid circuit, for connecting the outgoing signals to the incoming signal output of the hybrid circuit for device test and verification.

4. A communication device as claimed in claim 1, 2 or 3, comprising means, coupled to the display control means, for detecting tones coupled thereto from a telephone line.

5. A communication device as claimed in claim 4, wherein indications of the tones are displayed by the display means to provide indications as to the status of the telephone line.

6. A communication device as claimed in any one of the preceding claims, comprising tactile control means, coupled to the telephone line interface, for providing device control to a user of the device.

7. A communication device as claimed in claim 6, wherein the tactile control means comprises a keypad, for providing the user with tactile control input means, and an interface, coupled between the keypad and the telephone line interface, to provide an interface therebetween.

8. A communication device as claimed in any one of the preceding claims, comprising means, coupled to the telephone line interface means, for providing outgoing address signals to the telephone line.

9. A communication device as claimed in any one of the preceding claims, comprising means, coupled to the telephone line interface means, for providing pre-recorded outgoing announcements to the telephone line.

10. A communication device as claimed in any one of the preceding claims, comprising an audio monitor, switchably coupled to the input of the phoneme recogniser, for providing audible monitoring of speech.

11. A communication device as claimed in any one of the preceding claims, comprising means, coupled to the telephone line interface means, for translating user-coded text into user-supplied speech and providing the user supplied speech to the telephone line.

12. A communication device as claimed in any one of the preceding claims, comprising a microphone, coupled to the telephone line interface, for providing outgoing speech to the telephone line.

13. A communication device as claimed in any one of the preceding claims, located remotely from a user of the device.

14. A communication system, comprising at least one phoneme recogniser, for detecting phonemes present in speech; and means, coupled to said phoneme recogniser, for providing indications of said detected phonemes via communication links to displays located remotely from said phoneme recogniser.

1 5. A communication device or system substantially as described herein with reference to the drawings.

CLAIMS Amendments to the claims have been filed, and have the following effect: New or textually amended claims have been filed as foliows:

16. A communication device as claimed in claim 4 wherein the means for detecting tones are coupled to the means to interface, and the display means are controlled by the means for controlling as a function of outputs produced by the means for detecting tones and the phoneme recognizer.

17. A communication device in accordance with claim 2 wherein the 2-wire to 4-wire hybrid circuit has incoming and outgoing signal portions.

18. A communication device in accordance with claim 17 further comprising a test switch for connecting the outgoing signal portions to the incoming signal portions of the hybrid circuit for device test and verification.