JP6110539B1 - Speech translation device, speech translation method, and speech translation program - Google Patents

Abstract

An object of the present invention is to obtain an opportunity for a conversation with a foreigner who is a conversation person and to select or select a language of a conversation person in a natural conversation flow. A speech translation apparatus according to an aspect of the present invention includes an input unit for inputting a user's speech, a translation unit for translating the content of the input speech, and an output unit for outputting the translated content in speech or the like. . In addition, the speech translation apparatus includes a storage unit that stores a plurality of fixed sentences and translations in a plurality of languages, and a plurality of fixed forms for the user to select a specific fixed sentence from the plurality of fixed sentences. A fixed sentence presentation unit for displaying a sentence, a translated sentence presentation unit for displaying a translation in a plurality of languages for a specific fixed sentence, and a language selection means for a conversation person to select a desired language from a plurality of languages are displayed. And a language selection means presenting section. [Selection] Figure 5

Description

  The present invention relates to a speech translation device, a speech translation method, and a speech translation program.

  To make conversations between people who do not understand each other's language, for example, conversations between shop assistants (sales staff at restaurants, etc.) and foreigners (tourists from abroad, etc.) A speech translation technology has been proposed in which the text content is machine-translated into the language of the other party and displayed on the screen, or the text content is played back using speech synthesis technology (for example, Patent Document 1). In addition, a speech translation application that operates on an information terminal such as a smartphone that embodies such speech translation technology has been put into practical use (see, for example, Non-Patent Document 1).

Japanese Patent Laid-Open No. 9-34895

U-STAR Consortium Home Page [Search on January 25, 2016], Internet <URL: http://www.ustar-consortium.com/app_en/app.html>

  However, when trying to communicate with foreigners through conversations, it is sometimes impossible to create a chance to speak and to start the conversation itself. In addition, even if you have a speech translation application or speech translation device, you cannot select the language of the other party if you do not know the language spoken or understood by the interlocutor (foreigner). As a result, the speech translation application or speech translation device cannot be used.

  Therefore, the present invention has been made in view of such circumstances, and can provide a trigger for a conversation with a foreigner who is a conversation person, and selects the language of the conversation person in a natural conversation flow. Another object is to provide a speech translation device, a speech translation method, and a speech translation program that can be selected.

  In order to solve the above problems, a speech translation apparatus according to an aspect of the present invention provides an input unit for inputting a user's speech, a translation unit that translates the content of the input speech into content in different languages, and a conversation person And an output unit for outputting the translation content (parallel translation) of the input speech as speech and / or text. Then, the speech translation apparatus specifies a plurality of fixed sentences (question sentences, etc.), a storage unit for storing translated sentences in a plurality of languages for each of the plurality of fixed sentences, and a user specified from the plurality of fixed sentences To select a fixed phrase, a fixed sentence presentation unit that displays those fixed phrases, and a translation that displays translations in multiple languages for the specific fixed sentence when a specific fixed sentence is selected It further includes a presentation unit and a language selection unit presentation unit that displays a language selection unit for the conversation person to select a desired language from a plurality of languages.

  More specifically, the language selection means presenting unit displays the language selection means (button, check box) after displaying the translated sentences in the plurality of languages with respect to the specific fixed sentence or together with displaying the translated sentences (simultaneously). Etc.) may be displayed.

  Further, after the translation of the specific standard sentence in a plurality of languages is displayed, or together with the display of the translation (at the same time), a response means for the dialog person to reply to the above and the standard sentence (question sentence, etc.) May further include a response means presenting unit for displaying.

  A speech translation method according to an aspect of the present invention uses a speech translation apparatus according to the present invention including an input unit, a translation unit, an output unit, a storage unit, a fixed sentence presentation unit, a translation presentation unit, and a language selection means presentation unit. Is the method. That is, in the method, the step of inputting the user's voice, the step of the translation unit translating the content of the input voice into the content of a different language, and the step of outputting the translation content by voice and / or text And a step of storing a plurality of fixed sentences and translations in a plurality of languages for each of the plurality of fixed sentences, and a fixed sentence presenting part, wherein the user selects a specific fixed form from among the plurality of fixed sentences A step of displaying a plurality of standard sentences to select a sentence, and a step of displaying a translation in a plurality of languages for the specific standard sentence when the specific sentence is selected by the translation presentation unit And a language selection means presenting unit displaying a language selection means for the interlocutor to select a desired language from among a plurality of languages.

  A speech translation program according to an aspect of the present invention includes a computer (not limited to a single type or a single type, but may be a plurality or a plurality of types; the same applies hereinafter), an input unit for inputting a user's voice, A translation section that translates content into different language contents, an output section that outputs the translation contents in speech and / or text, a plurality of fixed sentences, and a plurality of language sentences for each of the plurality of fixed sentences are stored. When the user selects a specific fixed sentence from a plurality of fixed sentences, and when a specific fixed sentence presenting section that displays those fixed sentences is selected, A translation presenting unit for displaying a translation in a plurality of languages for the specific fixed phrase, and a language selection unit presenting unit for displaying a language selection unit for a dialogue person to select a desired language from the plurality of languages; It is intended to function Te.

  According to the present invention, a plurality of fixed phrases and a plurality of language translations for each of the plurality of fixed sentences are stored in advance, and the user selects an appropriate “specific fixed sentence” from them. be able to. Then, by presenting the translated texts in the plurality of languages to the conversation person, the opportunity for conversation can be obtained, so that smooth communication by conversation with the conversation person becomes possible. In addition, since a conversation can be displayed by displaying translations in a plurality of languages for such a specific fixed sentence to the conversation person, a desired language that the conversation person can speak or understand can be selected. Language selection can be performed in a natural conversation flow.

1 is a system block diagram schematically showing a preferred embodiment of a network configuration and the like related to a speech translation apparatus according to the present invention. It is a system block diagram which shows roughly an example of a structure of the information terminal which is a user apparatus in the speech translation apparatus by this invention. It is a system block diagram which shows roughly an example of a structure of the server in the speech translation apparatus by this invention. It is a flowchart which shows an example of the flow (a part) of the process in 1st Embodiment of the speech translation apparatus by this invention. (A) thru | or (D) is a top view which shows an example of the transition of the display screen in the information terminal by 1st Embodiment. (A) And (B) is a top view which shows an example of the transition of the display screen in the information terminal by 1st Embodiment. It is a flowchart which shows an example of the flow (a part) of the process in 2nd Embodiment of the speech translation apparatus by this invention. (A) And (B) is a top view which shows an example of the transition (part) in the screen display of the information terminal 10 by 2nd Embodiment.

  Hereinafter, embodiments of the present invention will be described in detail. The following embodiments are examples for explaining the present invention, and are not intended to limit the present invention only to the embodiments. The present invention can be variously modified without departing from the gist thereof. Furthermore, those skilled in the art can employ embodiments in which the elements described below are replaced with equivalent ones, and such embodiments are also included in the scope of the present invention. Furthermore, positional relationships such as up, down, left, and right shown as needed are based on the display shown unless otherwise specified. Furthermore, various dimensional ratios in the drawings are not limited to the illustrated ratios.

(Device configuration)
FIG. 1 is a system block diagram schematically showing a preferred embodiment such as a network configuration related to a speech translation apparatus according to the present invention. FIG. 2 is a system block diagram schematically illustrating an example of the configuration of the information terminal 10 that is a user apparatus in the speech translation apparatus 100. FIG. 3 is a system block diagram schematically showing an example of the configuration of the server 20 in the speech translation apparatus 100. In this example, the speech translation apparatus 100 includes a server 20 that is electronically connected to the information terminal 10 (user apparatus) used by the user via the network N (but is not limited to this).

  The information terminal 10 employs a user interface such as a touch panel and a display with high visibility, for example. The information terminal 10 here is a portable tablet terminal device including a mobile phone represented by a smartphone having a communication function with the network N. The information terminal 10 further includes a processor 11, a storage resource 12, a voice input / output device 13, a communication interface 14, an input device 15, a display device 16, and a camera 17. In addition, the information terminal 10 operates by the installed speech translation application software (at least a part of the speech translation program according to the embodiment of the present invention), so that a part of the speech translation apparatus according to the embodiment of the present invention or It functions as a whole.

  The processor 11 includes an arithmetic logic unit and various registers (program counter, data register, instruction register, general-purpose register, etc.). Further, the processor 11 interprets and executes speech translation application software, which is the program P10 stored in the storage resource 12, and performs various processes. The speech translation application software as the program P10 can be distributed from the server 20 through the network N, for example, and may be installed and updated manually or automatically.

  The network N includes, for example, a wired network (a short-range communication network (LAN), a wide-area communication network (WAN), a value-added communication network (VAN), etc.) and a wireless network (mobile communication network, satellite communication network, Bluetooth ( Bluetooth (registered trademark)), WiFi (Wireless Fidelity), HSDPA (High Speed Downlink Packet Access), etc.).

  The storage resource 12 is a logical device provided by a storage area of a physical device (for example, a computer-readable recording medium such as a semiconductor memory), and an operating system program, a driver program, various data, etc. used for processing of the information terminal 10 Is stored. Examples of the driver program include an input / output device driver program for controlling the audio input / output device 13, an input device driver program for controlling the input device 15, and a display device driver program for controlling the display device 16. Can be mentioned. Furthermore, the voice input / output device 13 is, for example, a general microphone and a sound player capable of reproducing sound data.

  The communication interface 14 provides a connection interface with the server 20, for example, and is configured from a wireless communication interface and / or a wired communication interface. The input device 15 provides an interface for accepting an input operation by a tap operation such as an icon, a button, or a virtual keyboard displayed on the display device 16, and is externally attached to the information terminal 10 in addition to the touch panel. Various input devices can be exemplified.

  The display device 16 provides various types of information as an image display interface to a user or a dialoguer (conversation partner) as necessary. Examples thereof include an organic EL display, a liquid crystal display, and a CRT display. The camera 17 is for capturing still images and moving images of various subjects.

  The server 20 is constituted by, for example, a host computer having a high arithmetic processing capability, and expresses a server function by operating a predetermined server program in the host computer, for example, a speech recognition server, a translation server, And a single or a plurality of host computers functioning as a speech synthesis server (in the drawing, it is indicated by a single, but is not limited thereto). Each server 20 includes a processor 21, a communication interface 22, and a storage resource 23.

  The processor 21 is composed of an arithmetic and logic unit for processing arithmetic operations, logical operations, bit operations and the like and various registers (program counter, data register, instruction register, general-purpose register, etc.), and is stored in the storage resource 23. P20 is interpreted and executed, and a predetermined calculation processing result is output. The communication interface 22 is a hardware module for connecting to the information terminal 10 via the network N. For example, the communication interface 22 is a modulation / demodulation device such as an ISDN modem, an ADSL modem, a cable modem, an optical modem, or a soft modem.

  The storage resource 23 is a logical device provided by, for example, a storage area of a physical device (a computer-readable recording medium such as a disk drive or a semiconductor memory), and each includes one or a plurality of programs P20, various modules L20, and various databases. D20 and various models M20 are stored. Further, the storage resource 23 includes a plurality of question templates (for example, Japanese text T3 to be described later) prepared in advance for the user to speak to a conversation person, and a plurality of languages for each of the plurality of question templates. Translations (for example, foreign language texts T6 and T11 described later) are also stored. As described above, the storage resource 23 functions as a “storage unit”.

  The program P20 is the above-described server program that is the main program of the server 20. In addition, the various modules L20 perform a series of information processing related to requests and information transmitted from the information terminal 10, so that they are appropriately called and executed during the operation of the program P10 (moduleized subprograms). ). Examples of the module L20 include a speech recognition module, a translation module, and a speech synthesis module.

  The various databases D20 include various corpora required for speech translation processing (for example, in the case of Japanese and English speech translation, a Japanese speech corpus, an English speech corpus, a Japanese character (vocabulary) corpus, an English character) (Vocabulary) Corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.), voice database, management database for managing information about users, and the like. Examples of the various models M20 include acoustic models and language models used for speech recognition.

  An example of operations and operations of speech translation processing in the speech translation apparatus 100 configured as described above will be further described below.

(First embodiment)
FIG. 4 is a flowchart showing an example of (a part of) the processing flow in the speech translation apparatus 100 of the first embodiment. 5A to 5D and FIGS. 6A and 6B are plan views showing an example of (a part of) transitions in the screen display of the information terminal 10 according to the first embodiment. is there. Here, the conversation is performed when the user of the information terminal 10 is a salesclerk who speaks Japanese, and the conversation person (conversation partner) is a foreigner who speaks English, Chinese (for example, two types of typefaces), or Korean. Assumes (however, the language is not limited to this).

  First, when the user (store clerk) starts the application (step SU1), the language selection screen for the conversation person shown in FIG. 5A is displayed on the display device 16 of the information terminal 10 (step SJ1). The language selection screen includes an English text T1 indicating that the conversation person is encouraged to talk using the application, and that the user agrees to the privacy policy of the application by selecting a language button described later. Then, an illustration 51 representing the conversation image is displayed. Further, below the English text T1 on the language selection screen, a language button 52 (language selection means) indicating a plurality of typical languages (English, Chinese, and Korean) is displayed. Further below that, a cancel button B1 for closing the language selection screen and ending the application is also displayed.

  Next, at this point, since the language that the interlocutor can speak or understand is unknown, the user taps the “English” portion of the language button 52 (language selection means) for the time being, for example. When English is selected, the processor 21 of the server 20 and the processor 11 of the information terminal 10 display a standby screen for voice input in Japanese and English on the display device 16 as a home screen (FIG. 5 (B); Step SJ2). On this standby screen, the Japanese text T2 asking which of the user's language and the talker's language is to be spoken, the Japanese input button 53a for inputting Japanese speech, and the English speech input. An English input button 53b is displayed. As described above, the processors 11 and 21 and the display device 16 function as a “language selection unit presenting unit”.

  Also, on this standby screen, a voice button 54 for selecting a list display of a plurality of preset canned sentences, the language selection screen shown in FIG. Also displayed are a language selection button 55 for redoing the language selection, a history button 56 for selecting a history display of the audio input contents made so far, and a setting button 57 for performing various settings of the application software. The

  Here, FIG. 4 shows a flow of dividing the case (step SU2) when attention is paid to whether or not the user taps the voice call button 54 as the subsequent processing. In normal speech translation processing, Voice input can be performed from the standby screen shown in FIG. Here, the flow of the normal speech translation process (that is, “No” in step SU2) will be described first.

[Normal speech translation processing]
That is, when the user taps the Japanese input button 53a and selects Japanese voice input on this standby screen, the voice input is enabled. In this state, when the user utters a matter to be communicated to the interlocutor, voice input is performed through the voice input / output device 13 (step SJ3). The processor 11 of the information terminal 10 generates an audio signal based on the audio input, and transmits the audio signal to the server 20 through the communication interface 14 and the network N. As described above, the information terminal 10 itself, or the processor 11 and the voice input / output device 13 function as an “input unit”.

  The processor 21 of the server 20 receives the voice signal through the communication interface 22 and performs voice recognition processing (step SJ4). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech recognition module, Japanese speech corpus, acoustic model, language model, etc.) from the storage resource 23, and obtains “sound” of the input speech. Convert to "reading" (character). As described above, the processor 21 or the server 20 functions as a “voice recognition server” as a whole.

  Here, when the input voice is recognized, the processor 21 shifts to a multilingual translation process for translating the “reading” (characters) of the recognized voice into another language (step SJ5). At this time, the processor 21 calls the necessary module L20 and database D20 (translation module, Japanese character corpus, Japanese dictionary, English dictionary, Japanese-English bilingual dictionary, Japanese-English bilingual corpus, etc.) from the storage resource 23 and recognizes them. The resulting input speech “reading” (character string) is properly sorted and converted into Japanese phrases, clauses, sentences, etc., the English corresponding to the conversion result is extracted, and these are sorted according to the English grammar. To natural English phrases, clauses, sentences, etc. As described above, the processor 21 also functions as a “translation unit”, and the server 20 also functions as a “translation server” as a whole. If the input voice is not recognized well, the voice can be re-input (screen display is not shown).

  Further, the processor 21 stores the content of the recognized input voice in the storage resource 23. Next, when the multilingual translation processing and the input speech content storage processing are completed, the processor 21 proceeds to speech synthesis processing (step SJ6). At this time, the processor 21 calls the necessary module L20, database D20, and model M20 (speech synthesis module, English speech corpus, acoustic model, language model, etc.) from the storage resource 23, and the English phrase that is the translation result, Convert clauses, sentences, etc. to natural speech. As described above, the processor 21 also functions as a “speech synthesizer”, and the server 20 also functions as a “speech synthesizer” as a whole.

  Next, the processor 21 generates a voice signal for voice output based on the synthesized voice, and transmits the voice signal to the information terminal 10 through the communication interface 22 and the network N. The processor 11 of the information terminal 10 receives the audio signal through the communication interface 14, and performs an audio output process using the audio input / output device 13 (step SJ7). As described above, the processor 11 and the voice input / output device 13 function as an “output unit”.

[Processing by voice function]
On the other hand, when the user taps the voice button 54 on the standby screen shown in FIG. 5B (“Yes” in step SU2), the processor 11 of the information terminal 10 is prepared to talk to the conversation person. A command signal for displaying a list of a plurality of question fixed phrases is transmitted to the server 20. Upon receiving the command signal, the processor 21 of the server 20 reads a plurality of question fixed phrases stored and held in the storage resource 23 and displays, for example, a fixed phrase display screen shown in FIG. SJ8). On the standard sentence display screen, Japanese text T3 of a plurality of question standard sentences is displayed in a list in horizontal writing in the vertical direction on the screen, and the user taps the text part of each question standard sentence, so that a desired question standard sentence ( A specific fixed phrase) can be selected. As described above, the processors 11 and 21 and the display device 16 also function as a “standard sentence presentation unit”.

  In the fixed sentence display screen, the language selected as the language of the conversation person (that is, the bilingual language) is English above and below the list of the Japanese text T3 in the language selection screen of FIG. And a close button B2 for closing the fixed sentence display screen and returning to the standby screen of FIG. 5B.

  Next, from the Japanese text T3 of the question template sentence listed on the template sentence display screen, the user describes, for example, “How was it done?” As a desired question phrase sentence (specific phrase). When the Japanese text T5 is selected (“Yes” in step SU3), the command signal is transmitted from the processor 11 of the information terminal 10 to the server 20. Upon receiving the command signal, the processor 21 of the server 20 translates a plurality of languages (English, Chinese, and Korean) stored in the storage resource 23 in association with the content of the selected Japanese text T5. Is generated, and a text signal including the contents is generated and transmitted to the information terminal 10 through the communication interface 22 and the network N. The processor 11 of the information terminal 10 receives the text signal through the communication interface 14 and displays the translated text display screen shown in FIG. 5D on the display device 16 (step SJ9). As described above, the processors 11 and 21 and the display device 16 also function as a “translation sentence presentation unit”.

  On this translated text display screen, a plurality of foreign language texts T6 indicating translated texts in each language are displayed in a list in the vertical direction on the screen along with Japanese text T5 of a specific fixed phrase selected by the user. The user can ask the dialogue person "How did you do it?" In multiple languages by presenting the translated text display screen to the dialogue person. In the translated text display screen, below the foreign language text T6, a plurality of response buttons 58a and 58b (both are response means) for responding to the question are displayed by the processors 11 and 21. That is, the response buttons 58a described as “YES” and “Please” are for requesting the user for assistance or support, and the response buttons 58b described as “NO” and “Thank you” It is for expressing that the support or assistance by the user is unnecessary. The interlocutor can select one of the reply buttons 58a and 58b by tapping (step SU4). As described above, the processors 11 and 21 and the display device 16 also function as a “response means presenting unit”.

  For example, when the dialogue person can understand any of the foreign language texts T6 and needs assistance or support by the user, the dialogue person taps the “YES” reply button 58a (in step SU4). Select “YES”). Then, the language selection screen shown in FIG. 5A is redisplayed on the display device 16 (step SJ10), and at this point, the dialog person can select the language himself. When the language selection by the dialogue person is completed, a voice input standby screen shown in FIG. 5B is displayed on the display device 16. As described above, the English input button 53b is displayed in FIG. 5B. However, if the language selected by the conversation person is other than English, the selected language is used instead of the English input button 53b. The input button is displayed.

  Next, on the voice input standby screen shown in FIG. 5B, when the conversation person taps the English input button 53b (or another language input button) and selects the voice input, the voice input is enabled. . In this state, when the conversation person utters a matter to be communicated to the user (contents that require assistance or support) (step SJ3), it is the same as the above-mentioned [normal speech translation processing] (however, instead of Japanese) The speech recognition process (step SJ4), the multilingual translation process (step SJ5), the speech synthesis process (step SJ6), and the speech output process (step SJ7) are sequentially executed. The conversation between the user and the talker progresses.

  On the other hand, for example, when the conversation person can understand any of the foreign language texts T6 and the assistance or support by the user is unnecessary, the conversation person taps the “NO” reply button 58b (step SU4). "NO" selection). Then, the command signal is transmitted from the processor 11 of the information terminal 10 to the server 20. The processor 21 of the server 20 that has received the command signal determines that the voice input of the conversation person is unnecessary, and, for example, a standard sentence “Please Slowly” stored in the storage resource 23 as a reply to the conversation person. And a text signal including the contents of the translations in a plurality of languages is generated and transmitted to the information terminal 10 through the communication interface 22 and the network N.

  The processor 11 of the information terminal 10 receives the text signal through the communication interface 14 and displays a response display screen shown in FIG. 6A on the display device 16 (step SJ11). In this response display screen, along with the Japanese text T7 of the standard sentence “Please Slowly”, the illustration 61 representing the image of the contents and a plurality of foreign language texts T8 indicating the translations in each language are written horizontally. Listed in direction.

  In order to explain the contents of the voice function to the user, for example, the tutorial (explanation) screen shown in FIG. 6B may be displayed by transitioning from the appropriate screen described above. . For example, the help button H shown in FIG. 5D may be displayed on an appropriate screen, and when the user taps the help button H, the tutorial screen may be displayed. On this tutorial screen, for example, Japanese text T9 for explaining the content or outline of the voice call function and an illustration 62 representing an image of the content are displayed.

(Second Embodiment)
FIG. 7 is a flowchart illustrating an example of (a part of) a processing flow in the speech translation apparatus 100 according to the second embodiment. FIGS. 8A and 8B are plan views showing an example of (a part of) transitions in the screen display of the information terminal 10 according to the second embodiment. In the processing flow of the second embodiment, step SU5 shown in FIG. 7 is executed instead of step SU4 shown in FIG. 4 in the first embodiment, steps SJ10 and SJ11 shown in FIG. 4 are not executed, and The processing flow in the first embodiment, except that the screens shown in FIGS. 8A and 8B are displayed in place of the screens shown in FIGS. 5C and 5D in the first embodiment, respectively. Performed in an equivalent procedure.

  That is, here, in step SJ8 shown in FIG. 7, the standard sentence display screen shown in FIG. 8 (A) is displayed, and the user starts from the Japanese text T3 of the question fixed phrase listed on the standard sentence display screen. For example, the Japanese text T10 in which “How is Japan?” Is selected as a desired question template (specific template) (“Yes” in step SU3).

  Thereby, the command signal is transmitted from the processor 11 of the information terminal 10 to the server 20, and the processor 21 of the server 20 generates a text signal including the selected specific question fixed phrase and its translation in a plurality of languages, The data is transmitted to the information terminal 10 through the communication interface 22 and the network N. The processor 11 of the information terminal 10 receives the text signal through the communication interface 14 and displays the translated text display screen shown in FIG. 8B on the display device 16 (step SJ9).

  On the translated text display screen, a plurality of foreign language texts T11 indicating translated texts in each language are displayed in a list in the vertical direction on the screen along with the Japanese text T10 of a specific fixed phrase selected by the user. The user can ask the conversation person “How is Japan?” In a plurality of languages by presenting the translated text display screen to the conversation person. In addition, a plurality of response buttons 71 (response means) for displaying an intention to respond to the question are displayed so as to be adjacent to each of the foreign language texts T11 on the translated text display screen. Can be selected by tapping any of them.

  In response to this question, if the interlocutor can understand any of the foreign language texts T11 and give some response to the user, the interlocutor will be able to speak or understand the language (specific The reply button 71 in which the bilingual translation is written is tapped (step SU4). The processor 11 of the information terminal 10 transmits a command signal indicating that the response button 71 of the language has been selected to the processor 21 of the server 20, and the processor 21 speaks based on the command signal. Or, determine the language that can be understood.

  Then, the processor 21 displays the voice input standby screen shown in FIG. 5B in the first embodiment on the display device 16 of the information terminal 10. For example, when it is determined that the language of the conversation person is English, the standby screen including the Japanese input button 53a that is the user's language and the English input button 53b that is the language of the conversation person (that is, the screen of FIG. 5B). ) Is displayed. Similarly, when it is determined that the language of the conversation person is another language, a standby screen including a Japanese input button 53a and another language input button is displayed. Then, the subsequent processes, that is, voice input by a conversation person (step SJ3), voice recognition process (step SJ4), multilingual translation process (step SJ5), voice synthesis process (step SJ6), and voice output process (step) SJ7) is sequentially executed in the same manner as the above-mentioned [normal speech translation processing].

  According to the speech translation apparatus 100 configured as described above, the speech translation method using the speech translation apparatus, and the speech translation program, a plurality of question template sentences (Japanese text T3) prepared in advance for the user to speak to the conversation person. And translations in a plurality of languages (for example, foreign language texts T6 and T11) for each of the plurality of question template sentences are stored, and the user selects an appropriate “specific template sentence” from them. Can do. At that time, since the translation of the selected specific fixed sentence in a plurality of languages is displayed on the display device 16 of the information terminal 10, by presenting it to the conversation person, an opportunity for conversation with the conversation person is obtained. be able to. As a result, the user can easily talk to the interlocutor, and smooth communication by the conversation with the interlocutor becomes possible.

Further, as in the first embodiment, for example, after a translated text display screen on which translated texts in a plurality of languages for a specific fixed sentence (Japanese text T5) shown in FIG. A language selection screen shown in FIG. 5 ( A ) is displayed, and a desired language that can be spoken or understood by the interlocutor can be selected. Therefore, the language selection of the conversation person can be performed in a natural conversation flow.

  Further, as in the second embodiment, for example, on the translation display screen on which translations in a plurality of languages for the specific fixed sentence (Japanese text T10) shown in FIG. 8B are displayed, the foreign language text T11 of each translation is displayed. An answer button 71 for expressing that the question is answered is displayed so as to be adjacent. Then, when the conversation person taps and selects one of them, the language that the conversation person can speak or understand is discriminated and determined. This can be done in the flow.

  Note that, as described above, each of the above embodiments is an example for explaining the present invention, and is not intended to limit the present invention to the embodiment. The present invention can be variously modified without departing from the gist thereof. For example, those skilled in the art can replace the resources (hardware resources or software resources) described in the embodiments with equivalents, and such replacements are also included in the scope of the present invention.

  Moreover, although each said embodiment described the example which performs each process of speech recognition, translation, and a speech synthesis by the server 20, you may comprise so that these processes may be performed in the information terminal 10. FIG. In this case, the module L20 used for these processes may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. Furthermore, the database D20 that is a voice database and / or a model M20 such as an acoustic model may be stored in the storage resource 12 of the information terminal 10 or may be stored in the storage resource 23 of the server 20. . As described above, the speech translation apparatus may not include the network N and the server 20.

  Furthermore, for example, all or part of the translations (foreign language texts T6 and T11) in a plurality of languages listed on the translation display screens of FIGS. 5D and 8B may be read out by voice. In this case, the user or the conversation person may select the translated sentence to be output as voice, or may automatically read it out without selecting it. Furthermore, each of the translations (foreign language texts T6 and T11) in a plurality of languages listed on the translation display screens of FIGS. 5D and 8B also serves as the reply buttons 58a, 58b, and 71. It may be configured. In this case, it is also preferable that the processor 21 of the server 20 discriminates the language of the conversation person by tapping and selecting each of the translated sentences (foreign language texts T6 and T11).

  Of course, a gateway server for converting a communication protocol between the information terminal 10 and the network N may be interposed. The information terminal 10 is not limited to a portable device, and may be a desktop personal computer, a notebook personal computer, a tablet personal computer, a laptop personal computer, or the like.

  According to the present invention, it is possible to obtain an opportunity for conversation with a foreigner who is a conversation person, and to select or select a conversation person language in a natural conversation flow. It can be widely used for activities such as design, manufacture, provision, and sales of programs, devices, systems, and methods in the field of providing services related to conversations between people who cannot understand each other's languages.

10 information terminal 11 processor 12 storage resource 13 voice input / output device 14 communication interface 15 input device 16 display device 17 camera 20 server 21 processor 22 communication interface 23 storage resource 51 illustration 52 language button 53a Japanese input button 53b English input button 54 Voice button 55 Language selection button 56 History button 57 Setting button 58a, 58b Reply button 61 Illustration 62 Illustration 71 Reply button 100 Speech translation device B1 Cancel button B2 Close button D20 Database H Help button L20 Module M20 Model N Network P10 Program P20 Program T1 English text T2 Japanese text T3 Japanese text T4 Japanese text T5 Japanese text T6 Foreign language text Kist T7 Japanese text T8 Foreign language text T9 Japanese text T10 Japanese text T11 Foreign language text

Claims (6)

A speech translation apparatus comprising: an input unit for inputting a user's speech; a translation unit that translates the content of the input speech into content of a different language; and an output unit that outputs the translation content of the input speech as speech and / or text Because
A storage unit for storing a plurality of question template sentences and translations in a plurality of languages for each of the plurality of question template sentences;
For the user to select a specific questions fixed phrase from the multiple question fixed text, and fixed text presentation unit for displaying a question boilerplate plurality of,
When the specific question template sentence is selected, a translated sentence presenting unit that displays a translation of the specific question template sentence in the plurality of languages;
Response means for presenting a response means for the interrogator to respond to the specific question template after the display of the translation in the plurality of languages for the specific question template, or together with the display of the translation A presentation unit;
A language selection means presenting unit for displaying a language selection means for the conversation person to select a desired language from among the plurality of languages after the conversation person operates the response means ;
A voice input standby screen display unit that displays a standby screen for voice input in the language of the user and the language of the dialog person when the dialog person selects the desired language;
A speech translation device further comprising: A speech translation apparatus comprising: an input unit for inputting a user's speech; a translation unit that translates the content of the input speech into content of a different language; and an output unit that outputs the translation content of the input speech as speech and / or text Because
A storage unit for storing a plurality of question template sentences and translations in a plurality of languages for each of the plurality of question template sentences;
In order for the user to select a specific question template sentence from the plurality of question template sentences, a template sentence presentation unit that displays the plurality of question template sentences;
When the specific question template sentence is selected, a translated sentence presenting unit that displays a translation of the specific question template sentence in the plurality of languages;
After each of the translations in the plurality of languages with respect to the specific question template, or together with the display of the translation, each of the above-mentioned dialogues for indicating the intention to respond to the specific question template A reply means presenting section for presenting reply means in the language of the translation of
After the conversation person operates the reply means, the language corresponding to the reply means is determined as the language of the talker, and a standby screen for voice input in the language of the user and the language of the talker is displayed. Voice input standby screen display
A speech translation device further comprising: The output unit outputs all or part of the translated sentences in the plurality of languages for the specific question fixed phrase by voice;
The speech translation apparatus according to claim 1 or 2. The translations in the plurality of languages corresponding to the specific question fixed phrase also serve as the function of the response means,
The voice input standby screen display unit determines that the language corresponding to the translated sentence is the language of the conversation person after the conversation person operates the translated sentence, and depends on the language of the user and the language of the conversation person Display the voice input standby screen,
The speech translation apparatus according to any one of claims 1 to 3. Using a speech translation device including an input unit, a translation unit, an output unit, a storage unit, a fixed phrase presentation unit, a translation presentation unit, a response unit presentation unit, a language selection unit presentation unit, and a voice input standby screen display unit ,
The input unit inputs a user's voice;
The translation unit translating the content of the input speech into content of a different language;
The output unit outputting the translated content in speech and / or text;
The storage unit stores a plurality of question template sentences and translations in a plurality of languages for each of the plurality of question template sentences;
The typical sentence presentation unit is, for the user to select a specific questions fixed phrase from the multiple question fixed sentence, and displaying the questions boilerplate plurality of,
The translated sentence presenting unit, when the specific question template sentence is selected, displaying the translated sentences in the plurality of languages for the specific question template sentence;
The response means presenting section is for the dialog person to reply to the specific question template after the display of the translation in the plurality of languages with respect to the specific question template, or together with the display of the translation Presenting a response means;
The language selection means presenting unit displaying a language selection means for the conversation person to select a desired language from the plurality of languages after the conversation person operates the response means ;
The voice input standby screen display unit displaying a standby screen for voice input in the language of the user and the language of the conversation when the conversation person selects the desired language;
Speech translation method including Computer
An input unit for inputting the user's voice;
A translation unit that translates the content of the input speech into a different language
An output unit that outputs the translated content in audio and / or text;
A storage unit for storing a plurality of question template sentences and translations in a plurality of languages for each of the plurality of question template sentences;
For the user to select a specific questions fixed phrase from the multiple question fixed text, and fixed text presentation unit for displaying a question boilerplate plurality of,
When the specific question template sentence is selected, a translated sentence presenting unit that displays a translation of the specific question template sentence in the plurality of languages;
Response means for presenting a response means for the interrogator to respond to the specific question template after the display of the translation in the plurality of languages for the specific question template, or together with the display of the translation A presentation unit;
A language selection means presenting unit for displaying a language selection means for the conversation person to select a desired language from among the plurality of languages after the conversation person operates the response means ;
A voice input standby screen display unit that displays a standby screen for voice input in the language of the user and the language of the dialog person when the dialog person selects the desired language;
Speech translation program that works

Images