EP4506924A1 - Method for receiving and processing audio data and triggering of associated sound effects according to prosody and/or movement - Google Patents

Abstract

The invention relates to a method for receiving and processing audio data comprising words corresponding to the real-time reading of a source text for triggering sound effects synchronized with said reading of the text, characterized in that it comprises a step (110) of determining a position index (12) of the speaker in the source text, by detecting a correspondence between the received audio data and the source text, a step (112) of receiving at least one piece of data representative of a prosody piece of data (10), and/or of receiving at least one piece of movement data, a step (140) of determining the sound effect to be triggered as a function of the position index (12), and as a function of the prosody piece of data and/or the movement piece of data, a step (150) of triggering the determined sound effect.

Description

Technical field of the invention

The invention relates to a method for receiving and processing audio data, and to a system for receiving and processing audio data. In particular, the invention relates to a method for triggering sound effects associated with a source text during reading of the source text by a human speaker, taking into account the prosody of the reading and/or movement during reading.

Technological background

The invention is placed in the field of reading and proposes to accompany the reading of a book by triggering sound effects linked to the source text which is read.

Various interactive reading techniques have already been proposed to allow the triggering of sound effects during reading.

Some techniques, for example, provide playback speed estimation to allow sound effects to be triggered and preloaded at the appropriate time. These techniques are simple, but the risk of triggering a sound effect at the wrong time is high.

Other techniques rely on recognizing the reading of predetermined keywords in the text, to trigger the associated sound effect. However, these word detection techniques are more subject to recognition and reading tracking errors.

Furthermore, current techniques generally only offer one sound effect intended for each passage of the text and do not allow for variation the reading experience with each new reading aloud of the source text. They also do not allow the sound effect to be adapted to the speaker's emotions and reading style. They also do not allow sound effects to be triggered when the speaker moves the book for interaction with the book.

In particular, the sound effects used are based on a principle of "linear music" which uses sounds played in a loop and a transition from one loop to another by cross-fading at the end of predetermined sequences.

The inventors have thus sought to provide a method overcoming these drawbacks, by allowing the integration of a method similar to the principle of adaptive music when reading a text, in particular a book.

Objectives of the invention

The invention aims to provide a method, a computer program product and a system for receiving and processing audio data comprising words corresponding to the real-time reading of a source text by a speaker, for triggering sound effects synchronized with said reading of the text.

The invention aims in particular to provide, in at least one embodiment, a method, a computer program product and a system for receiving and processing audio data allowing the taking into account of interactions and variation of emotions when reading the text, for a better renewal of the reading experience, in particular to adapt the sound effect to the emotion that the speaker wants to transmit.

The invention also aims to provide, in at least one embodiment, a method, a computer program product and a system for receiving and processing audio data allowing better monitoring of the reading of the source text to guarantee appropriate triggering of the sound effects at the right time.

The invention also aims to provide, in at least one embodiment, a method, a computer program product and a system for receiving and processing audio data which can be embedded in a portable device of reduced size and which can operate without an Internet connection.

Disclosure of the invention

• une étape de détermination d'un index de position du locuteur dans le texte source, par détection d'une correspondance entre les données audios réceptionnées et le texte source enregistré dans un module de stockage du système de réception et de traitement,
• une étape de réception d'au moins une donnée représentative d'une valeur de prosodie des données audios, dite donnée de prosodie, et/ou de réception d'au moins une donnée représentative de la présence ou de l'absence d'un mouvement d'un dispositif de reconnaissance de mouvement durant la lecture du texte et représentative de caractéristiques d'un mouvement présent, dite donnée de mouvement,
• une étape de détermination de l'effet sonore à déclencher en fonction de l'index de position, et en fonction de la donnée de prosodie et/ou de la donnée de mouvement,
• une étape de déclenchement de l'effet sonore déterminé.

To this end, the invention relates to a method for receiving and processing, in a receiving and processing system, audio data comprising words corresponding to the real-time reading of a source text by at least one speaker, for triggering sound effects synchronized with said reading of the text,
characterized in that it comprises:

• a step of determining a position index of the speaker in the source text, by detecting a correspondence between the received audio data and the source text recorded in a storage module of the reception and processing system,
• a step of receiving at least one piece of data representative of a prosody value of the audio data, called prosody data, and/or of receiving at least one piece of data representative of the presence or absence of a movement of a movement recognition device during the reading of the text and representative of characteristics of a present movement, called movement data,
• a step of determining the sound effect to be triggered based on the position index, and based on the prosody data and/or the movement data,
• a step of triggering the determined sound effect.

A reception and processing method according to the invention therefore allows prosody and/or movement data to be taken into account to determine whether a sound effect should be triggered when the position index corresponds to a sound effect triggering position. Taking this additional data into account allows for a renewal of the reading experience and a more immersive and personalized reading experience, a sound effect being able to be triggered or selected, or modified or not by taking into account the reading variations. The objective is to adapt the sound effect to the emotion that the speaker of the source text wants to convey.

The sound effect triggering is thus adaptive, and can evolve over time real at each position index based on movements and/or prosody, which are respectively representative of actions and emotions. The operation is thus similar to the principle of adaptive music, used in particular in video games where the music changes dynamically depending on the player's activity. The sound effects are divided into fragments and several layers which can be selected and possibly combined depending on the movements and prosody.

In particular, taking into account the prosody of reading allows to recognize the emotions of the speaker, for example adapting the sound effects triggered to the sound intensity of reading, to the speed of reading, to the tone of reading, etc. A combination of these characteristics of the words constituting the audio data constitutes prosody. More generally, prosody is defined as the set of non-verbal characteristics of a spoken word, in particular the variations of flow, pitch (tone and intonation) and variation of duration (accentuation and rhythm), which can be representative of an emotion transmitted in the words when reading the text. The same text can transmit different emotions depending on the prosody of the speech during its reading by a speaker without considering the meaning of the words and sentences pronounced. Prosody can also characterize a sociolinguistic accent such as a regional accent and can also add context and meaning to a reading.

Prosody is detected and recognized using at least one preconfigured dedicated algorithm, for example implementing at least one pre-trained learning model with annotated prosody data, or for example using pre-determined prosody thresholds, adjusted using speaker-specific audio data. The algorithm(s) take into account the combination of prosody characteristics of the audio data to determine a prosody of the audio data, which may be associated with an emotion conveyed when reading the text. The method may also implement a learning model selected from a group of learning models based on the speaker's gender and/or estimated age and/or the language spoken by the speaker, each learning model of the group being trained with data at least partly different associated with a particular gender, age group and/or language.

The algorithm is preferably fast to execute and self-contained, particularly requiring few computing resources and not requiring a connection to the Internet or any external database.

Taking motion into account makes it possible to associate physical interactions with the reading on the motion recognition device during the reading of the text, and to be able to link the reading and the motion to the triggering of a sound effect. The motion data may be data indicating the absence of motion. The motion data may also include information representative of the speed and/or acceleration of the motion.

The motion is detected and recognized by using at least one pre-configured dedicated algorithm, for example implementing at least one pre-trained learning model with annotated motion data, or for example using pre-determined motion models, adjusted using the speaker-specific motion data and/or relative to the initial position of the motion recognition device. The at least one algorithm takes into account the combination of motion features detected by the motion recognition device to determine the recognized motion.

Advantageously and according to the invention, the source text is recorded in the processing device in the form of a list of phonemes, and the processing device is configured to detect in the audio data the presence of a phoneme corresponding to the source text.

According to this aspect of the invention, the decomposition of the text into phonemes makes it possible to improve the accuracy of the position index as well as the calculation speed and thus to improve the triggering conditions. The phoneme detection allows a better result than a word detection and is more compatible with choppy reading, a rewind in reading, pauses in reading, etc.

Furthermore, phoneme detection requires fewer resources than word detection and thus can be implemented by a portable and/or embedded device, even in the absence of an Internet connection. In particular, the number of different phonemes to be detected is much lower than the number of words that can be detected in methods using word detection and phoneme detection does not require deduction of the next word. The phoneme detection speed is thus of the order of 200ms.

• une sous-étape de réception de la position actuelle de l'index dans la liste des phonèmes du texte source,
• une sous-étape de comparaison d'au moins un phonème détecté dans les données audios avec au moins un phonème attendu parmi les phonèmes suivants dans la liste de phonèmes du texte source,
• si aucun phonème détecté dans les données audios ne correspond avec les phonèmes suivants, une sous-étape de réception d'une pluralité de phonèmes détectés dans les données audios et une sous-étape de recherche dans au moins une partie du texte source d'une séquence de phonèmes dans la liste de phonèmes du texte source correspondant à ladite pluralité de phonèmes détectés,
• si la pluralité de phonèmes détectés dans les données audios ne correspondant avec aucune séquence de phonèmes dans la liste de phonèmes du texte source, une sous-étape de recherche dans la totalité du texte source d'une séquence de phonèmes dans la liste de phonèmes du texte source correspondant à ladite pluralité de phonèmes détectés,
• en cas de détection d'une correspondance d'un phonème ou d'une pluralité de phonème détectés avec la liste de phonèmes du texte source, une sous-étape de mise à jour de l'index avec ledit phonème correspondant ou le dernier phonème de la séquence de phonèmes correspondante.

Advantageously and according to the invention, the position index of the speaker corresponds to a phoneme from the list of phonemes of the source text, and the step of determining the position index of the speaker in the source text comprises:

• a sub-step of receiving the current position of the index in the list of phonemes of the source text,
• a sub-step of comparing at least one phoneme detected in the audio data with at least one phoneme expected among the following phonemes in the phoneme list of the source text,
• if no phoneme detected in the audio data matches with the following phonemes, a sub-step of receiving a plurality of phonemes detected in the audio data and a sub-step of searching in at least a part of the source text for a sequence of phonemes in the phoneme list of the source text corresponding to said plurality of detected phonemes,
• if the plurality of phonemes detected in the audio data do not correspond to any phoneme sequence in the phoneme list of the source text, a sub-step of searching in the entire source text for a phoneme sequence in the phoneme list of the source text corresponding to said plurality of detected phonemes,
• in case of detection of a correspondence of a phoneme or of a plurality of detected phonemes with the list of phonemes of the source text, a sub-step of updating the index with said corresponding phoneme or the last phoneme of the corresponding sequence of phonemes.

According to this aspect of the invention, tracking the correspondence of phonemes as described allows efficient determination of the position index for triggering sound effects.

The substep of searching at least a portion of the source text for a sequence of phonemes may be repeated repeatedly by gradually expanding the size of the portion of the source text in which the plurality of detected phonemes are searched.

The sub-step of searching the entire source text for a sequence of phonemes can also be performed continuously and in parallel with the other sub-steps, so as to provide a result quickly if the previous sub-steps have not been conclusive and/or if the confidence index of the correspondence of the sequence of phonemes with the plurality of detected phonemes is high.

Advantageously, if no phoneme detected in the audio data matches the following phonemes, the step of determining the speaker's position index in the source text also includes a step of advancing the index to the next phoneme. By means of this step, the method can continue to advance in the list of phonemes to avoid losing the correspondence between the audio data and the list of phonemes.

Advantageously and according to the invention, the comparison between the audio data and the list of phonemes comprises a determination of an index representative of the resemblance between the phoneme detected in the audio data and the phoneme of the list of phonemes. This index representative of the resemblance is also representative of the distance between these phonemes according to its value, for example a value equal to zero indicates a perfect match, a value close to zero indicates a resemblance and a higher positive value indicates a distance between the phonemes. For example, in the French language, the phonemes beginning with the letters "p" and "b" are considered to be similar and therefore with a low distance, in the same way as the letters "t" and "d". If the value of the index is lower than a predetermined threshold, it can be considered that there is a match of the phonemes and the index can be advanced. In the variants of the invention where a confidence index as described below is implemented work, this confidence index can be lowered if the correspondence of the phonemes is not perfect (non-zero resemblance index but lower than the predetermined threshold). Thus, advantageously and according to the invention, the sub-step of comparing at least one phoneme detected in the audio data with at least one phoneme expected among the following phonemes in the list of phonemes of the source text comprises a determination of an index representative of the resemblance between the phoneme detected in the audio data and the phoneme in the list of phonemes, and if the index representative of the resemblance is lower than a predetermined threshold, a sub-step of updating the index with said detected phoneme and a sub-step of reducing the confidence index.

Each of these sub-steps has its own error tolerance, for example whether or not to allow the substitution of a phoneme with a similar phoneme, or has a threshold for the percentage of correspondence between a list of detected phonemes and a list of phonemes in the source text.

Advantageously and according to the invention, the step of triggering the sound effect comprises a sub-step of verifying the value of a confidence index and in that the sound effect is triggered only if the confidence index is greater than a predetermined threshold.

According to this aspect of the invention, the confidence index thus makes it possible to avoid triggering a sound effect in the event of a lack of confidence in the position tracking. In particular, this makes it possible to avoid triggering a sound effect at an inopportune moment due to an incorrect determination of the position index.

Each sound effect can be tied to its own predetermined threshold.

Advantageously and according to the invention, the method comprises a step of increasing the value of the confidence index in the event of detection of a correspondence of a phoneme or of a plurality of phonemes detected with the list of phonemes of the source text and comprises a step of decreasing the value of the confidence index if no phoneme detected in the audio data corresponds with the following phonemes.

According to this aspect of the invention, the increase and decrease of the confidence index are linked to the detection of phoneme correspondence, in particular the confidence index thus makes it possible to avoid the triggering of sound effect if the detection of phoneme correspondence is not carried out with sufficient confidence.

• une sous-étape de récupération, en fonction de l'index de position, du groupe d'effets sonores associé à la séquence temporelle dans laquelle se trouve l'index de position,
• une sous-étape de détermination, en fonction de la donnée de prosodie et/ou de la donnée de mouvement, de l'effet sonore à déclencher parmi les effets sonores du groupe d'effets sonores.

Advantageously and according to the invention, all of the sound effects associated with the source text are distributed in a plurality of time sequences associated with portions of the source text, each time sequence being associated with a sound effect or a group of sound effects comprising several sound effects, and when the position index corresponds to a time sequence associated with a group of sound effects, the step of determining the sound effect to be triggered comprises:

• a sub-step of retrieving, as a function of the position index, the group of sound effects associated with the time sequence in which the position index is located,
• a sub-step of determining, based on the prosody data and/or the motion data, the sound effect to be triggered among the sound effects of the sound effect group.

According to this aspect of the invention, the time sequences are each associated with one or more sound effects and the triggered sound effects are adapted to the determined prosody and/or movement data. The objective is to propose, for at least certain time sequences, sound effect variants to be triggered for the same position index.

Advantageously and according to the invention, the sound effects that can be triggered in each sound effect group can also depend on parameters predefined in advance by the speaker, or can vary depending on the prosody data and the confidence index in the associated variants.

For example, the speaker can select a "day" or "night" mode that prevents certain sound effects from being triggered in the sound effect group. Taking into account the confidence index allows also to limit the risks of untimely triggering of sound effects at the wrong time, for example by delaying the triggering and/or not triggering a specific sound effect.

The speaker can also enable or disable motion or prosody detection for reading.

• des données sur l'intensité des paroles des données audios,
• des données sur la fréquence et/ou la fréquence fondamentale des paroles des données audios,
• des données sur l'intensité et/ou de fréquence et/ou de fréquence fondamentale des consonnes prononcées dans les données audios,
• des données sur l'intensité et/ou de fréquence et/ou de fréquence fondamentale des voyelles prononcées dans les données audios,
• des données sur la longueur des voyelles et/ou des paroles prononcées dans les données audios,
• des données sur la longueur des consonnes prononcées dans les données audios,
• des données sur le débit de parole des données audios.

Advantageously and according to the invention, the prosody data comprises data or a combination of data from different types of data in the following list:

• data on the intensity of speech in audio data,
• data on the frequency and/or fundamental frequency of the speech of the audio data,
• data on the intensity and/or frequency and/or fundamental frequency of consonants pronounced in the audio data,
• data on the intensity and/or frequency and/or fundamental frequency of vowels pronounced in the audio data,
• data on the length of vowels and/or spoken words in the audio data,
• data on the length of consonants pronounced in audio data,
• data on the speech rate of audio data.

According to this aspect of the invention, the prosody depends on several factors that make it possible to symbolize the emotion of the speaker of the source text during the reading aloud of the text. The prosody data can be based on an absolute value of each data of the list or on a temporal variation of the values thereof, for example a gradual increase or decrease in sound intensity.

Advantageously and according to the invention, the method comprises a calibration step making it possible to determine or estimate, from the audio data, the gender of the speaker and/or the age of the speaker and/or the language spoken by the speaker.

According to this aspect of the invention, determining or estimating the age or gender of the speaker, as well as the language spoken, can allow the thresholds to be adjusted. prosody variation detection to better adapt sound effect triggering. In particular, a man or a woman, an adult and a child, or two speakers of different languages generally have different average prosody variations.

Advantageously and according to the invention, the movement data comprises data relating to a movement and/or a rotation in space and/or to the speed of movement of the movement recognition device, and/or data relating to the correspondence of a movement or a combination of movements detected by the movement recognition device with a predetermined movement from a list of predetermined movements recorded in a storage module of the processing system.

According to this aspect of the invention, different types of movement may be taken into account for the selection of the sound effect to be triggered. In particular, the method may comprise detecting a predetermined movement, for example if the user makes a circle with the motion recognition device.

• translation verticale,
• translation latérale,
• translation longitudinale,
• tangage,
• lacet,
• roulis
• mouvement de secousse,
• chocs brefs.

Movement in space is known to consist of one or a combination of the following movements:

• vertical translation,
• lateral translation,
• longitudinal translation,
• pitch,
• lace,
• roll
• shaking movement,
• short shocks.

The speed and/or acceleration of the movement can also be integrated into the motion data.

Advantageously and according to the invention, the sound effect triggering step is executed when the position index is at an equal position or after a predetermined trigger index, or within a predetermined trigger window, and in that the sound effect can be immediately emitted as soon as the position index reaches or exceeds the trigger index or reaches the trigger window, or emitted in a delayed manner after the end of a sound effect currently being emitted.

According to this aspect of the invention, the sound effect triggering may be immediate or delayed, for example the determined sound effect may be triggered at the end of a time sequence. The temporality of the triggering depends in particular on the position or trigger window in which the position index is located.

The sound effect can also be triggered at an absolute time point (e.g. at time t) or relative time point (e.g. in x seconds or depending on the tempo, e.g. after the last note of a measure of a piece of music). The sound effect can also be programmed to be triggered multiple times, e.g. once immediately and once x seconds later.

The invention also relates to a computer program product for receiving and processing audio data corresponding to the real-time reading of a source text by at least one speaker, for triggering sound effects synchronized with said reading of the text, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computing device, the steps of the method according to the invention.

The computer program product is advantageously stored in the processing device, in particular in a portable computing device, preferably a smartphone, a digital tablet or a smart watch, for example in the form of an application.

• un module de stockage du texte source,
• un module de réception des données audios,
• un module de traitement des données audio, configuré pour la détermination d'un index de position du locuteur dans le texte source, par détection d'une correspondance entre les données audios réceptionnées et le texte source enregistré dans le module de stockage,
• un dispositif de reconnaissance de mouvement configuré pour fournir au moins une donnée représentative de la présence ou de l'absence d'un mouvement d'un dispositif de reconnaissance de mouvement durant la lecture du texte et représentative de caractéristiques d'un mouvement présent, dite donnée de mouvement, et/ou un module d'analyse de la variation de la prosodie des données audios configuré pour fournir au moins une donnée représentative d'une variation de prosodie des données audios
• un module de détermination de l'effet sonore à déclencher en fonction de l'index de position, et en fonction de la donnée de prosodie et/ou de la donnée de mouvement,
• un dispositif d'émission sonore configuré pour l'émission de l'effet sonore déterminé.

The invention also relates to a system for receiving and processing audio data corresponding to the real-time reading of a source text by at least one speaker, for triggering sound effects synchronized with said reading of the text, characterized in that it includes:

• a source text storage module,
• an audio data reception module,
• an audio data processing module, configured to determine a speaker position index in the source text, by detecting a correspondence between the received audio data and the source text recorded in the storage module,
• a motion recognition device configured to provide at least one piece of data representative of the presence or absence of a movement of a motion recognition device during the reading of the text and representative of characteristics of a present movement, called motion data, and/or a module for analyzing the variation in the prosody of the audio data configured to provide at least one piece of data representative of a variation in the prosody of the audio data
• a module for determining the sound effect to be triggered based on the position index, and based on the prosody data and/or the movement data,
• a sound emitting device configured to emit the determined sound effect.

Advantageously, the reception and processing system according to the invention is configured to implement the reception and processing method according to the invention.

Advantageously and according to the invention, the reception and processing method according to the invention is configured to be implemented by a reception and processing system according to the invention.

Advantageously and according to the invention, the system comprises a portable computer device, preferably a smartphone, a digital tablet or a smart watch , comprising the source text storage module, the audio data reception module, the audio data processing module, the text recognition device, the audio data processing module ... movement and/or prosody variation analysis module, sound effect determination module and sound emission device.

In accordance with this aspect of the invention, most or all of the components of the system may be integrated into a portable computing device such as a smart phone (commonly referred to as a smartphone ), so as to combine the functionality in a single system. The portable computing device may also display the source text or preferably be attached to a physical medium of the source text, particularly where the source text is printed, such as a book.

The computing device is configured to implement the reception and processing method according to the invention in particular thanks to a combination of one or more computing components such as a processor (CPU for Central Processing Unit in English), and/or a graphics processor (GPU for Graphics Processing Unit in English), and/or a digital signal processor (DSP for Digital Signal Processor in English), and/or one or more memories, and/or an analog/digital converter, a microphone, an accelerometer, a gyroscope, etc.

Advantageously and according to the invention, the system comprises a motion recognition device and a device for attaching the motion recognition device, configured to allow the motion recognition device to be attached to a medium on which the source text is printed, so that the motion recognition device is mechanically secured to said medium when reading the medium.

According to this aspect of the invention, the motion recognition device can be directly associated with the physical medium on which the source text is printed, such that a movement of the physical medium causes a movement of the motion recognition device. Thus, a movement of the physical medium is detected and is taken into account in the motion data.

When the motion recognition device is integrated into a portable computing device, the attachment device is configured to attach the portable computing device to the physical medium.

• des éléments élastiques reliés au support sur lequel est imprimé le texte source, configurés pour un maintien en position du dispositif de reconnaissance de mouvement,
• une poche agencée dans la couverture dudit support, configurée pour accueillir le dispositif de reconnaissance de mouvement,
• un compartiment agencé dans la couverture dudit support, configurée pour accueillir le dispositif de reconnaissance de mouvement,
• un aimant permanent agencé sur ledit support et configuré pour une aimantation avec un élément magnétique agencé sur le dispositif de reconnaissance de mouvement, ou un aimant permanent agencé sur le dispositif de reconnaissance de mouvement et configuré pour une aimantation avec un élément magnétique agencé sur ledit support,
• une pochette clipsée sur ledit support, configurée pour accueillir le dispositif de reconnaissance de mouvement.

Advantageously and according to the invention, the attachment device is formed from an element or a combination of elements from the elements in the following list:

• elastic elements connected to the support on which the source text is printed, configured to hold the motion recognition device in position,
• a pocket arranged in the cover of said support, configured to accommodate the motion recognition device,
• a compartment arranged in the cover of said support, configured to accommodate the motion recognition device,
• a permanent magnet arranged on said support and configured for magnetization with a magnetic element arranged on the motion recognition device, or a permanent magnet arranged on the motion recognition device and configured for magnetization with a magnetic element arranged on said support,
• a pocket clipped onto said support, configured to accommodate the motion recognition device.

According to this aspect of the invention, these different attachment variants allow compatibility with different types of motion recognition device, in particular when the latter is integrated into a portable computing device.

The invention also relates to a receiving and processing method, a computer program product and a receiving and processing system characterized in combination by all or part of the features mentioned above or below.

List of figures

• [Fig. 1] est une vue schématique d'un procédé de réception et de traitement de données audios selon un mode de réalisation de l'invention,
• [Fig. 2] est une vue schématique d'un texte source et d'une liste de phonèmes tels qu'enregistrés dans un module de stockage d'un système de réception et de traitement selon un mode de réalisation de l'invention,
• [Fig. 3] est un graphique illustrant schématiquement une variation de prosodie de données audios traitées par un procédé de réception et de traitement selon un mode de réalisation de l'invention,
• [Fig. 4] est un graphique illustrant schématiquement trois pistes audios comprenant des effets sonores pouvant être déclenchés lors de l'exécution d'un procédé de réception et de traitement selon un mode de réalisation de l'invention,
• [Fig. 5] est une vue schématique d'un système de réception et de traitement de données audios selon un mode de réalisation de l'invention et selon une vue de face et de dos d'un livre formant support du texte source.

Other aims, characteristics and advantages of the invention will appear on reading the following description given solely for non-limiting purposes and which is refers to the attached figures in which:

• [ Fig. 1 ] is a schematic view of a method for receiving and processing audio data according to one embodiment of the invention,
• [ Fig. 2 ] is a schematic view of a source text and a list of phonemes as recorded in a storage module of a reception and processing system according to one embodiment of the invention,
• [ Fig. 3 ] is a graph schematically illustrating a variation in prosody of audio data processed by a reception and processing method according to an embodiment of the invention,
• [ Fig. 4 ] is a graph schematically illustrating three audio tracks comprising sound effects that can be triggered when executing a receiving and processing method according to one embodiment of the invention,
• [ Fig. 5 ] is a schematic view of a system for receiving and processing audio data according to one embodiment of the invention and according to a front and back view of a book forming a support for the source text.

Detailed description of an embodiment of the invention

In the figures, scales and proportions are not strictly respected, for the purposes of illustration and clarity.

Furthermore, identical, similar or analogous elements are designated by the same references throughout the figures.

There figure 1 schematically represents a method 100 for receiving and processing audio data according to an embodiment of the invention. The method allows the reception and processing of audio data comprising words corresponding to the real-time reading of a source text by a speaker, for the triggering of sound effects synchronized with said reading of the text. The method is here implemented in a system for receiving and processing audio data, an embodiment of which is described below with reference to the figure 5 .

The method comprises a step 110 of determining a position index of the speaker in the source text, by detecting a correspondence between the received audio data and the source text recorded in a module of storage of the receiving and processing system.

The source text is recorded in the processing device in the form of a list of phonemes, and the processing device is configured to detect in the audio data the presence of a phoneme corresponding to the source text. The list of phonemes is created beforehand, automatically, manually or semi-automatically (for example with manual correction of an automatic pre-processing). The creation of the list of phonemes is generally carried out outside the system implementing the method according to the invention.

le t∫ æt gr im pa d æ nz la r b re » et l'index attribué à chaque phonème associé à cette phrase exemple, l'index étant compris dans cet exemple entre 1 et 19. D'autres numérotations d'index peuvent être utilisées, par exemple le premier index peut correspondre à un index zéro, etc.There figure 2 schematically represents an example 200 of a sentence in French from the source text as recorded in the storage module "In a flash, the cat climbed the tree.", its transposition into phonemes "εn Λn é k le

the t∫ æt gr im pa d æ nz la rb re » and the index assigned to each phoneme associated with this example sentence, the index being in this example between 1 and 19. Other index numberings can be used, for example the first index can correspond to a zero index, etc.

• une sous-étape 112 de réception de la position actuelle de l'index dans la liste des phonèmes du texte source. Au départ, l'index de position correspond à l'index du premier phonème, par exemple l'index 1 dans la phrase exemple de la figure 2.
• une sous-étape 114 de comparaison d'au moins un phonème détecté dans les données audios avec au moins un phonème attendu parmi les phonèmes suivants dans la liste de phonèmes du texte source. Les phonèmes suivants dans la liste des phonèmes de la phrase exemple sont les phonèmes « Λn », « é », « k » et « le ».
• si aucun phonème détecté dans les données audios ne correspond avec les phonèmes suivants, une sous-étape 116 de réception d'une pluralité de phonèmes détectés dans les données audios et une sous-étape 118 de recherche dans au moins une partie du texte source d'une séquence de phonèmes dans la liste de phonèmes du texte source correspondant à ladite pluralité de phonèmes détectés. Par exemple, dans la phrase exemple de la figure 2, les phonèmes « le t∫ æt gr im pa » sont détectés dans les données audios et l'index de position se place alors à la fin de cette suite de phonèmes dans le texte source, c'est-à-dire à l'index 12 qui correspond au dernier phonème de la suite.

• si la pluralité de phonèmes détectés dans les données audios ne correspondant avec aucune séquence de phonèmes dans la liste de phonèmes du texte source, une sous-étape 120 de recherche dans la totalité du texte source d'une séquence de phonèmes dans la liste de phonèmes du texte source correspondant à ladite pluralité de phonèmes détectés. Cette sous-étape permet la recherche d'une séquence de phonèmes dans l'ensemble du texte source, par exemple les phrases suivant la phrase exemple ou les phrases précédant la phrase exemple, non visibles sur les figures.
• en cas de détection d'une correspondance d'un phonème ou d'une pluralité de phonème détectés avec la liste de phonèmes du texte source, une sous-étape 122 de mise à jour de l'index avec ledit phonème correspondant ou le dernier phonème de la séquence de phonèmes correspondante. Cette étape finale du cycle permet la mise à jour de l'index en fonction de la correspondance qui a été établie dans les sous-étapes précédentes. L'index mis à jour est récupéré lors d'une nouvelle exécution de la sous-étape 112 de réception de la position actuelle de l'index.

The speaker position index corresponds to the index of the corresponding phoneme in the phoneme list of the source text, and step 110 of determining the speaker position index in the source text comprises:

• a substep 112 of receiving the current position of the index in the list of phonemes of the source text. Initially, the position index corresponds to the index of the first phoneme, for example index 1 in the example sentence of the figure 2 .
• a substep 114 of comparing at least one phoneme detected in the audio data with at least one phoneme expected among the following phonemes in the list of phonemes of the source text. The following phonemes in the list of phonemes of the example sentence are the phonemes “Λn”, “é”, “k” and “le”.
• if no phoneme detected in the audio data matches with the following phonemes, a sub-step 116 of receiving a plurality of phonemes detected in the audio data and a sub-step 118 of searching in at least a part of the source text for a sequence of phonemes in the source text phoneme list corresponding to said plurality of detected phonemes. For example, in the example sentence of the figure 2 , the phonemes "le t∫ æt gr im pa" are detected in the audio data and the position index is then placed at the end of this sequence of phonemes in the source text, that is to say at index 12 which corresponds to the last phoneme of the sequence.

• if the plurality of phonemes detected in the audio data do not correspond to any phoneme sequence in the phoneme list of the source text, a sub-step 120 of searching in the entire source text for a phoneme sequence in the phoneme list of the source text corresponding to said plurality of detected phonemes. This sub-step allows the search for a phoneme sequence in the entire source text, for example the sentences following the example sentence or the sentences preceding the example sentence, not visible in the figures.
• in case of detection of a correspondence of a phoneme or of a plurality of detected phonemes with the list of phonemes of the source text, a sub-step 122 of updating the index with said corresponding phoneme or the last phoneme of the corresponding sequence of phonemes. This final step of the cycle allows the updating of the index according to the correspondence which was established in the preceding sub-steps. The updated index is retrieved during a new execution of the sub-step 112 of receiving the current position of the index.

The method may also include other substeps not described for ensuring the matching of phonemes detected in the audio text and phonemes as recorded by the source text.

The method then comprises a step 130 of receiving at least one piece of data representative of a prosody value of the audio data, called prosody data 10, and/or of receiving at least one piece of data representative of the presence or absence of a movement from a movement recognition device. during the reading of the text and representative of characteristics of a present movement, called movement data 12.

Prosody is representative of the emotion given to the speaker of the source text when reading it. Taking movement into account allows for interactivity when reading the source text.

Prosody and movement are considered independently or in combination in the rest of the process. This choice can be made by a user of the process, for example through an interface allowing him to select whether prosody and/or movement are taken into account in future readings.

• des données sur l'intensité des paroles des données audios,
• des données sur la fréquence et/ou la fréquence fondamentale des paroles des données audios,
• des données sur l'intensité et/ou de fréquence et/ou de fréquence fondamentale des consonnes prononcées dans les données audios,
• des données sur l'intensité et/ou de fréquence et/ou de fréquence fondamentale des voyelles prononcées dans les données audios,
• des données sur la longueur des voyelles et/ou des paroles prononcées dans les données audios,
• des données sur la longueur des consonnes prononcées dans les données audios,
• des données sur le débit de parole des données audios.

Prosody data includes data or a combination of data from different data types in the following list:

• data on the intensity of speech in audio data,
• data on the frequency and/or fundamental frequency of the speech of the audio data,
• data on the intensity and/or frequency and/or fundamental frequency of consonants pronounced in the audio data,
• data on the intensity and/or frequency and/or fundamental frequency of vowels pronounced in the audio data,
• data on the length of vowels and/or spoken words in the audio data,
• data on the length of consonants pronounced in audio data,
• data on the speech rate of audio data.

A combination of this data makes it possible to better characterize the emotion and adapt the triggering of sound effects accordingly.

There figure 3 is an illustrative 300 graph showing an example of prosody versus time as detected from the audio data. For simplicity and for illustrative purposes only, the prosody in graph 300 considers only the intensity and fundamental frequency of the audio data. The value assigned to the prosody data here evolves in two dimensions but in practice the value of the prosody data is characterized by the variation of several data for example gathered in a vector or a matrix.

During a first time interval, called calibration interval 310, the intensity and the fundamental frequency are measured to obtain a value that will be considered as average for the speaker of the audio data. The calibration interval 310 can be defined by a trigger, for example from a first predefined position index, and end from a second predefined position index.

The average prosody value determined during the calibration interval 310 is assigned to a zone called the average zone 312. In the event of a variation in the prosody value over time, this value is likely to be found in other zones, a high zone 314 defined by the values above a high threshold 315 and a low zone 316 defined by the values below a low threshold 317. The variation in the value is for example representative of a percentage variation in the fundamental frequency and a percentage variation in the intensity, these two values being able to be weighted differently to give the total value. More zones can be defined according to the data taken into account in the prosody, for example four zones, five zones, or more zones. The zones thus described are mainly for illustrative purposes and the interpretations of the prosody values can be based on other classification methods, for example decision trees, etc.

The threshold values defining each zone may also depend on calibration data obtained at a calibration step making it possible to determine or estimate, from the audio data, the gender of the speaker and/or the age of the speaker and/or the language spoken by the speaker. These calibration data may also make it possible to determine a machine learning model to be implemented for determining the prosody data, from a group of machine learning models embedded in the system implementing the method, each machine learning model being trained at least in part by data related to the gender of the speaker and/or the age of the speaker and/or the language spoken by the speaker and therefore adapted to the analysis of the prosody. for this gender, age or language.

Each zone then corresponds to an audio track containing the sound effects to be triggered. When a sound effect is to be triggered, for example at times 318a, 318b, 318c, 318d represented by circles, the sound effect of the audio track associated with the zone in which the prosody value is located can be triggered. Here, times 318a, 318c and 318d are linked to the high zone and time 318b is linked to the low zone. The prosody data can also be used in real time to vary the current sound effect, for example by increasing or decreasing its volume, pitch, tempo, echo, etc.

The triggering of the sound effect can of course be subject to other conditions, in particular to the movement data.

The motion data comprises data relating to a movement and/or rotation in space and/or to the speed of movement of the motion recognition device, and/or data relating to the correspondence of a movement or a combination of movements detected by the motion recognition device with a predetermined movement among a list of predetermined movements recorded in a storage module of the processing system. The absence of movement is also motion data which can be interpreted by the method.

Further details on the motion data are described below with reference to the figure 5 illustrating a system for receiving and processing audio data.

Back to the figure 1 , the method then comprises a step 140 of determining the sound effect to be triggered as a function of the position index, and as a function of the prosody data and/or the movement data. The sound effect triggered is thus adapted as a function of the reading context provided by the prosody and/or the movements.

• une sous-étape 142 de récupération, en fonction de l'index de position, du groupe d'effets sonores associé à la séquence temporelle dans laquelle se trouve l'index de position,
• une sous-étape 144 de détermination, en fonction de la donnée de prosodie et/ou de la donnée de mouvement, de l'effet sonore à déclencher parmi les effets sonores du groupe d'effets sonores.

In particular, in one embodiment of the invention, all of the sound effects associated with the source text are distributed in a plurality of time sequences associated with portions of the source text, each time sequence being associated with a sound effect or a group of sound effects comprising several sound effects, and in that when the position index corresponds to a time sequence associated with a group of sound effects, step 140 of determining the sound effect to be triggered comprises:

• a sub-step 142 of recovering, as a function of the position index, the group of sound effects associated with the time sequence in which the position index is located,
• a sub-step 144 of determining, as a function of the prosody data and/or the movement data, the sound effect to be triggered among the sound effects of the group of sound effects.

There figure 4 schematically represents a time graph 400 comprising three audio tracks, a medium track 412, a high track 414 and a low track 416, corresponding respectively to the sound effects to be triggered according to the medium, high and low zones described above. As described above, these tracks are mainly for illustrative purposes and in practice the triggering of sound effects may not be associated with particular tracks.

Audio tracks play from left to right as the position index and playback progress. Audio track sound effects are grouped into groups of one to three sound effects associated with time sequences.

The first time sequence 421 is associated with a group that includes for example a sound effect 421a that is applied regardless of the area reached by the prosody value. The second time sequence 422 is associated with a group that also includes a sound effect 422a, in particular because these first two time sequences correspond to the calibration interval as described above.

The following two sequences 423 and 424 are associated with groups each comprising two sound effects, 423a and 423b on the one hand and 424a and 424b on the other hand. Thus, if the prosody value is in the high zone, the sound effects 423b and 424b will be played while if the prosody value is in the low zone medium or low, sound effects 423a and 424a will be played.

Finally, the following sequences 425 to 429 are associated with groups comprising a sound effect for each zone, referenced respectively 425a, 425b and 425c, 426a, 426b and 426c, 427a, 427b and 427c, 428a, 428b and 428c, 429a, 429b and 429c. Thus, a sound effect is available for the high zone, the middle zone and the low zone reached by the prosody value.

The sound effect triggering step 150 is executed when the position index is at a position equal to or after a predetermined trigger index, or within a predetermined trigger window, and the sound effect may be immediately output as soon as the position index reaches or exceeds the trigger index or reaches the trigger window, or output in a delayed manner after the end of a sound effect currently being output. If the position index reaches, for example, the mark 430, and the prosody value has changed since the start of the time sequence 424, the sound effect played may be changed immediately. Alternatively, if the position index reaches, for example, the mark 431, and the prosody value has changed since the start of the time sequence 426, the sound effect played may be changed only at the start of the time sequence 427.

According to a non-represented embodiment, these audio tracks correspond to a sound background and additional sound effects can be added during playback depending on the prosody data and/or the motion data and/or the position index. Furthermore, several sound effects can be combined, depending on the prosody and/or the motion. For example, for a reading of a text, a first sound effect comprising the sound of a single musical instrument can be played and if the sound intensity detected by determining the prosody increases, one or more other sound effects adding musical instruments can be combined with the first sound effect.

Additionally, the number of soundtracks and sound zones may be greater than represented.

The method finally comprises a step 150 of triggering the effect determined sound. The sound effect is in particular emitted by a sound emission device of the reception and processing system. This sound emission device is for example a loudspeaker or an audio headset.

The sound effect thus increases immersion when reading the source text.

Step 150 of triggering the sound effect comprises a sub-step 152 of verifying the value of a confidence index and the sound effect is triggered only if the confidence index is greater than a predetermined threshold.

The confidence index is managed by a subprocess 160 for managing the confidence index, which comprises a step 162 for increasing the value of the confidence index in the event of detection of a correspondence of a phoneme or of a plurality of phonemes detected with the list of phonemes of the source text and it comprises a step 164 for decreasing the value of the confidence index if no phoneme detected in the audio data corresponds with the following phonemes, or if the detected phoneme is sufficiently close to the expected phoneme to allow the advancement of the position index but does not have an exact correspondence. The confidence index thus makes it possible not to trigger the sound effect in the event of too great an uncertainty on the precision of tracking of the phonemes. The confidence index can also be impacted by other parameters.

Additionally, a confidence index can also be applied independently to the prosody data.

There figure 5 schematically represents a system 500 for receiving and processing audio data according to an embodiment of the invention. The system allows the reception and processing of audio data corresponding to the real-time reading of a source text by at least one speaker, for the triggering of sound effects synchronized with said reading of the text, by implementing the steps of the reception and processing method described above.

The reception system 500 here comprises a portable computing device formed by a smart phone 510 (better known as a smartphone in English) comprising a source text storage module, the audio data reception module, an audio data processing module, a motion recognition device and a prosody variation analysis module, a sound effect determination module and a sound emission device, adapted to implement the associated steps of the audio data reception and processing method. The motion recognition device is for example an accelerometer, a gyrometer, a magnetometer or a combination. The different modules are managed for example by the processor and the storage memory(ies) of the smart phone 510.

The receiving system 500 also comprises a device for attaching the motion recognition device, more particularly in this embodiment a device for attaching the smart phone 510 comprising this motion recognition device, configured to allow the motion recognition device to be attached to a medium on which the source text is printed, such that the motion recognition device is mechanically secured to said medium when reading the medium. The medium on which the source text is printed is here a book whose figure 5 represents a view 590a of the inside of the book and a view 590b of the outside of the book.

In particular, the attachment device here comprises an elastic band 512 passing through the back cover of the book through notches 514 of the attachment device. The smartphone 510 may for example be arranged at the last guard page 592a of the book (also called the third cover, third cover or back cover). Alternatively, not shown, the smartphone 510 may be arranged with the same attachment device on the back cover 592b (also called the fourth cover).

• des éléments élastiques reliés au support sur lequel est imprimé le texte source, configurés pour un maintien en position du dispositif de reconnaissance de mouvement,
• une poche agencée dans la couverture dudit support, configurée pour accueillir le dispositif de reconnaissance de mouvement,
• un compartiment agencé dans la couverture dudit support, configurée pour accueillir le dispositif de reconnaissance de mouvement,
• un aimant permanent agencé sur ledit support et configuré pour une aimantation avec un élément magnétique agencé sur le dispositif de reconnaissance de mouvement, ou un aimant permanent agencé sur le dispositif de reconnaissance de mouvement et configuré pour une aimantation avec un élément magnétique agencé sur ledit support,
• une pochette clipsée sur ledit support, configurée pour accueillir le dispositif de reconnaissance de mouvement
• etc.

Other types of fastening devices can be implemented:

• elastic elements connected to the support on which the source text is printed, configured to hold the motion recognition device in position,
• a pocket arranged in the cover of said support, configured to accommodate the motion recognition device,
• a compartment arranged in the cover of said support, configured to accommodate the motion recognition device,
• a permanent magnet arranged on said support and configured for magnetization with a magnetic element arranged on the motion recognition device, or a permanent magnet arranged on the motion recognition device and configured for magnetization with a magnetic element arranged on said support,
• a pocket clipped onto said support, configured to accommodate the motion recognition device
• etc.

• une rotation du livre à la manière d'un volant de véhicule, et déclencher des effets sonores associés si l'index de position de la lecture est associé à un tel déclenchement,
• un tapotement du livre pour mimer un toquage de porte, ou jouer d'un instrument de percussion,
• secouer le livre pour mimer la secousse de maracas, un mouvement d'un éventail,
• lever le livre,
• tangage gauche/droite pour mimer le bercement d'un animal, d'un bébé, pour le mouvement d'un bâton de pluie,
• roulis avant/arrière pour accélérer/ralentir la musique,
• etc.

The link between the motion recognition device and the support makes it possible to detect a movement of the support, here the book, during reading and to propose a triggering of an associated sound effect. A movement can for example consist of:

• a rotation of the book in the manner of a vehicle steering wheel, and trigger associated sound effects if the reading position index is associated with such triggering,
• tapping the book to mime a door knock, or playing a percussion instrument,
• shake the book to mime the shaking of maracas, a movement of a fan,
• lift the book,
• left/right pitching to mime the rocking of an animal, a baby, for the movement of a rain stick,
• roll forward/backward to speed up/slow down the music,
• etc.

The movement may also need to be performed at a minimum number of occurrences or with a minimum rotation angle to validate the triggering of the sound effect.

• les effets sonores peuvent être prédéterminés ou générés en temps réel, par exemple via une intelligence artificielle générative,
• le dispositif de reconnaissance de mouvement peut ne pas être attaché au support du texte source s'il est en mouvement en même temps que le texte source (s'il est intégré dans une montre intelligente portée par le locuteur par exemple),
• une détection de certains mots en complément des phonèmes peut être mise en oeuvre pour la détection de cas particulier, en particulier pour la détection de mots personnalisés remplaçant ou complétant une portion du texte source,
• plusieurs locuteurs peuvent être détectés pendant la lecture du texte source et leur prosodie respective prise en compte.

The invention is not limited to the embodiments described. In particular:

• sound effects can be predetermined or generated in real time, e.g. example via generative artificial intelligence,
• the motion recognition device may not be attached to the source text carrier if it is moving at the same time as the source text (for example, if it is integrated into a smart watch worn by the speaker),
• a detection of certain words in addition to phonemes can be implemented for the detection of special cases, in particular for the detection of personalized words replacing or completing a portion of the source text,
• multiple speakers can be detected during reading of the source text and their respective prosody taken into account.

Claims (15)

Method for receiving and processing, in a reception and processing system, audio data comprising words corresponding to the real-time reading of a source text by at least one speaker, for triggering sound effects synchronized with said reading of the text, characterized in that it comprises: - a step (110) of determining an index (12) of the speaker's position in the source text, by detecting a correspondence between the audio data received and the source text recorded in a storage module of the reception and processing system, - a step (112) of receiving at least one piece of data representative of a prosody value of the audio data, called prosody data (10), and/or of receiving at least one piece of data (12) representative of the presence or absence of a movement of a movement recognition device during the reading of the text and representative of characteristics of a present movement, called movement data, - a step (140) of determining the sound effect to be triggered as a function of the position index (12), and as a function of the prosody data and/or the movement data, - a step (150) of triggering the determined sound effect. Reception and processing method according to claim 1, characterized in that the source text is recorded in the processing device in the form of a list of phonemes, and the processing device is configured to detect in the audio data the presence of a phoneme corresponding to the source text. Reception and processing method according to claim 2, characterized in that the speaker position index (12) corresponds to a phoneme from the list of phonemes of the source text, and in that the step (110) of determining the speaker position index in the source text includes: - a sub-step (112) of receiving the current position of the index (12) in the list of phonemes of the source text, - a sub-step (114) of comparing at least one phoneme detected in the audio data with at least one phoneme expected among the following phonemes in the list of phonemes of the source text, - if no phoneme detected in the audio data corresponds with the following phonemes, a sub-step (116) of receiving a plurality of phonemes detected in the audio data and a sub-step (118) of searching in at least part of the source text for a sequence of phonemes in the list of phonemes of the source text corresponding to said plurality of detected phonemes, - if the plurality of phonemes detected in the audio data do not correspond to any sequence of phonemes in the list of phonemes of the source text, a sub-step (120) of searching in the entire source text for a sequence of phonemes in the list of phonemes of the source text corresponding to said plurality of detected phonemes, - in case of detection of a correspondence of a phoneme or of a plurality of detected phonemes with the list of phonemes of the source text, a sub-step (122) of updating the index with said corresponding phoneme or the last phoneme of the corresponding sequence of phonemes. Reception and processing method according to one of claims 1 to 3, characterized in that the step of triggering the sound effect comprises a sub-step (152) of verifying the value of a confidence index and in that the sound effect is triggered only if the confidence index is greater than a predetermined threshold. Reception and processing method according to a combination of claims 3 and 4, characterized in that the comparison sub-step (114) of at least one phoneme detected in the audio data with at least one phoneme expected among the following phonemes in the list of phonemes of the source text comprises a determination of an index representative of the resemblance between the phoneme detected in the audio data and the phoneme in the list of phonemes, and in that if the index representative of the resemblance is less than a predetermined threshold, a sub-step (122) of updating the index with said detected phoneme and a sub-step of decreasing the confidence index. Reception and processing method according to a combination of claims 2 and 4, characterized in that it comprises a step (162) of increasing the value of the confidence index in the event of detection of a correspondence of a phoneme or of a plurality of phonemes detected with the list of phonemes of the source text and in that it comprises a step (164) of decreasing the value of the confidence index if no phoneme detected in the audio data corresponds with the following phonemes. Reception and processing method according to one of claims 1 to 6, characterized in that all of the sound effects associated with the source text are distributed in a plurality of time sequences associated with portions of the source text, each time sequence being associated with a sound effect or a group of sound effects comprising several sound effects, and in that when the position index corresponds to a time sequence associated with a group of sound effects, the step of determining the sound effect to be triggered comprises: - a sub-step (142) of recovering, as a function of the position index, the group of sound effects associated with the time sequence in which the position index is located, - a sub-step (144) of determining, as a function of the prosody data and/or the movement data, the sound effect to be triggered among the sound effects of the group of sound effects. Reception and processing method according to one of claims 1 to 7, characterized in that the prosody data comprises data or a combination of data from different types of data in the following list: - data on the intensity of speech in audio data, - data on the frequency and/or fundamental frequency of the speech of the audio data, - data on the intensity and/or frequency and/or fundamental frequency of consonants pronounced in the audio data, - data on the intensity and/or frequency and/or fundamental frequency of the vowels pronounced in the audio data, - data on the length of vowels and/or spoken words in the audio data, - data on the length of consonants pronounced in the audio data, - data on the speech rate of audio data. Reception and processing method according to one of claims 1 to 8, characterized in that the movement data comprises data relating to a movement and/or a rotation in space and/or to the speed of movement of the movement recognition device, and/or data relating to the correspondence of a movement or a combination of movements detected by the movement recognition device with a predetermined movement from a list of predetermined movements recorded in a storage module of the processing system. A method of receiving and processing according to one of claims 1 to 9, characterized in that the sound effect triggering step (150) is executed when the position index (12) is at a position equal to or after a predetermined trigger index, or within a predetermined trigger window, and in that the sound effect can be immediately emitted as soon as that the position index reaches or exceeds the trigger index or reaches the trigger window, or emitted in a delayed manner after the end of a sound effect currently being emitted. Computer program product for receiving and processing audio data corresponding to the real-time reading of a source text by at least one speaker, for triggering sound effects synchronized with said reading of the text, said computer program product comprising program code instructions for executing, when said computer program product is executed on a computing device, the steps of the method according to one of claims 1 to 10. System for receiving and processing audio data corresponding to the real-time reading of a source text by at least one speaker, for triggering sound effects synchronized with said reading of the text, characterized in that it comprises: - a source text storage module, - an audio data reception module, - an audio data processing module, configured to determine a speaker position index in the source text, by detecting a correspondence between the received audio data and the source text recorded in the storage module, - a motion recognition device configured to provide at least one piece of data representative of the presence or absence of a movement of a motion recognition device during the reading of the text and representative of characteristics of a present movement, called motion data, and/or a module for analyzing the variation of the prosody of the audio data configured to provide at least one piece of data representative of a variation of prosody of the audio data - a module for determining the sound effect to be triggered based on the position index, and based on the prosody data and/or the movement data, - a sound emitting device configured to emit the determined sound effect. System according to claim 12, characterized in that it comprises a portable computer device (510), preferably a smartphone, a digital tablet or a smart watch, comprising the source text storage module, the audio data reception module, the audio data processing module, the motion recognition device and/or the prosody variation analysis module, the sound effect determination module and the sound emission device. System according to one of claims 12 or 13, characterized in that it comprises a motion recognition device and a device (512) for attaching the motion recognition device, configured to allow the motion recognition device to be attached to a medium on which the source text is printed, so that the motion recognition device is mechanically secured to said medium when the medium is read. System according to claim 14, characterized in that the attachment device is formed of an element or a combination of elements from the elements in the following list: - elastic elements (512) connected to the support on which the source text is printed, configured to hold the motion recognition device in position, - a pocket arranged in the cover of said support, configured to accommodate the motion recognition device, - a compartment arranged in the cover of said support, configured to accommodate the motion recognition device, - a permanent magnet arranged on said support and configured for magnetization with a magnetic element arranged on the motion recognition device, or a permanent magnet arranged on the motion recognition device and configured for magnetization with a magnetic element arranged on said support, - a pocket clipped onto said support, configured to accommodate the motion recognition device.

Images