RU2759310C1 - Method and system for bionic control of robotic devices - Google Patents

Abstract

FIELD: robotics.

SUBSTANCE: invention relates to medicine, namely to methods of bionic control of robotic devices. The method performs the formation of a mental command, the formation of an electroencephalographic signal analysis channel, in which the formation of an EEG signal corresponding to a mental command is carried out, the registration of an EEG signal, the formation of a training sample of EEG signals, neural network training, neural network analysis of the registered EEG signal, the formation of a control command for a controlled robotic device and the execution of a command by a controlled robotic device. Additionally, a combined EEG analysis is carried out in the electroencephalographic signal analysis channel. Combined EEG analysis reveals EEG signs of a mental command in the frequency-time and amplitude-phase regions. The result of the combined EEG analysis is selected. When choosing the result of a combined EEG analysis, a final decision is made based on particular decisions obtained as a result of EEG analysis by different methods. A voice signal analysis channel is formed, in which a dictionary of voice commands is formed, acoustic vibrations are generated by a human voice apparatus, access to a controlled robotic device is provided using user voice authentication, a word is formed by converting an acoustic vibration into an electrical signal using a microphone, filtering based on the Hilbert-Huang transformation and word identification by comparison with words from the dictionary of voice commands is carried out.

EFFECT: increase in the reliability of recognition of the control command is achieved.

4 cl, 25 dwg

Description

FIELD OF TECHNOLOGY

The invention relates to medical equipment and can be used for remote control of controlled robotic devices in conditions of limited physical capabilities of the operator, and is also intended for the rehabilitation of disabled people who have lost motor activity.

LEVEL OF TECHNOLOGY

There is a known method of exroskeleton control [1], which consists in registering the user's motor intentions, based on measuring the neuromuscular activity signals of the preserved muscle fibers using surface sensors of electromyographic signals fixed at the location of motor points. The bioelectric effect transmitted from the central nervous system to the muscles is reflected by an increase in the amplitude of the electromyographic signal at motor points. The area of the motor point is an individual feature of a person and is characterized by the most excitable muscle area. The control of biopotentials at the locations of motor points allows one to obtain initial signals for controlling the exoskeleton.

The essence of the known methods based on the analysis of electromyographic signals is to study the registered bioelectric potentials arising in human skeletal muscles upon excitation of muscle fibers.

The disadvantage of this method is the low accuracy of recognition of the user's motor intentions due to the difficulty of obtaining an electromyographic signal about the activity of a particular muscle. Therefore, in order to recognize various movements by electromyographic signals, it is necessary to increase the number of electrode systems applied to the muscle, which brings discomfort to the user.

Another disadvantage of the known method is the difficulty of identifying motor points, individual for each person.

The known method of bionic control of a technical device [2], including the formation of a control action by registering the electrical impedance from the antagonist muscles when performing movements.

The disadvantages of this method are the insufficient accuracy of recognition of bioelectric signals due to the presence of cross-talk from neighboring muscle groups, which limits the possibility of operational control of the actuators of robotic mechanisms.

The closest to the proposed invention in terms of its technical essence and the achieved result is a method for classifying electroencephalographic (EEG) signals in the brain-computer interface [3], which forms a mental command, forms an EEG signal analysis channel, in which an EEG signal is formed corresponding to a mental command , registration of an EEG signal, formation of a training sample of EEG signals, training of a neural network, neural network analysis of a recorded EEG signal, formation of a command and execution of a command by a controlled robotic device.

The system that implements the known method for classifying EEG signals in the brain-computer interface contains a serially connected encephalograph, an analog-to-digital converter, a computer processor, an adapter of a controlled robotic device and a controlled robotic device, while the electrode system of the encephalograph is placed on the user's head.

As follows from the description, the well-known method of classifying EEG signals in the brain-computer interface is an implementation of a non-invasive hardware-software complex "brain-computer-interface" (Brain-Computer-Interface (BCI)), which converts the electrical activity of the brain - impulse and total - in a specific action in which the human body does not take any part. By means of a computer, a completely paralyzed person can control a controlled robotic device (for example, a wheelchair [3]), establish contact with others, be actively involved in the surrounding social environment and, possibly, perform certain tasks, engage in useful and interesting work activities.

The disadvantages of BCI in non-invasive EEG recording are:

- low spatial resolution. For a non-invasive electroencephalogram, it is impossible to determine the electrical activity of individual neurons in the brain; the electrical activity of brain regions in which a large number of neurons exhibit synchronous activity is analyzed. For a non-invasive electroencephalogram, this is about 50 thousand synchronously working neurons [4, 5];

- the presence of noise and artifacts in the EEG signal (the parameters of noise and artifact signals exceed the parameters of signals of electrical activity of the brain). Noise is random vibrations of various physical nature, characterized by the complexity of the temporal and spectral structure [6]. An artifact is understood as a process or formation that is not characteristic of the object under study in the norm and usually arises in the course of its research [7].

Figure 1 shows an algorithm for a known method for classifying electroencephalographic signals in the brain-computer interface.

Figure 2 shows a table of coding classes of mental movements of the user in the control commands of the controlled robotic device.

Figure 3 shows a block diagram of a system that implements a known method for classifying electroencephalographic signals in the brain-computer interface.

Figure 4 shows an electroencephalogram with eye movement artifacts [5].

Figure 5 shows possible physiological artifacts when registering an electroencephalogram [4].

Let us consider the known method, the implementation algorithm of which is shown in figure 1. It is known [4] that a meaningful signal can be seen on the surface of the BCI user's head during non-invasive recording of an electroencephalogram. It is this digitized and filtered EEG signal that is subjected to neural network analysis in the known method. In advance, according to the table of coding classes of mental movements in control commands (see figure 2), the trained neural multilayer network (NMS) analyzes the local positive maxima of the amplitude of the EEG signals from all leads. In this case, the formation of the input vector is carried out as follows: if the values of two neighboring positive maxima differ by less than the threshold of psychophysiological perception of a person, then they are considered equal and the second maximum is excluded from the subsequent analysis, simultaneously with the allocation of the first positive _t maximum. Then, from the reference lead, the values of the amplitudes of the EEG signals are recorded for all other leads, as a result, a set of amplitudes is obtained, which is the first input vector.

The procedure for generating the input vectors is repeated for each subsequent positive maximum of the reference lead and for each individual lead, each time taken as a reference lead, until each of the leads fulfills the function of the reference lead, resulting in a multidimensional array of input vectors from a specific user.

The output array of the NMS is an array formed during the training of the NMS of indicators of the classes of mental movements performed by the user.

In the known method according to the algorithm (see figure 1), when identifying the mental movement, an array of input vectors is fed to the NMS to calculate the output vector, which determines the class of the user's mental movement (see figure 2). The block diagram of the system that implements the known method of classifying EEG signals in the brain-computer interface is shown in figure 3.

In the description of the invention, the authors write [3]: “The task solved in the claimed invention is to reduce the identification time of the user's mental command in the BCI system to 2 seconds while increasing the identification accuracy of the user's mental command to 90% due to the use of previously unknown specific information components of the general spatial -temporal pattern and processing according to the classification algorithm of the selected specific fragments of the EEG signal of the NMS adapted for solving this problem.

The specified technical result is achieved by the fact that the method for classifying EEG signals in the brain-computer interface consists in testing the user, separating specific information components from the general spatio-temporal pattern, creating a sample of digitized fragments of the EEG signal from multiple leads for training the classifier, calculating weight coefficients and classifying fragments of the EEG signal to identify the classes of the user's mental command corresponding to the control signals ”.

In this case, "the values of two adjacent positive local maxima of the EEG signal amplitudes differ by at least 5%."

As follows from the above quotation, in the known method, the EEG analysis of evoked action potentials is carried out only in the amplitude-time domain. For completely paralyzed patients, according to the authors of the present invention, this is not enough and it is necessary to use EEG analysis in the time-frequency (study of the EEG spectrum) and phase-time (study of the EEG attractor) areas.

Indeed, the complex signal-interference environment during the implementation of the known method will not allow a completely paralyzed patient to control a controlled robotic device, for example, a wheelchair, as the authors of the known method claim. Since a wheelchair user cannot be made to sit motionless, which is a prerequisite for successful non-invasive EEG recording [4, 5], in a complex signal-interference environment, according to the authors of the present invention, it is difficult to isolate “specific information components from the general space-time patterns "(see Figures 4 and 5), differing" by at least 5% "only in the amplitude-time domain.

Thus, the disadvantage of the known method for classifying EEG signals in the brain - computer interface is the low reliability of recognition of the mental command, as well as the lack of redundancy of information collection channels, unprotected access to use, which does not allow to fully solve the assigned tasks in the operating conditions of a controlled robotic device (disabled person). stroller).

Indeed, as follows from the above description of the use of the known method of obtaining a control command for a controlled robotic device, a number of difficult circumstances arise. The classification of mental commands in the prototype is aimed at determining the control command by analyzing the EEG signal, moreover, only in the amplitude-time domain of the EEG signal presentation. To obtain a non-noisy informative EEG signal, it becomes necessary to use wet surface electrodes or to use an invasive method, recording an EEG signal. All this, while reducing the noise level of the EEG signal, does not eliminate the problem of artifacts and can lead to a deterioration in the user's health, and lead to a number of complications, such as infection. Consideration should also be given to the effect of electrode polarization, high skin impedance, reliability of electrode attachment, and connective tissue overgrowth of invasive electrodes. It should be noted that the known method does not contain a channel for reservation of information collection, which reduces the reliability of receiving a control command. Also, the absence in the known system of means of ensuring safe access to the controlled robotic device (wheelchair) can lead to unauthorized access and, as a result, harm to the user.

DISCLOSURE OF THE INVENTION

The invention is aimed at increasing the reliability of recognition and reliability of functioning when generating a control command for a controlled robotic device by expanding the field of presentation of initial information by analyzing the characteristic features of information parameters of an EEG signal that are different in nature and duplicating the channel for obtaining initial information when generating a control command for a controlled robotic device ...

For this, in the known method, which forms a mental command, the formation of an analysis channel of electroencephalographic signals, in which the formation of an EEG signal corresponding to a mental command, registration of an EEG signal, formation of a training sample of EEG signals, training of a neural network, neural network analysis of a registered EEG signal, formation of a team is carried out and execution of the command by the controlled robotic device, additionally carried out:

- combined analysis of EEG in the channel for analysis of EEG signals, selection of the result of combined analysis of the EEG signal; in the combined analysis of EEG, EEG identification of the signs of a mental command in the time-frequency and amplitude-phase regions is carried out;

- formation of a voice signal analysis channel, in which:

- formation of a dictionary of voice commands;

- generation of acoustic vibrations by the human vocal apparatus;

- providing access to the controlled robotic device using the authentication of the user's voice;

- word formation by:

- converting acoustic vibrations into an electrical signal using a microphone;

- filtering based on the Hilbert-Huang transformation;

- word identification by comparison with words from the voice command dictionary.

In this case, the analysis of the EEG in the time-frequency domain is carried out by determining the time-frequency parameters of the EEG, determining the EEG signs of a mental command in the time-frequency domain, comparing the time-frequency parameters of the EEG with the signs of a mental command in the time-frequency domain, issuing the conclusion “The presence of signs mental command "in case of coincidence when comparing and issuing a conclusion" Signs of a mental command were not found "in case of non-coincidence when comparing.

In this case, the analysis of the EEG in the amplitude-phase region is carried out by determining the amplitude-phase parameters of the EEG, determining the EEG signs of a mental command in the amplitude-phase region, comparing the amplitude-phase parameters of the EEG with the EEG signs of a mental command in the amplitude-phase region, issuing the conclusion “Availability signs of a mental command "in case of a coincidence of the comparison result and the issuance of a conclusion" No signs of a mental command were found "in case of a mismatch of the comparison result.

The system implementing the method contains a serially connected encephalograph, an analog-to-digital converter, a computer processor, an adapter of a controlled robotic device and a controlled robotic device, while the electrode system of the encephalograph is placed on the user's head, additionally contains a serially connected microphone and a computer sound card connected to the second the input of the computer processor.

The essence of the proposed invention is to increase the reliability of the functioning of the method for classifying EEG signals in the brain-computer interface by increasing the reliability of recognition of a mental command "(MC), duplicating the execution of the MC by a voice control system and equipping the bionic control system with means of ensuring safe access to the use of a controlled robotic device (KRU ) such as a wheelchair.

The introduced actions with their connections show new properties that make it possible to increase the reliability of MC recognition and expand the functionality of the method for processing the EEG signal. Indeed, according to the EEG signal, MC recognition is characterized by EEG information parameters (IP), therefore the choice of the EEG analysis method is determined by the analyzed EEG PI. It is impossible to use for the analysis of any EEG PI a method not intended for processing this PI. EEG analysis methods are based on information processing methods and are designed to extract relevant data from the EEG PI. Regarding IP EEG, we can say that they are "encrypted" in the following EEG components: time, frequency, phase. Undoubtedly, methods for analyzing EEG signals should contain algorithms for converting information in these areas of EEG signal presentation.

The approach proposed by the authors is similar to the use of a medical consultation: the final decision is made on the basis of private decisions of independent experts (algorithms). At the same time, a collective solution is formed based on the "integration" of private solutions.

The process of determining the MC can formally be represented in the form of a system of objective functions:

- решение об отсутствии МК,

- решение о наличии МК.where Р (Н ^* | Н) is the conditional probability of the event Н ^* , calculated under the assumption that the event Н has occurred, Н ₀ is the hypothesis of the absence of MC, Н ₁ is the hypothesis of the presence of MC,

- the decision on the absence of MK,

- decision on the availability of MK.

According to the authors, the use of different methods for analyzing IP EEG signals gives a more complete picture of the presence of an MC of a switchgear user, satisfies criterion (1) and increases the probability (P) of correctly detecting the presence of an MC in a switchgear user.

BRIEF DESCRIPTION OF DRAWINGS

Figure 6 shows an algorithm for the proposed method for bionic control of robotic devices.

Figure 7 shows typical wavelet spectrograms of an electroencephalogram of a healthy person.

Figure 8 shows typical attractors of an electroencephalogram of a healthy person.

Figure 9 shows the algorithm for the combined analysis of the EEG signal.

The figure 10 shows the algorithm of the method "Time-frequency analysis of the EEG signal" of the proposed method of bionic control of robotic devices.

Figure 11 shows a diagram of the choice of the result of the combined EEG analysis

Figure 12 shows an image of the human vocal apparatus.

Figure 13 shows a circuit for generating an acoustic signal.

Figure 14 shows a block diagram of a system that implements the proposed method for bionic control of robotic devices.

Figure 15 shows an algorithm for the interaction of hardware and software channel analysis of a voice signal in word recognition.

Figure 16 shows the functionality of modern voice assistants.

Figure 17 shows an algorithm for processing a voice signal of the proposed method for bionic control of robotic devices.

Figure 18 shows an algorithm for processing a voice signal of the proposed method for bionic control of robotic devices.

Figure 19 shows the Hilbert-Huang transform algorithm.

Figure 20 shows the extraction of the envelope of the voice signal.

Figure 21 shows the result of filtering a voice signal using the Hilbert-Huang transform.

Figure 22 shows an algorithm for identifying a voice command.

Figure 23 shows a decomposition representation of a voice signal.

Figure 24 shows a graphical interpretation of the formation of a set of empirical modes.

Figure 25 shows the result of determining the location of the voice command in the voice signal.

The proposed method is based on the combined use of a complex analysis of the EEG signal and the analysis of the voice signal, as shown in figure 6.

According to the authors, the proposed approach to processing EEG IP based on the use of different methods of EEG analysis makes it possible to increase the probability (P) of correct MC recognition in a completely paralyzed switchgear user in comparison with the method of analysis of only one of the EEG IP. In other words, to increase the sensitivity of the EEG analysis by extracting additional information.

It is known [8, 9] that the use of time-frequency analysis (PTA) and amplitude-phase analysis (AFA) of EEG signals allows obtaining a larger amount of reliable information about the biopotential of the brain, since the main stochastic functions demonstrate continuous changes. The generation of brain biopotentials gives a clearly pronounced pattern precisely within the framework of frequency and phase characteristics (attractors) and allows not only recording a person's mental activity, but also diagnosing various diseases (see Figures 7 and 8).

Let's consider the introduced actions.

Combined EEG analysis. This action includes the following analysis methods (see Figure 9):

- time-frequency EEG analysis, which converts discrete reports of the selected space-time pattern into time-frequency EEG parameters and their comparison with template values. In this case, the determination of the time-frequency parameters of the EEG, the determination of the EEG signs of MC in the time-frequency domain, comparison of the frequency-time parameters of the EEG with the EEG signs of MC in the time-frequency domain, issuing the conclusion "Presence of a mental command" in case of coincidence in the comparison and issuing the conclusion "No signs of a mental command were found" in the case of a discrepancy in the comparison;

- amplitude-phase EEG analysis, which converts discrete reports of the selected spatio-temporal pattern into amplitude-phase EEG parameters and their comparison with template values. In this case, the determination of the amplitude-phase parameters of the EEG is carried out, the determination of the EEG signs of MC in the amplitude-phase area, the comparison of the amplitude-phase parameters of the EEG with the EEG signs of the MC in the amplitude-phase area, the issuance of the conclusion "Presence of a mental command" in the case of the coincidence of the comparison result and the issuance the conclusion "No signs of a mental command were found" in the case of a mismatch in the comparison result.

Selection of the result of the combined EEG analysis. As a result of this action, a final decision is made based on particular decisions obtained as a result of EEG analysis by different methods. There are various approaches to the integration of particular solutions, for example, the majority principle (or the principle of voting) [10]. According to the majority principle, the final decision will correspond to the decision made by the majority of experts (algorithms): (F = A # D # C). The decisions of experts (algorithms) are independent and consist of two judgments (see figure 10):

1. Availability of MK;

2. No signs of MC were found.

Since there are only two possible options, and the final decision is selected from four methods of EEG analysis according to the majority principle, the function of choosing the result of the combined EEG analysis is represented as a voting function "3 out of 4" (see figure 11):

where R _} , R ₂ , R ₃ , R ₄ are the results of the work of each of the methods of combined EEG analysis, which take the value "1" if the corresponding method revealed the presence of MC in the switchgear user, and the value "0" if the corresponding method did not reveal user of the switchgear of the presence of MK.

It is assumed that for each method of EEG analysis, the probability of an error-free result or the probability (P) of the correct detection of the presence of MC in a completely paralyzed user of the switchgear is known, that is, the sensitivity of the MC recognition method. Then the result at each element of the conjunction Q _i (i = 1, 2, ..., 4) (see expression 2 and figures 9 and 11) will be error-free if all data supplied to the conjunction element are error-free.

In this case, the probability of issuing an error-free result by the conjunction element will be calculated using the formula for the probability of the simultaneous occurrence of three independent events:

where P (Q) is the probability of issuing an error-free result by a conjunction element, P (R _N1 ) is the probability of issuing an error-free result by one of the EEG analysis methods fed to the first input of a conjunction element, P (R _N2 ) is the probability of issuing an error-free result by one of the analysis methods EEG fed to the second input of the conjunction element, P (R _N3 ) is the probability of an error-free result being produced by one of the EEG analysis methods fed to the third input of the conjunction element.

Signals from all conjunction elements go to the disjunction element (see figure 11). The probability of issuing an error-free result by a disjunction element is determined by the formula for the probability of occurrence of at least one of four independent events:

where P (R) is the probability of issuing an error-free result by the disjunction element (final result), P (Q _i ) is the probability of issuing an error-free result by the i-th element of the conjunction.

Let us show that the probability of an error-free result in a combined EEG analysis is higher than the probability of an error-free result of any, separately taken, method involved in a combined EEG analysis. Using formula (4), we calculate the probability of the system producing an error-free result in complex analysis, when the probability of an error-free result using the amplitude-time analysis (ABA) method is 0.9, and the NDA and AFA is at least 0.75:

P (R) = 1- (1-0.9) (1-0.75) (1-0.75) = 1-0.00625 = 0.994.

The resulting error-free result is greater than any method used alone.

Thus, in the proposed method of EEG analysis for MK recognition, the use of different methods of EEG analysis provides the following conditions:

where max _pre , min _pre are the values of the maximum and minimum of the objective function of the proposed method of EEG processing to detect the presence of MC, max _iz , min _iz are the values of the maximum and minimum of the objective function of the known method of EEG processing to detect the presence of MC.

The technical result [27] from the implementation of the combined method of EEG analysis is to increase the reliability of recognition of a mental command in a completely paralyzed switchgear user by transforming a material object - an EEG signal - and analyzing its features in the time-frequency and amplitude-phase regions.

Analysis of the voice signal. As follows from figure 6, in addition to the implementation of the combined method of EEG analysis in the channel for analyzing electroencephalographic signals, the algorithm of the proposed method for bionic control of robotic devices contains a channel for analyzing the voice signal. The introduction by the authors of the present invention of the channel for the analysis of the voice signal is due to the following considerations: according to the statistics of diseases of the musculoskeletal system [11], mainly people of working age who can lead an active lifestyle suffer. At the same time, the proportion of diseases of the musculoskeletal system is constantly increasing by 30% every 10 years. Among patients with disorders of the musculoskeletal system, approximately 6% of people who have problems with the speech apparatus can be distinguished, and 94% of people who have no disorders of the speech apparatus [11].

These statistics determine the feasibility of using a voice control system (SGU) in switchgear and serve as a justification for expanding the scope, application of the proposed invention for users of switchgear with musculoskeletal disorders and lack of speech impairment. It turns out that equipping the switchgear with SGU increases the possible number of switchgear users by 94/6 = 15.6 times.

At present, the speech recognition segment accounts for 3.5% of the share of biometrics technologies in the world market, and this share has a constant growth trend. The speech technology sector is recognized as one of the fastest growing in the world. According to a report by MarketsandMarkets, the global speech technology market will grow from $ 3.7 billion in 2019 to $ 12 billion by 2022 [12]. All leading car manufacturers equip their premium cars with SSU. SSUs have been developed with less than 4% inaccurate results (for example, voice biometrics measurements incorrectly identify and reject the voice of a person who has access) [13].

In the channel for analyzing the voice signal (see figure 6), the following is additionally carried out:

- generation of acoustic vibrations;

- user authentication;

- providing access;

- word formation.

Similarly to the calculation of the increase in the reliability of MK recognition in the combined analysis method, we will calculate the probability of the recognition reliability of the control command for the switchgear as a result of the joint functioning of the electroencephalographic signal analysis channel and the voice signal analysis channel. From the theory of probability [14] it is known that:

where Р is the probability of identifying the control command, Q is the probability of error in identifying the control command. Thus, the probability of successful identification of the control team will be determined as:

Therefore, the total probability of an error in identifying a control command in the presence of two channels for receiving an informative signal will be:

where Q _EEG is the probability of an error in identifying a control command when processing an EEG signal, Q _Voice is the probability of an error in identifying a control command when processing a voice signal.

The probability of identifying a control command in the channel for analyzing electroencephalographic signals is 99.4% (see expression (4)). The probability of identification of the SSU control team is 96% [13, 15, 16].

According to formulas (6-8), the values of the probabilities are determined during the functioning of the proposed method and system of bionic control of robotic devices:

the probability of an error in identifying the control command is equal to:

Q = (1-0.994) ⋅ (1-0.96) = 0.00024,

and the probability of successful identification of the control command is:

P = 1-0.00024 = 0.99976.

From the above calculations, it follows that the introduction of a voice signal analysis channel to obtain an additional informative signal together with a combined analysis method in the electroencephalographic signal analysis channel makes it possible to increase the accuracy of determining the control command for switchgear by 9.99% and achieve almost 100% of MC recognition.

We will consider the operation of the proposed system of bionic control of robotic devices (SBURU) taking into account the introduced actions. SBURU works as follows: the generation of acoustic oscillations is carried out by the human vocal apparatus (see figure 12A) [17]. The generation of acoustic oscillations is carried out by (see figure 13) mechanical action of the muscles of the chest on the lungs, as a result of which exhalation is carried out, while the exhaled air causes vibration of the vocal cords, as a result of which sound appears. The larynx of a person expands during inhalation (see figure 12B) and exhalation contracts (see figure 12B), where 1 is the epiglottis, 2 are the vocal cords. Sound, encountering obstacles on its way, which create certain positions of the tongue, lips and teeth for it, is converted into an acoustic vibration containing information of a certain meaningful word [17].

The block diagram of the system that implements the proposed method of bionic control of robotic devices is shown in figure 14, and the algorithm for the interaction of the hardware and software of the channel for analyzing the voice signal during word recognition is shown in figure 15. As a rule, the design of the voice control system combines hardware and software ... The hardware part includes a microphone (mandatory in any SSU), it will perceive voice commands and filter noise [18]. The circuitry of SSU is presented in detail in [19]. The main module of the SGU hardware is a computer that receives and processes commands, and then transmits signals to the executive device.

The software part of the SSU includes the operating system and original software that provide the implementation of algorithms for processing voice signals. The most famous operating systems are iOS, Android or Windows. Sometimes DGS manufacturers develop their own operating system based on already known ones, thereby providing more opportunities for performing a new number of functions. Of the original software tools that provide the implementation of algorithms for processing voice signals, the most famous are:

- Alexa from Amazon [20];

- OK Google [21];

- Siri from Apple [22];

- Cortana from Microsoft [23];

- Alice from Yandex [24].

The functionality of modern voice assistants is shown in Table 2 (see figure 16).

Voice control systems rightfully belong to bionic methods of object control. Indeed, the technology of processing speech signals in voice control systems is a set of methods and tools for the implementation of the language model (see figure 17), within which acoustic, linguistic and semantic analyzes are carried out, providing reliable recognition of MC [16]. Analysis of Figure 16 shows that the main elements of the speech signal processing paradigm are the acoustic and language models:

- an acoustic model is a function that takes a small portion of an acoustic signal (frame) as input and outputs the probability distribution of various phonemes on this frame;

- a language model is a probability distribution on a set of vocabulary sequences that reflects both semantics and morphology. This type of language models is called N-gram language models [25].

The speech interface is characterized by the following advantages:

- speech is a natural interface for anyone, even an unprepared person. Speech reduces the psychological distance between humans and computers or robotic machinery;

- speech itself is not mechanically tied to a computer and can be connected to it through communication systems, for example, a telephone. The speech human-machine interface shortens the physical distance between humans and robotic machinery;

- it is possible to handle a robotic mechanism in complete darkness, with closed eyes, in conditions of holding hands with control levers, with tied hands and in other extreme conditions, which increases the efficiency and mobility of communication, freeing hands and unloading the visual channel of perception when receiving information.

The word recognition algorithm of the proposed method for bionic control of robotic devices is shown in figure 18 and implements the following actions: converting acoustic signals into an electrical signal; filtering noise; word identification; recognized word.

The word recognition algorithm works as follows: acoustic vibrations are converted into an electrical signal using a microphone. Then the incoming signal is filtered. The Hilbert-Huang transform [26] is used as a filter. The Hilbert-Huang transformation algorithm is shown in Figure 19.

The work of the Hilbert-Huang transformation algorithm consists in performing the following steps. Determination of the maxima {x _{max, k} } and minima {x _{min, k} } of the signal under study:

- the value of the k-th sample is the maximum if the condition

- the value of the k-th sample is the minimum if the condition

where k are discrete time samples.

Determination of the upper h (t) and lower l (t) signal envelopes using cubic spline interpolation from the found local extrema:

where α _k , β _k , γ _k , δ _k are the coefficients for each value of the kth sample of the upper and lower signal envelopes (can be calculated under the condition that the full interpolation function was continuous up to the second order derivatives).

An example of envelope extraction of a speech command signal is shown in FIG. 20.

Calculation of the average value of the signal envelopes in accordance with the expression:

Checking the condition the average value of the two envelopes should be approximately equal to zero (m (t) = 0). If the condition is not met, then the local remainder is calculated and used as a signal for further screening x (t) = c (t):

where c (t) is the local remainder. If the condition is met, then the local remainder is used as the empirical mode (EM) of the i-th level of the decomposition of the empirical mode function IMF _i (t) = c (t).

Calculation of the remainder of the signal by the formula:

where r (t) is the remainder of the signal.

Calculation of the stop criterion value. As a criterion for stopping the decomposition, the value of the normalized quadratic difference is used, defined as:

where T is the total number of points in the sequence, IMF _i (t) and r (t) are the last EM and the remainder, respectively.

Checking the stop condition. At this stage, the value of the remainder of the signal is compared with the threshold SD≤δ. Experimentally, the threshold for speech signals is set at δ = 0.25. If the condition is not met, then the remainder is used as a signal for a further level of decomposition. If the condition is met, then a set of EMs obtained at all levels of decomposition is output.

As a result of decomposition, a finite number of EM IMF _i (t) and the resulting remainder are extracted from the original signal x (t):

The result of filtering the voice signal using the Hilbert-Huang transform is shown in figure 21.

The speech command identification algorithm (see figure 22) includes:

- adaptive decomposition of a speech command into EM using the method of complementary multiple decomposition into empirical modes (CMDEM);

- formation of a set of informative signals, which are the differences between the original signal of the speech command and non-informative EM. In other words, informative signals consist only of modes containing information about the unique properties of speech.

- determination of the chalk-frequency capsetral coefficients (MFCC) of a set of informative signals, the formation of a template base and the implementation of direct recognition.

In the presented algorithm, input of a speech signal without pauses х (n) (n is a discrete time count 0 <n≤N, N is the number of discrete counts in the signal) is carried out with the following parameters: recording duration - no more than 30 ms, sampling frequency 8000 Hz, quantization capacity 16 bits.

CMDEM settings: the level of the white noise amplitude is 0.1 mV, the number of decomposition cycles is J = 50 (J = 1, 2,… J). Decomposition of a noisy speech signal consists in obtaining averaged values of EM and residual:

An example of a voice decomposition representation is shown in Figure 23.

и остаток r_I(n), где i-номер ЭМ, I - количество ЭМ, устанавливается номер моды фрагмента и дальнейшая работа осуществляется с каждой ЭМ в отдельности. Коэффициенты а, b устанавливаются равными единице и в последующем определяют, какие ЭМ будут использоваться в формировании набора информативных сигналов.After the completion of the decomposition, the noisy speech signal is a set of EM

and the remainder r _I (n), where i is the number of EM, I is the number of EM, the mode number of the fragment is set and further work is carried out with each EM separately. Coefficients a, b are set equal to one and subsequently determine which EM will be used in the formation of a set of informative signals.

To form a set of informative signals, it is necessary to classify the informativity of all EMs extracted from the signal.

Provided that the speech signal has a finite energy, the number of EMs during decomposition is always finite. For a completely arbitrary signal, all EMs can be divided into two categories:

- informative EMs with noise and signal EMs;

- non-informative EMs with trend and compensating EMs. Informative EM in decomposition always reflects the internal structure and characteristics of the signal. These include noise and signal EMs. The appearance of the former in the expansion is explained by the presence of residual noise in the original signal, while the latter are directly related to the useful signal and its components.

Non-informative EMs are slowly changing functions. Among them, there are trend EMs that describe the true dynamics of the mean value of the signal and compensating EMs arising during decomposition. Trending EMs appear, for example, when the sum of a harmonic signal and a polynomial trend is decomposed. Compensating (false) EMs are the result of imperfection of the decomposition algorithm itself (criteria for stopping the screening process, inaccuracies in calculations, rounding errors).

The formation of a set of informative signals consists in subtracting the speech command from the original signal: informative noise and non-informative modes. Informative noise is usually the first two or three EMs, depending on the intensity of the noise present in the signal. The last three or four EMs are uninformative, depending on the total number of modes (the number of EMs is approximately equal to the binary logarithm of the number of samples in the signal). A graphic interpretation of the formation of a set of useful signals is shown in figure 23. The formation of a set of informative signals is carried out according to the formula:

where x _{ab, i} (n) is an informative signal, x (n) is the original signal of the speech command, i is the number of the EM, I is the number of EMs, and, b are the coefficients that determine the participation of EMs in the formation of a set of informative signal signals (see. Figure 24).

The purpose of forming a set of informative signals is the ability to select one signal containing the maximum amount of information about the unique properties of the user's speech. Since each word is invariant with respect to: tonality (frequency change), sound intensity, speed of pronunciation or sound, timbre color, intonation and various defects of the user's speech (accent, stress, mispronunciation, etc.).

At this stage of the proposed method, a user authentication procedure is performed. Based on the pre-recorded user code word, a check is made to obtain access to the use of a controlled robotic device (wheelchair). User authentication provides the function of programming the switchgear "for himself" and allows the subsequent processing of the recorded EEG signal and speech signal. Thus, it is possible to encrypt from prying ears what you turned on or off while controlling the switchgear.

In the subsequent actions of the word recognition algorithm, the most acceptable informative signal will be selected, providing the smallest difference between the speech command entering the system and the template obtained during the training of the voice control subsystem.

Segmentation of EM into fragments is carried out according to the following formulas:

where IMF _i (n) is the EM signal, S is the number of fragments in the EM, L is the number of discrete samples in one fragment.

where IMF _{i, s + 1} (n) is the fragment of the i-th EM, s = (0, 1, 2,… S-1) is the number of the fragment.

MChKK includes two basic concepts: cepstrum and chalk-scale.

Cepstrum is a discrete-cosine transformation of the amplitude spectrum of a signal on a logarithmic scale. The signal cepstrum is determined by the formula:

where DCT is the discrete cosine transform, X is the spectral representation of the signal x (n).

The concept of cepstrum allows you to take advantage of the spectral representation of a signal. When converting a signal from the time domain to the frequency domain, information is compressed, the initial spectral information about the signal becomes more visual, detailed and compact, and is presented even more compactly - in the form of a cepstrum (“spectrum spectrum”).

The chalk scale is a scale for the frequency response of pitch changes. Chalk is a psychophysical unit of pitch. Pitch is mainly related to vibration frequency. For this reason, people perceive small changes in sound much better at low frequencies than at high frequencies, that is, the chalk scale simulates the frequency sensitivity of human hearing.

Conversion from the hertz scale to the chalk scale and vice versa takes place according to the following formulas:

where m is the frequency in mela, ƒ is the frequency in hertz.

The spectrum of EM fragments is obtained according to the formula:

y _{s + 1} (n) -fragment of the EM signal, Y _{s + 1} (k) -spectrum of the fragment of the EM signal, 0≤k <N-1 is the number of complex amplitudes of sinusoidal signals that compose the original signal, l = (1, 2, … L) is the number of the sample of the EM signal fragment, L is the number of samples in the fragment, j is the imaginary unit.

The value of k determines the frequencies of the signal components:

where F _s is the signal sampling rate.

The periodogram of EM fragments is obtained according to the formula

The resulting fragment periodograms contain an excessive amount of information about the frequencies for the recognition task. For this reason, for a more compact presentation of information, periodograms are divided into frequency ranges. A triangular window function is applied to each range - a chalk filter, which allows you to sum up the amount of energy in each frequency range of the periodogram and determine the chalk coefficients.

The formation of a set of chalk filters is carried out according to the following method:

- set the number of chalk filters G and the lower ƒ _l , upper ƒ _h boundaries of the frequency range in which the filtering will be applied;

- in accordance with the formula (24), the range boundaries are converted from hertz to chalk (m _l , m _h );

- on a chalk scale, the segment [m _l , m _h ] is divided into G + 1 disjoint segments of length

- the center frequencies of the subsections are determined by the following formula:

where g = 1, 2,… G is the filter number;

- the center frequencies are converted to hertz ƒc (g) according to the formula (25), they correspond to the center frequencies of triangular chalk filters;

- the center frequencies of the triangular filters are converted from hertz to the numbers of samples of the periodogram P _{s + 1} (k):

- for each chalk filter, the samples of the periodogram P _{s + 1} (k) are multiplied by the corresponding filter:

where k = (1, 2,… K) is the number of samples in one fragment.

After choosing the triangular filters of the fragments, the energy is logarithmized according to the following formula:

The last step is to compute the discrete cosine transform of the logarithm of the energy of the filter bank. Since all of the filter passbands overlap, the energies in the filter bank are fairly correlated with each other, so decorrelation needs to be done using the following formula:

where c = 1, 2, ... C is the number of the MCCC, C is the desired number of coefficients.

Usually, 12-15 MFCC are used for recognition, since the higher the coefficient index, the faster the energy in the set of filters changes.

As a result, the first MCCC mainly carries information about the intensity of speech signals. Since in the voice control subsystems the registration of speech signals can occur at different levels, the information of the first MFCC becomes redundant. In the developed algorithm, the first MCCC is not used in further analysis.

The normalization operation is used to impart equivalence to each MCCC in a fragment. As you know, high frequencies are less susceptible and MFCC at these frequencies are less important than MFCC at low frequencies. MChKK at high frequencies practically does not affect the result. Normalization of the MCCC is the multiplication of each coefficient by a number that increases with the coefficient number. Thus, the first coefficients by level are decreasing, and the last coefficients are increasing by level. For this operation, the formula is used:

where L _ƒ is a value chosen empirically and is equal to 22.

Finding the first and second increments of the MCCC values allows you to obtain some dynamic information about the static coefficients. The vector of coefficients describes a fixed spectral envelope of one fragment, but it is obvious that speech signals also carry information about the dynamics in the form of insignificant changes in the coefficients over time.

The following formulas are used to calculate the MCCC increments:

where MFCC_D _{s + 1} (c), MFCC_DD _{s + 1} (c) are the first and second increments of MFCC, MFCC _{s + 1} (c) are static MFCC, D is a typical value of the increment equal to 2.

The formation of a database - templates and a data set of the MFCC is a combination of all types of informative parameters (MFCC _{s + 1} (c), MFCC_N _{s + 1} (c), MFCC_D _{s + l} (c), MFCC_DD _{s + 1} (c)) into one vector.

After the formation of the first informative signal, subtracting one informative noise and one uninformative mode (i = 0), the condition i = 2 is checked. If the condition is met, the condition a = 1 is checked. If the condition is not met, then the value is set equal to i = i + 1 and a return to the formation of a set of informative signals is carried out by setting the EM number.

After the formation of all informative signals with the values of the coefficients equal to a = 1, b = 1, the conditions a = 1, b = 1 are checked and the new values of the coefficients are set. Setting the values equal to a = 1, b = 0 corresponds to the formation of informative signals by subtracting only noise EMs. Setting the values equal to a = 0, b = 1 corresponds to the formation of informative signals by subtracting only uninformative EM.

Recognition represents the process of comparing the speech command received by the system with a template from the database obtained during the training of the voice control subsystem. One speech command can be pronounced in different ways, since different parts of a word are pronounced at different speeds. Time alignment should be performed to determine the discrepancy between the speech command entering the system and the template, represented as MFCC vectors. For this purpose, the method of dynamic time transformation is used for recognition, which is a technique for elastic comparison of fragments of a speech command and a pattern at regular intervals.

The process of comparing the MCCC vectors by the incoming speech command and with the template begins with calculating the local deviations between the values of the two vectors:

- determination of the correlation coefficient by the formula:

where r (x _{s + 1} , y _{s + 1} ) are the elements of the deviation matrix, x _{s + 1} = MFCC _comp. (c) vector MFCC of the incoming speech fragment ₍ commands, y _{s + 1} = MFCC _mod. (c) - MFCC template fragments, s = (0, 1, 2, ... S-1) - number of the fragment.

- calculation of the absolute deviation (Euclidean distance) by the formula:

where d (x _{s + 1} , y _{s + 1} ) are the elements of the deviation matrix.

The use of two methods of calculating the deviation, to determine the estimate of the discrepancy, will improve the quality of recognition.

The result of the comparison will be the vector for which the minimum discrepancy between the incoming speech command and the pattern was found. Next, the minimum global estimate of the divergence for the route is calculated as the sum of the local distances between the fragments of the speech command and the pattern:

where G (i, j) is a global estimate for the point (i, j), d (i, j) is a local estimate.

Figure 25 shows the result of determining the location of the voice command in the voice signal (highlighted in bold).

After performing the recognition of all informative signal signs, the formation of the command word is carried out by subtracting the EM from each speech command.

Each command word coming from the MCCC normalization unit corresponds to the output vector β (see figure 2), on the basis of which the control command c ⁱ is formed (or determined) for the controlled device.

The above description of the work of the proposed SBURU, in the opinion of the authors of the present invention, showed that the introduction of additional blocks in the channel. analysis of electroencephalographic signals and the analysis channel of voice signals led to the emergence of new properties and increased:

- the reliability of the system when forming the control team;

- independence and continuity of communication between the user and the controlled device (wheelchair);

- safety of access to use by a controlled device (wheelchair);

while retaining the advantages of the known method for classifying electroencephalographic signals in the brain-computer interface.

The technical result [27] of the proposed invention consists in:

- development of new methods and means of bionic control of controlled robotic devices available for mass use in conditions of limited physical capabilities of the operator, as well as for the rehabilitation of disabled people who have lost motor activity;

- processing of electroencephalographic and voice signals, providing increased reliability of recognition of the control command;

- increasing the accessibility of a controlled robotic device (wheelchair) for all groups of paralyzed patients, which corresponds to the conditions for the implementation of the national project "Health" in terms of rehabilitation of disabled people within the framework of the Federal project "Development of a network of national medical research centers and the introduction of innovative medical technologies" [28 ].

Sources of information

1. Pat. 2364385 Russian Federation, IPC A61N 3/00. Wearable locomotion assistive device and control program / Sankai Yoshiyuki, Applicant and Patentee University PF Tsukuba. - No. 2007132169/14; declared November 22, 2005; publ. 20.08.2009 Bul. No. 23. - 60 p.

2. Pat. 2627818 Russian Federation, IPC А61В 5/0488, A61F 2/54. Method of bionic control of technical devices / Shchukin S.I., Kobelev A.V., Sergeev I.K., Naraikin O.S., applicants and patentees Shchukin S.I., Kobelev A.V., Sergeev I.K. , Naraikin O.S. - No. 2016109214; declared 03/15/2016; publ. 11.08.2017 Bul. No. 23. - 12 p.

3. Pat. 2415642 Russian Federation, IPC А61В 5/0476 A61F2 / 72. Method of classification of electroencephalographic signals in the brain-computer interface / Vladimirsky B.M. and others, applicant and patentee of the Ministry of Education and Science of the Russian Federation, Federal State Autonomous Educational Institution of Higher Professional Education "Southern Federal University". - No. 2009133005/14; declared 09/03/2009; publ. 04/10/2011 Bul. No. 10. - 35 p.

4. Zvezdochkina N.V. Study of the electrical activity of the brain / Kazan: Kazan Publishing House, un-t, 2014. - 59 p.

5. Chernysheva E.G., Chernyshev B.V. Practical classes in applied psychophysiology / Moscow: Publishing house of the State University - Higher School of Economics, 2010. - 54 p.

6. GUFO.ME: [Electronic resource]. - Access mode: https://gufo.me/dict/bse/noise, free - (28.07.2020).

7. Explanatory dictionary of translation: [Electronic resource]. - Access mode: https://znachenie-slova.ru/artefact, free - (28.07.2020).

8. Vokhmina Yu.V., Eskov V.V., Gorbunov D.V., Shadrin G.A. Chaotic dynamics of parameters of electroencephalograms // Bulletin of new medical technologies. - 2015. - T. 22 - No. 2 - P.38 - 43.

9. Sushkova, O.S., Korolev, M.S., Gabova, A.V., Illarioshkin, S.N., Karabanov, A.V., Obukhov K.Yu., Obukhov Yu.V. Joint analysis of signals from electroencephalograms, electromyograms and mechanical tremor in early Parkinson's disease // Journal of Radioelectronics. - 2014. - No. 5. - URL: http://jre.cplire.ru/jre/may14/index.html

10. Gellert, Manfred. All about team building. A guide for trainers / per. with him. - Moscow. Vertex. - 2006.

11. Official site of the Ministry of Health of the Russian Federation: [Electronic resource]. - Access mode: https://www.rosminzdrav.ru/ministry/61/22/stranitsa-979/statisticheskie-i-infomatsionnye-materialy/statisticheskie-materialy, free - (28.07.2020).

12. Speech Technology and CRM magazines: [Electronic resource]. - Access mode: https://www.speechtechmag.com/Articles/Editorial/FYI/Speech-Market-to-Reach-$12-Billion-by-2022-112674.aspx, free - (28.07.2020).

13. MDG group. Voice recognition, speech synthesis, recording and analysis, face and voice identification [Electronic resource]. - Access mode: https://www.speechpro.ru/, free - (06.08.2020).

14. Wentzel, E.S., Ovcharov L.A. Probability theory and its engineering applications. - 2nd ed., Erased. - M .: Higher school, 2000 .-- 480 p.

15. Prospects for the development of speech recognition systems (excerpt from the study): [Electronic resource]. - Access mode: https://habr.com/ru/post/232613/, free - (06/08/2020).

16. Lobanov B.M., Tsirulnik L.I. "Computer synthesis and speech cloning". - Minsk: "Belarusian Science". - 2008 .-- 316 s.

17. Kolesnikova, L.L. Anatomy, physiology and biomechanics of the dentition / Edited by L.L. Kolesnikova and others - M .: GEOTAR-Media, 2009 .-- 304 p.

18. IT Consulting: [Electronic resource]. - Access mode: http://www.terralab.ru/299680/?rl=rss&r2=remote, free - (06/08/2020).

19. Simple interesting DIY radio circuits: [Electronic resource]. - Access mode: httys: //radioskot.ru/publ/unch/skhema_dlja_golosovogo_upravlenija/6-1-0-914, free - (06.08.

20. Amazon Echo: [Electronic resource]. - Access mode: https://voiceapp.ru/articles/amazon-echo-review, free - (06/08/2020).

21. Google Assistant: [Electronic resource]. - Access mode: https://voiceapp.ru/articles/google-assistant-download, free - (06.08.2020).

22. Siri voice assistant: [Electronic resource]. - Access mode: https://voiceapp.ru/articles/siri, free - (06/08/2020).

23. Microsoft Cortana: [Electronic resource]. - Access mode: https://ai-golos.ru/microsoft/microsoft-cortana-est-li-polza-v-nyneshnih-realiyah/free - (06.08.

24. Alice - a voice assistant from Yandex: [Electronic resource]. - Access mode: https://voiceapp.ru/articles/alisa, free - (06/08/2020).

25. Volkova I.A. Introduction to Computational Linguistics. Practical aspects of creating linguistic processors. Textbook for students of the Faculty of CMC MSU. - Publishing Department of the Faculty of Computational Mathematics and Cybernetics, Moscow State University M.V. Lomonosov Moscow, 2006 .-- 44 p.

26. The Hilbert-Huang transform an4 its applications / editors, Norden E. Huang, Samuel S.P. Shen. - World Scientific Publishing Co. Pte. Ltd. 5 TohTuck.Link, Singapore 596224.

27. On approval of the Administrative Regulations for the provision by the Federal Service for Intellectual Property of the state service for the state registration of an invention and the issuance of a patent for an invention, its duplicate: Order of the Ministry of Economic Development of Russia dated 05.25.2016 No. 315 (revised on 07.06.2017) (Registered with the Ministry of Justice of Russia on 14.07 .2016 # 42843).

28. National project "Health": [Electronic resource]. - Access mode: https://strategy24.ru/rf/health/projects/natsionalnyy-proekt-zdravookhranenie, free - (06.08.

Claims (4)

1. The method of bionic control of robotic devices, carrying out the formation of a mental command, the formation of a channel for the analysis of electroencephalographic signals, in which the formation of an EEG signal corresponding to a mental command, registration of an EEG signal, formation of a training sample of EEG signals, training of a neural network, neural network analysis of a registered EEG signal is carried out, formation of a control command for a controlled robotic device and execution of a command by a controlled robotic device, characterized in that it is additionally carried out: combined analysis of the EEG in the analysis channel of electroencephalographic signals, and in the combined analysis of the EEG, the EEG signs of a mental command are identified in the time-frequency and amplitude-phase regions ; selection of the result of the combined EEG analysis, and when choosing the result of the combined EEG analysis, a final decision is made based on particular decisions obtained as a result of EEG analysis by different methods; formation of a voice signal analysis channel, in which a vocabulary of voice commands is formed, acoustic oscillations are generated by the human vocal apparatus, access to a controlled robotic device is provided by means of user voice authentication, a word is formed by converting acoustic oscillations into an electrical signal using a microphone, filtering based on transformation Hilbert-Huang and word identification by comparison with words from the voice command dictionary. 2. The method according to claim 1, characterized in that the analysis of the EEG in the time-frequency domain is carried out by determining the time-frequency parameters of the EEG, determining the EEG signs of a mental command in the time-frequency domain, comparing the time-frequency parameters of the EEG with the signs of a mental command in time-frequency domain, issuing the conclusion "The presence of signs of a mental command" in case of coincidence when comparing and issuing the conclusion "No signs of a mental command were found" in case of mismatch during comparison. 3. The method according to claim 1, characterized in that the analysis of the EEG in the amplitude-phase region is carried out by determining the amplitude-phase parameters of the EEG, determining the EEG signs of a mental command in the amplitude-phase region, comparing the amplitude-phase parameters of the EEG with the EEG signs of a mental command the amplitude-phase area, issuing the conclusion "The presence of signs of a mental command" in the case of a coincidence of the comparison result and issuing the conclusion "No signs of a mental command were found" in the case of a mismatch of the comparison result. 4. The method according to claim 1, characterized in that it is implemented by means of a system that contains a serially connected encephalograph, an analog-to-digital converter, a computer processor, an adapter of a controlled robotic device and a controlled robotic device, wherein the electrode system of the encephalograph is placed on the user's head, and the system additionally contains a serially connected microphone and a computer sound card connected to the second input of the computer processor.

Images