[go: up one dir, main page]

WO2012116232A1 - Systèmes et procédés de décodage de signaux neuraux - Google Patents

Systèmes et procédés de décodage de signaux neuraux Download PDF

Info

Publication number
WO2012116232A1
WO2012116232A1 PCT/US2012/026400 US2012026400W WO2012116232A1 WO 2012116232 A1 WO2012116232 A1 WO 2012116232A1 US 2012026400 W US2012026400 W US 2012026400W WO 2012116232 A1 WO2012116232 A1 WO 2012116232A1
Authority
WO
WIPO (PCT)
Prior art keywords
neural
electrodes
neural signal
frequency
brain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2012/026400
Other languages
English (en)
Inventor
Bradley E. GREGER
Paul A. HOUSE
Spencer Kellis
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
University of Utah Research Foundation Inc
Original Assignee
University of Utah Research Foundation Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University of Utah Research Foundation Inc filed Critical University of Utah Research Foundation Inc
Publication of WO2012116232A1 publication Critical patent/WO2012116232A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7264—Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/24—Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00—Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/04—Arrangements of multiple sensors of the same type
    • A61B2562/046—Arrangements of multiple sensors of the same type in a matrix array
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the present disclosure relates to the general fields of bioengineering and computer technology, and more particularly to methods and systems for decoding neural signals, and in particular, discrete and/or continuous decoding of neural signals, e.g., local field potentials, recorded from a cortical surface.
  • Some embodiments of the present technology provide systems and methods for decoding microscale neural signals to extract information on attempted and/or actual movement and/or action of a subject. Implementations of these embodiments can be used to provide discrete and/or continuous decoding of neural signals. Discrete and/or continuous decoding can be used to decode micro-scale neural signals for intent. In some embodiments, the neural signals can be e.g., local field potentials, recorded relative to a given spatial location of the brain.
  • Discrete decoding can comprise, for example, the classification of neural signals into discrete categories.
  • discrete decoding can be used to classify or identify attempted and/or actual words and/or movements from a subject.
  • Continuous decoding can comprise, for example, the decoding of neural signals to provide information on continuously varying attempted and/or actual movements.
  • continuous decoding can comprise proportional decoding of attempted and/or actual movement of a part of a subject body.
  • Such decoding can provide data on position, velocity, and/or acceleration of movement of the part of the subject's body.
  • some embodiments provide methods and systems that can detect signals from portions of the central nervous system and/or peripheral nervous system.
  • Some embodiments can provide a brain-machine .interface that can be used in various applications.
  • a system for decoding neural signals can comprise a receiver configured to receive a neural signal from each of a plurality of electrodes and a processor configured to convert the neural signals and to apply a classifier to the converted signals.
  • the system can be configured to determine the movement the patient is making. In addition or alternatively, the system can be configured to determine the movement the patient is attempting to make. Further, in addition or alternatively, the system can be configured to determine the part of the body that is moving or attempting to be moved.
  • kits for decoding neural signals can comprise a plurality of electrodes, a receiver, and a processor.
  • the plurality of electrodes can be configured to detect neural signals corresponding to different spatial positions on or within the brain of a patient. Each neural signal can be emanated from the brain when the patient moves and/or attempts to move a part of the body.
  • the receiver can be configured to receive a neural signal from each of a plurality of electrodes. Further, each neural signal can correspond to a different spatial position of the brain of a patient. In addition, each neural signal can emanate from the brain when the patient moves and/or attempts to move a part of the body.
  • the processor can be configured convert the neural signals into frequency and time domain information.
  • the processor can also apply a classifier to the frequency and time domain information so as to determine the movement the patient is making and/or is attempting to make.
  • the electrodes contact a surface of the brain.
  • the electrodes can contact a cortical surface of the brain.
  • the electrodes can also be spaced from a surface of the brain.
  • embodiments can provide a desired relative spatial positioning of the electrodes, whether or not one or more of the electrodes is in direct contact with a surface of the brain.
  • the electrodes contact a face motor cortex. Further, the electrodes can contact other cortical areas. For example, electrodes can contact arm and/or hand areas of the primary motor cortex, Wernicke' s area, Broca' s area, the premotor cortex, and other areas of the cortex.
  • each neural signal comprises a local field potential from the cortical surface.
  • the electrodes comprise micro-electrodes.
  • the frequency and time domain information for each neural signal comprises spectral power of the neural signal at different times.
  • the frequency and time domain information for each neural signal also comprises voltage of the neural signal at different times.
  • the spectral power for each neural signal comprises power at different frequency components.
  • the movement of the part of the body comprises movement of an individual finger of a hand.
  • a kit can be provided that is operative to implement embodiments of the methods disclosed herein.
  • the kit can comprise electrodes for implantation in contact with neural tissue.
  • the electrodes can be configured as a grid of micro-electrodes.
  • surgical tools can be provided to implant the electrode grid.
  • the kit can also comprise a data acquisition system capable of recording neural signals from the electrodes.
  • the data acquisition system can record data from a grid of micro-electrodes, as specified in terms of number of electrodes, sampling rate, bandwidth, and other desired parameters.
  • the kit may also comprise a computation unit (such as a computer processor) capable of executing an appropriate algorithm.
  • the computation unit can execute the decoding algorithm in near real-time. For example, the computation unit can execute the algorithm sufficiently quickly so the patient does not notice any lag.
  • an algorithm can comprise enable the computation unit to perform classification or continuous decoding of neural signals.
  • the kit can optionally comprise an effector device.
  • the effector device can be a computer interface, a virtual reality environment, a prosthetic device, a speech synthesizer, or other machine useful for executing the output of the decoding algorithm.
  • a method for decoding neural signals comprises receiving a neural signal from each of a plurality of electrodes, wherein each neural signal corresponds to a different spatial position of the brain of a patient, each neural signal emanated from the brain when the patient moves. A part of the body.
  • the method also comprises converting the neural signals into frequency and time domain information, and applying a classifier to the frequency and time domain information so as to determine the patient's movement of, and/or attempt to move, a part of the body.
  • a non- transitory machine-readable medium is provided.
  • the machine-readable medium comprises instructions stored therein, which when executed by a machine, cause the machine to perform operations for decoding neural signals.
  • the operations comprise receiving a neural signal from each of a plurality of electrodes, wherein each neural signal corresponds to a different spatial position of the brain of a patient, each neural signal emanated from the brain when the patient moves and/or attempts to move a part of the body.
  • the operations can also comprise converting the neural signals into frequency and time domain information, and applying a classifier to the frequency and time domain information.
  • Operation of the computer can also comprise determining information relating to attempted and/or actual movement of a body part of the patient. Operation of the computer can comprise determining the movement the patient is making. In addition or alternatively, operation of the computer can comprise determining the movement the patient is attempting to make. Further, in addition or alternatively, operation of the computer can comprise determining the part of the body that is moving or attempting to be moved.
  • Fig. la shows an example of a 16-channel 4x4 micro-electrode array.
  • Fig. lb shows placement of two micro-electrode arrays over a cortical surface, in which one of the micro-electrode arrays is placed over the face motor cortex and the other micro-electrode array is placed over Wernicke's area.
  • Fig. lc shows an audio waveform (top) of a verbal task and a
  • Fig. Id shows an audio waveform (top) of conversation, verbal task and verbal reward (top) and a corresponding spectrogram (bottom) of neural data recorded from a single channel over Wernicke's area.
  • Fig. 2a shows windows temporally aligned to spoken words that contain a frequency-domain structure in a spectrogram of neural data recorded from a micro-electrode over the face motor cortex.
  • Fig. 2b shows power spectra calculated for multiple trials and multiple electrodes.
  • Fig. 2c shows a two-dimensional matrix of micro-electrode power spectra and trial information.
  • Fig. 2d shows a principal component analysis performed on micro- electrode power spectra and trial information for two words.
  • Fig. 3a shows a distribution of performance results for each unique combination of two- word through ten- word combinations.
  • Fig. 3b shows a topography of channel performance for micro-electrodes resting over the face motor cortex.
  • Fig. 3c shows a topography of channel performance for micro-electrodes resting over Wernicke's area.
  • Figs. 4a-c illustrate plots of the coherence, separation, and frequency of LFPs measured from the cerebral cortex, according to some embodiments.
  • Fig. 5 shows an array of spectrograms for speaking the word “thirsty,” according to some embodiments.
  • Fig. 6 shows an array of spectrograms for speaking the word “thirsty,” according to some embodiments.
  • Fig. 7 shows a distribution of performance results for two through ten classifications.
  • Fig. 8 shows a block diagram of a system for recording and analyzing data from a micro-electrode array according to some embodiments.
  • Figs. 9a and 9b show grids or arrays of electrodes for use in systems or kits, according to some embodiments.
  • Fig. 10a shows an array of spectrograms for movement of the index finger of a hand according to some embodiments.
  • Fig. 10b shows an array of spectrograms for movement of the middle finger of the hand according to some embodiments.
  • Fig. 10c shows an array of spectrograms for movement of the thumb of the hand according to some embodiments.
  • Fig. 11 illustrates a method for decoding neural signals according to some embodiments.
  • Figs. 12a and 12b illustrate maps comparing hand position measurements against decoded local field potentials.
  • Pathological conditions such as amyotrophic lateral sclerosis or damage to the brainstem can leave patients severely paralyzed but fully aware, in a condition known as locked-in syndrome. Communication in this state is laborious, often reduced to selecting individual letters or words by arduous residual movement. More intuitive communication may be possible by directly interfacing with language areas of the cerebral cortex. Many studies of neural interfaces for communication have focused on the challenging problem of reconstructing continuous, dynamic speech.
  • embodiments described herein are tractable approaches of extracting information from microscale signals. Implementations of these embodiments can be used to provide discrete and/or continuous decoding of neural signals. Thus, embodiments can provide classification of a set of attempted and/or actual words and/or kinematics of a patient.
  • a plurality of electrodes can be used to monitor neural signals.
  • the electrodes can be a grid or array of subdural, nonpenetrating, high- impedance micro-electrodes are used to record local field potentials ("LFPs") from the cortical surface.
  • LFPs local field potentials
  • electrodes can be placed over (relative to) the most relevant areas of the brain. Although there is not a comprehensive list of the precise relevant areas of the brain, various areas of the brain are known at this time as providing the desired data for embodiments of the methods and systems disclosed herein. For example, in accordance with some embodiments, the electrodes can be placed over the face motor cortex and/or
  • the electrodes can contact the frontal lobe and/or other cortical areas.
  • electrodes can contact arm and/or hand areas of the primary motor cortex, Wernicke's area, Broca's area, the premotor cortex, and/or other areas of the cortex.
  • the electrodes can comprise a plurality of microwires.
  • the A LFP may be an electric field potential from a group of neurons located near the corresponding electrode. Neural data from many regions of the brain may be used to decode speech; however, data from electrodes over the face motor cortex were found to be the most accurately decodable. Some embodiments can provide a trial-by-trial decoding of spoken words from cortical surface LFPs in the human neocortex, as discussed further below.
  • BCIs brain computer interfaces
  • Penetrating electrodes have been used to perform rapid decoding of continuous motor movements from neuronal activity in the primary motor area of human neocortex; however, because of the risks associated with implantation in language centers, few studies have explored their use in speech BCIs.
  • the neurotrophic electrode is a penetrating electrode designed to mitigate the risks of chronic implantation that has been used to decode the formant frequencies of speech from neuronal activity in the left ventral premotor cortex.
  • LFPs on a cortical surface of the brain can be recorded from one or more micro-electrode arrays.
  • a micro-electrode array may comprise a plurality of nonpenetrating, 40- ⁇ microwires with 1-mm inter-electrode spacing.
  • Such micro-electrode grids or arrays have been shown to support high temporal- and spatial- resolution recordings.
  • some embodiments decode speech by classifying finite sets of words from cortical surface LFPs, thereby reducing the complexity of the problem to determining a limited number of classes.
  • Fig. la shows an example of a single 16-channel 4x4 micro-electrode grid or array that may be used to record LFPs on the cortical surface.
  • the micro- electrode array is shown next to a U.S. quarter-dollar coin for size comparison.
  • Fig. lb shows two 16-channel 4x4 micro-electrode arrays placed beneath the dura closely approximated to the cortical surface over the face motor cortex and Wernicke's area.
  • the wire bundle 112a leads to the array 110a over Wernicke' s area and the wire bundle 112b leads to the array 110b over the face motor cortex.
  • EoG electrocorticographic
  • Fig. lc shows an audio waveform (top) of a verbal task, in which a patient repeated the word "yes.”
  • Fig. lc also shows a corresponding spectrogram (bottom) of neural data recorded from a single channel or micro-electrode over the face motor cortex.
  • Fig. lc includes a normalized power scale indicating the power levels in the spectrogram. As shown in Fig. lc, the spectrogram reveals frequency-domain structure aligned to the individual words during the verbal task.
  • Fig. Id shows an audio waveform (top) of conversation, verbal task and verbal reward and a corresponding spectrogram (bottom) of neural data recorded from a single channel over Wernicke's area.
  • Wernicke's area is predominantly active when the patient converses and receives verbal rewards after completing an experiment, and was less active during the verbal task.
  • PCA principal component analysis
  • a center of mass, or centroid can be calculated as the average of the coordinates of all projected trials belonging to a particular word.
  • trials are projected into the principal component space and classified as specific words by their proximity to a centroid. An example of this is illustrated in Figs. 2a- 2d.
  • Fig. 2a shows an example of spectrograms 210a-210d of neural data for four different electrodes of a micro-electrode array placed over the face motor cortex.
  • a particular word is repeated three times during a verbal task with each repetition of the word corresponding to a trial.
  • the subject may speak the word or attempt to speak the word for the case where the subject is unable to intelligibly vocalize the word.
  • Fig. 2a shows three 500-msec windows 220a-220c where each window is temporally aligned to one instance of the spoken word.
  • Fig. 2a shows an example of spectrograms 210a-210d of neural data for four different electrodes of a micro-electrode array placed over the face motor cortex.
  • a particular word is repeated three times during a verbal task with each repetition of the word corresponding to a trial.
  • the subject may speak the word or attempt to speak the word for the case where the subject is unable to intelligibly vocalize the word.
  • the windows 220a- 220c contain frequency-domain structure in each spectrogram 210a-210d corresponding to the spoken word at the three trials.
  • Fig. 2b shows a power spectra for each electrode 210a- 210d and each trial.
  • Fig. 2c shows a two-dimensional matrix of micro-electrode power spectra and trial information for a word.
  • power spectra information is collected for each of N electrodes of the array and each of M trials.
  • Fig. 2d shows a principal component analysis performed on micro- electrode power spectra and trial information for the words "hungry” and "thirsty.”
  • principal component analysis performed on micro-electrode power spectra and trial information for the word "hungry” generates a cluster 250 in the principal component space, where each point in the cluster 250 represents one trial.
  • principal component analysis performed on micro-electrode power spectra and trial information for the word "thirsty” generates a cluster 255 in the principal component space.
  • three dimensions of the principal component space are shown for ease of illustration, although it is to be understood that the principal component space may comprise any number of dimensions.
  • a center of mass or centroid may be computed for each cluster corresponding to a particular word.
  • the word may be classified by performing principal component analysis on micro-electrode spectra information from the patient to project the spectra information into the principal component space and then determining its nearest centroid. The word that the patient spoke or attempted to speak can be then classified based on the word corresponding to the nearest centroid.
  • classification may also be used to decode a word based on the micro-electrode spectra information. Examples of other types of classification include maximum likelihood, support vector machine and Bayesian classification.
  • Vocal dynamics such as varied pitch or inflection could contribute to lower-than-expected performance in discriminating some word combinations.
  • decoding accuracies that were well above chance and the timing of the increased spectral power suggest that the micro-electrode array over the face motor cortex recorded signals involved in speech production.
  • activity recorded over Wernicke' s area appears to be involved in speech processing but likely represents language at a more abstract level.
  • FIG. 3b,c shows performance results for individual electrodes over the face motor cortex for different words
  • Fig. 3c shows performance results for individual electrodes over Wernicke's area for different words. Examining the mean performance of each word against all other words, it was found that electrode 14 ranged from 51.5% accuracy for the word "cold” to 81.5% accuracy for the word "yes.” The standard deviation of performance across all 16 motor-sensory electrodes was measured as 6.6 + 1.5 percentage points, suggesting that surface LFPs recorded from some electrodes corresponded to aspects of speech production present in some words but not others.
  • micro-electrode that provided the highest accuracy for any single word varied. Selecting the five electrodes of the array with best overall accuracy from the face motor cortex improved classification accuracy to 89.6 + 10.8% of two-word combinations (median 90.0%; Fig. 3a). However, selecting the five highest-performing electrodes over Wernicke's area did not improve performance (73.5 + 16.4% of two-word combinations correctly classified; median 73.3%) when compared with using all 16 electrodes over that region of cortex. Some micro-electrodes over the face motor cortex may not have recorded neural signals useful in decoding the specific set of words presented, indicating a more concrete mapping of the neural signal onto patterns of speech articulation. Conversely, most of the 16 micro-electrodes over Wernicke's area appear to have recorded neural signal related to language processing, supporting a more distributed and abstract encoding of speech.
  • Figs. 4a-c illustrate coherence functions for a micro-ECoG grid placed on the cerebral cortex.
  • Figs. 4a-c illustrate the coherence, separation, and frequency of LFPs measured from the cerebral cortex.
  • Fig. 4a is a mesh showing the coherence plotted against both separation distance and frequency.
  • Fig. 4b shows coherence plotted against frequency, with color representing separation (increasing with coherence).
  • Fig. 4c shows coherence plotted against separation with color representing frequency (increasing with coherence).
  • Figs. 4a-c demonstrate that the coherence of neural signals falls off within the scale of a few millimeters. Thus, this data indicates that neural signals are encoded in a micro-spatial scale and that this is the proper scale at which to space electrodes, such as micro-electrodes.
  • Fig. 5 shows an array of spectrograms for speaking the word "thirsty,” according to some embodiments. These spectrograms illustrate data from 32 electrodes while the patient said the word “Thirsty.” The spectrograms illustrate a complex structure of spectral power in both frequency and time.
  • Fig. 6 also shows an array of spectrograms. However, Fig. 6 illustrates the spectral power when the patient speaks the word "hello,” according to some embodiments. This can be contrasted with the structure observed in Fig. 5 when saying "hello" on the next slide. This observation led to the new decode shown in Fig. 11 and discussed further below.
  • Fig. 7 illustrates a distribution of multi-word performance results for two through ten classifications.
  • the tight inter-electrode spacing and small number of electrodes can produce a limited spatial coverage of the micro-electrode grid or array.
  • An optimized grid design with larger spacing and more electrodes would likely cover a larger number of relevant neural signals and allow better decoding accuracy. Performance could likely be further improved with patient training to stereotype word articulation.
  • the invasiveness of the micro-electrode grids or array could be reduced with epidural placement, as shown for similar recording devices.
  • the electrodes can be configured to communicate wirelessly with the system.
  • a micro-electrode grid or other embodiments of an electrode such components can communicate wirelessly with one or more components of the system.
  • a wireless implementation of the system might be practical given the relatively low bandwidth required to capture cortical surface LFPs.
  • Fig. 8 is block diagram showing an example of a system 450 for recording and processing LFPs from an micro-electrode array 410 that may be used for various embodiments.
  • the system 450 may include a receiver 452, a processor 455, and a memory 460.
  • the receiver 452 may be used to condition the electrical signals from the micro- electrode array 410 for processing by the processor 455.
  • the receiver 452 may include one or more of the following components: amplifiers (e.g., low- noise amplifiers) for amplifying the electrical signals, a filter for isolating electrical signals within a desired frequency bandwidth, and an analog-to-digital converter for digitizing the electrical signals for processing by the processor 455.
  • amplifiers e.g., low- noise amplifiers
  • filter for isolating electrical signals within a desired frequency bandwidth
  • an analog-to-digital converter for digitizing the electrical signals for processing by the processor 455.
  • the processor 455 may comprise a general purpose processor, digital signal processors (DSPs), application specific integrated circuit (ASICs), discrete hardware components, or any combination thereof.
  • DSPs digital signal processors
  • ASICs application specific integrated circuit
  • Methods for decoding speech using neural signals from the array 410 according to various embodiments discussed above may be embodied in software code that is stored in the memory 460 and executed by the processor 455.
  • the memory 460 may comprise any computer-readable media known in the art including volatile memory, nonvolatile memory, a Random Access Memory (RAM), a flash memory, a Read Only Memory (ROM), a removable disk, a CD-ROM, a DVD, any other suitable storage device, or a combination thereof.
  • RAM Random Access Memory
  • ROM Read Only Memory
  • the system 450 can be configured such that the memory 460 may be an on-board component.
  • the system 450 can be configured such that the memory 460 is off -board and system 450 wirelessly communicates with the off -board memory component.
  • the processor 455 may also output raw electrical signals, processed electrical signals, and/or results of analysis to an output device 465, including, but not limited to, a display for viewing by a neurologist, a printer for generating a computer readout, a computer-readable media, and/or to another computer via a computer network connection.
  • the output device 465 may also include an audio output device that outputs the decoded word as an audio output, e.g., a synthetic voice vocalizing the decoded word.
  • the processor 455 may decode a word by receiving neural signals, e.g., local field potentials, from the micro-electrode array 410 when the patient speaks the word or attempts to speak the word.
  • the processor 455 may then convert the neural signals into frequency-domain information, e.g., power spectra, for one or more electrodes of the array.
  • the processor 455 may then classify the frequency-domain information for the one or more electrodes into one of a set of words. For example, the processor 455 may perform principal component analysis on the frequency-domain information to project the frequency-domain information into the principal component space and determine its nearest centroid in the principal component space, as described above.
  • the processor 455 may display the decoded word on a display and/or vocalize the decoded word from an audio output device.
  • the processor 455 may be trained to classify a particular word using the methods described above with reference to Figs. 2a- 2d.
  • Neural data were recorded from 4x4-channel grids of nonpenetrating microwires at 30,000 samples per second (Cerebus, Blackrock Microsystems) while a patient performed speech tasks. During the speech task, the patient repeated a word between 10 and 20 times, with approximately one second separation between trials. Trial start times were noted for later analysis. [0091] Time and Frequency Domain Signal Processing
  • Frequency data were averaged to represent 10-Hz bins, and bins with line noise (60 Hz) or harmonics were deleted.
  • discrete decoding can comprise, for example, the classification of neural signals into discrete categories.
  • discrete decoding can be used to classify or identify attempted and/or actual words and/or movements from a subject.
  • a variety of classification algorithms may be applied to these time and frequency domain features to decode words, kinematics, and/or attempted kinematics.
  • Fisher' s linear discriminant may be used.
  • the classifier can be trained by finding the mean and covariance for each class in the training set. These parameters are used to determine a projection of the multidimensional features onto a linear direction such that the separation between classes can be maximized and the separation within classes can be minimized. If the data are separable, the trials of a given class can cluster along this new direction. This projection can be applied to each trial in the test set, and the trial can be assigned to a class based on its proximity to the clusters identified during training.
  • continuous decoding can comprise, for example, the decoding of neural signals to provide information on continuously varying attempted and/or actual movements.
  • continuous decoding can comprise proportional decoding of attempted and/or actual movement of a part of a subject body.
  • Such decoding can provide data on position, velocity, and/or acceleration of movement of the part of the subject's body.
  • Kalman Filtering may be used.
  • a Kalman Filter has been used to decode hand position from the neural signals recorded on micro-ECoG grids placed over the hand and arm areas of pre-motor cortex in a human patient. If a decoder is trained on other parameters of attempted and/or actual movements, i.e. force, velocity, acceleration, etc., then it can decode these parameters as well as position.
  • Raw neural data were first downsampled from 30,000 samples/sec to 3,000 samples/sec. To mitigate the effects of 60-Hz noise, the trial-averaged spectrum for each nonpenetrating microwire was calculated to determine the width and amplitude of the 60-Hz noise band. Across ail nonpenetrating microwires in P2, for example, noise levels in this band ranged from 5 dB through 20 dB above the normal spectrum, meaning that a single filter might effectively attenuate noise in a few channels but would leave large banding in most of the data. Therefore, a fifth-order Chebyshev Type-II filter was designed to individually mirror the calculated width and height of the 60-Hz noise.
  • Trial components were marked by points where the velocity of the patient's motion crossed a threshold of one standard deviation from rest. This model was used to mark both outward and return movement; additionally, baseline trials were marked at periods of no movement between trials. Only outward movement trials were included in the present analysis. The ti mestamps of these outward movements were used as markers for reading electrode data. For spectrograms, one second of data, with movement aligned at 0.5 seconds, was used.
  • Spectrograms were generated using the Chronux (chronux.org) package with 250 msec windows and 50 msec step size: tapering parameters were set to a time-bandwidth product of five, and nine leading tapers. Spectrum plots of the raw data were characterized by a power law trend whose features dominated frequency analysis. For this reason, the spectrograms were normalized to the trial-averaged spectrum for all trials of like movement.
  • Pairwise cross- correlation analysis between each set of nonpenetrating microwires was performed on neuronal recordings from a period during arm movement to explore the strength of the linear relationship between the signals recorded by nonpenetrating microwires within close proximity. Because the correlation metric indicates the degree to which two sequences are linearly related, this estimation of independence, while not a complete metric, can approximate the possible usefulness of the high spatial-resolution recording afforded by the nonpenetrating microwire devices. In particular, the correlation analysis should help to clarify whether the nonpenetrating microwire array could be replaced by a single large electrode, or if the tighter spacing of the nonpenetrating microwires allows for recording of potentially unique signals.
  • the data were further processed so that the decode could operate on continuously presented frames of data.
  • the spectral data were averaged into frequency bins covering 0-5 Hz, 5-13 Hz, 13-30 Hz, 30-80 Hz, 80-200 Hz, and 200- 500 Hz. Ail frequencies within +5 Hz of 60 Hz or its harmonics were removed due to line noise contamination.
  • the movement data were downsampled to the same sampling rate as the spectrograms (4 samples/sec). An offset of 150 msec was introduced between the movement data and the spectral data to model delay between neural activity and motor output.
  • the final feature vector » consisted of 6 frequency bins per channel and the hand state xk was represented by a 6- dimensional vector comprising x and y position, velocity, and acceleration. These vectors were defined for k - 1, 2, ... M, where M was the number of samples in the data set.
  • the decoding process required learning the parameters A, H, W, and Q from the training data, then predicting the hand kinematics at each time given the prior estimate of the state and new measurements of neural data.
  • Neural data recorded during two different task sessions were used for training and testing.
  • the means of the movement and neural data features were calculated in order to center the data, and the data were orthogonalized using PCA, with tailing principal components contributing less than 1% of the variance discarded.
  • the principal components and means found during training were applied to the testing data to stay as close as possible to the real-time case where such information would be unavailable.
  • K k P- k H T (HP-H T + QY 1 [00122]
  • the output of the Kalman filter was analyzed by calculating the correlation coefficient between the actual trajectory, recorded during the task session, and the predicted trajectory output from the Kalman filter. To understand which frequencies were important to the Kalman filter, frequency features were averaged into 10 Hz frequency bins between 0 and 500 Hz (from all channels). These narrowband features were tested individually with the Kalman filter, and the correlation coefficients for the x- and y-positions were recorded for each run.
  • An algorithm useful for embodiments of the methods and systems disclosed herein may be able to take advantage of the processing of information in neural structures at the micro-scale using features in space, time, and frequency.
  • the algorithm can correlate the dynamics in local field potential voltage and spectral power (frequency range, e.g., about 0 - 1000 Hz or more) across time with attempted and/or actual movements (speaking, finger flexion and extension, and reaching, etc.).
  • the dynamics in the neural signals across the spatial extent of the grid can also be correlated with movement.
  • decoding can consist of detection and classification of specific types of attempted and/or actual movements, e.g., saying a specific word or making a specific grasping movement. Further, in some embodiments, decoding can consist of detection and classification of specific types of continuously varying parameters of attempted and/or actual movement, e.g. position or acceleration of one or more body parts. Principal Components Analysis (PCA) can be applied to provide classification of attempted and/or actual movements. Further, as noted herein, Kalman Filtering can be used to continuously decode hand position.
  • PCA Principal Components Analysis
  • FIG. 10a shows an array of spectrograms 510 recorded on individual electrodes of the 96 channel micro-ECoG grid when the subject moved an index finger.
  • the electrodes can be in contact with a cortical surface of the brain to sense LFPs from the cortical surface.
  • the electrodes can be in contact with the face motor cortex, premotor cortex, Wernike's area, Broca's area, and/or other areas of the brain.
  • the electrodes can also be spaced from a surface of the brain.
  • embodiments can provide a desired relative spatial positioning of the electrodes, whether or not one or more of the electrodes is in direct contact with a surface of the brain.
  • some embodiments provide methods and systems that can detect signals from portions of the central nervous system and/or peripheral nervous system.
  • embodiments of the methods and systems can be applied to grids placed on the cerebral cortex, but embodiments of the methods and systems can also be to neural signals recorded from sub-cortical structures and the peripheral nervous system.
  • the ranges provided for the physical structure of electrodes can equally apply to sub-cortical and peripheral nerve structures.
  • Some embodiments of the systems and kits disclosed herein can comprise one or more electrodes.
  • the electrodes can be formed in a grid or array. Some embodiments can use one or more electrodes having a size or surface area of between about 0.0001 mm 2 to about 1 mm 2 . Further, in some embodiments, the electrode can have a size or surface area of between about 0.0025 mm 2 to about 0.50 mm 2 . Moreover, some embodiments can use one or more electrodes having a diameter of between about 0.01 mm to about 1 mm. Further, in some embodiments, the electrode can have a diameter of between about 0.05 mm to about 0.50 mm.
  • the electrodes can define a desired impedance.
  • the electrodes can have impedances in the range of about 10 kQ To about 1 ⁇ . Further, the electrodes can have impedances within a range of about 100 kQ to about 500 kQ
  • the total number of electrodes in an array or grid can be between about 4 and about 65536.
  • An array can be provided having between about 4 to about 1000 microelectrodes or more. In many implementations, an array can have between about 4 to about 200 microelectrodes or more. For example, an array can have 16, 32, or 64 microelectrodes arranged in a grid. Further, in some embodiments, the grid can be arranged in a square.
  • the inter-electrode spacing can be between about 100 microns and about 10000 microns. In some embodiments, inter- electrode spacing can be between about 0.05 mm to about 5mm. In some embodiments, optimal spacing of the microelectrodes can be between about 2 mm to about 3 mm apart.
  • the sampling rate can be between 0 and about 2500 Hz to provide a desired temporal resolution. In some embodiments, the sampling rate can be between about 10 and about 500 Hz.
  • some embodiments can use one or more electrodes can have a thickness of between about 0.010 mm and about 5 mm. Some embodiments can be configured such that the electrodes define a thickness of between about 0.020 mm and about 3 mm. For example, some embodiments can use electrodes having a thickness of between about 0.050 mm and about 0.500 mm.
  • an electrode grid can be provided.
  • the grid can comprise one or more small protuberances that elevate an active electrode site above a base plane of the grid.
  • Fig. 9a illustrates a grid 500 having a plurality of protuberances 502 having a generally frustro- conical shape, attached to a base plane 504 of the grid 500.
  • Fig. 9b illustrates a grid 600 having a plurality of protuberances 602 having a generally frustro-conical shape, attached to a base plane 604 of the grid 600.
  • protuberances can aid in making electrical contact the neural tissue and minimizing any shunting of electrical signal through bodily fluids, e.g., cerebrospinal fluid.
  • bodily fluids e.g., cerebrospinal fluid.
  • protuberance can be of various geometries, such as semi-spherical, cylindrical, conical, etc.
  • the electrical ground and reference for the electrodes can consist of low-impedance electrodes that are in contact with the fluid space in which the electrodes are implanted, e.g., the sub-dural cerebrospinal fluid.
  • ground and reference can be placed either the epidural space or the surface of the scalp. The electrical ground and reference may be placed together in any of these spaces, or could be placed separately in any combination of these spaces.
  • the electrodes can comprise microelectrodes, microwire arrays and/or other equipment for recording LFPs from the cortical surface of the brain to provide a desired spatial resolution.
  • the electrodes can define one or more unique sizes.
  • the electrodes can be round, square, and/or have serrated edges.
  • the electrodes can comprise microelectrodes that are formed in grids or arrays.
  • the array of microelectrodes can be embedded in a polymer or other material that allows the grid to be flexible and achieve a micro-thickness.
  • a rubbery clear silicone or other such materials can be acceptable for some uses and may be used in some embodiments.
  • other polymers can be acceptable for some uses and may include polyurethane and poly-imide.
  • each spectrogram 510 shows measured power as a function of frequency and time for the corresponding electrode of the micro-ECoG grid, where the vertical axis represents frequency and the horizontal axis represents time.
  • each spectrogram 510 shows measured power over a frequency range of 0 to 500 Hz and a time range of between approximately between one second before and one second after the finger movement.
  • the array of spectrograms 510 show time and frequency domain data at different spatial positions on the brain.
  • the array of spectrograms 510 represent time, frequency and spatial information that can be used to decode attempted and/or actual finger movement.
  • Fig. 10b shows an array of spectrograms 510 recorded on individual electrodes of the 96 channel micro-ECoG grid when the subject moved a middle finger
  • Fig. 10c shows an array of spectrograms 510 recorded on individual electrodes of the 96 channel micro-ECoG grid when the subject moved a thumb, where the index finger, middle finger and thumb were from the same hand.
  • Figs. 10a, 10b and 10c clearly show different spatial and temporal patterns of spectral power correlated with movement of different fingers of the subject's hand.
  • Each spectrogram 510 shows a temporal pattern of spectral power for the corresponding electrode and the array of spectrograms shows a spatial pattern of the spectral power across the electrodes of the micro-ECoG grid.
  • Decoding of neural signals may be performed using principal components analysis (PCA) and Kalman filtering on the micro-ECoG signals.
  • PCA principal components analysis
  • the features for attempted and/or actual finger movement may be refined through PCA into a principal component space and classification may be performed in the principal component space (e.g., nearest- neighbor clustering or other classification algorithm) to classify (decode) the attempted and/or actual finger movement.
  • classification algorithms include k-nearest neighbors, Bayesian classification, Hidden Markov models (HMM) and learning vector quantization (LVQ).
  • HMM Hidden Markov models
  • LVQ learning vector quantization
  • Other algorithms that may be used include linear regression, Fisher's linear discriminants and support vector machines (SVM).
  • Fig. 11 illustrates a method for decoding neural signals according to some embodiments.
  • data is received from M channels, where each channel corresponds to an electrode of an electrode array and each electrode receives neural signals emanating from a different spatial position of the brain.
  • spectral power e.g., power at different frequency components
  • the spectral power over time provides information about the temporal dynamics of the spectral power.
  • time domain data of the corresponding signal may also be obtained, for example, voltage of the signal over time (voltage at different times), as shown in Fig. 11. This time domain data provides additional information that can be used to improve decoding performance.
  • the spectral power e.g., power at different frequency components
  • fi(t) . . . f (t) the time domain data
  • v(t) e.g., voltage over time
  • the features for the different channels correspond to different spatial positions of the brain and therefore provide spatial information (a spatial pattern).
  • feature information may be collected across the channels for each of a plurality of trials, where each trial may correspond to attempted and/or actual movement of a particular finger or an attempt to speak a particular word.
  • Fig. 11 shows a two-dimensional matrix 620 in which each row represents a different trial and each column represents features for a particular channel.
  • Fig. 11 shows an example in which trials for the words "hungry” and "thirsty" are mapped to the principal component space 630.
  • three dimensions of the principal component space are shown for ease of illustration, although it is to be understood that the principal component space may comprise any number of dimensions.
  • a centroid may be calculated for each word or type of attempted and/or actual finger movement to be classified (decoded).
  • the attempted and/or actual word or finger movement may be classified by performing principal component analysis on the corresponding features from the M channels (e.g., fi(t) . . . fN(t) and v(t)) to refine the features into the principal component space 630 and determining its nearest centroid in the principle component space 630.
  • the word that the patient spoke and/or attempted to speak and/or the attempted and/or actual finger movement can be then classified (decoded) based on the attempted and/or actual word and/or finger movement corresponding to the nearest centroid.
  • Other classification algorithms may also be used.
  • the method was described using the example of finger movement decoding, those skilled in the art will appreciate that the method may be used to decode attempted and/or actual movements of other parts of the body (e.g., toe movement, arm movement, etc.). The method may also be used to decode one or more characteristics of attempted and/or actual finger movement such as position of the finger, force exerted by the finger, acceleration of the finger.
  • Figs. 12a and 12b illustrate the correlation of LFPs with kinematics, and in particular, with reaching movements.
  • Figs. 12a and 12b show maps that compare hand position measurements against decoded local field potentials.
  • LFPs can be used to perform proportional decoding and/or classification of attempted and/or actual movement.
  • a Kalman filter was used to map frequency and time domain features onto the x-position, y-position, velocity, and acceleration of the hand.
  • Figs. 12a and 12b show the position of the hand measure with a mouse (shown in blue) and the position decoding from the LFP (shown in red). While the position decoding from the LFP demonstrates a generally larger amplitude in both maps, there is a high degree of correlation between the position of the hand measure with a mouse and the position decoding from the LFP.
  • embodiments of the methods and systems disclosed herein can be used to decode kinematics of the body. Further, some embodiments can be used to decode kinematics of one or more parts of the body individually and/or together with other parts of the body.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Fuzzy Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physiology (AREA)
  • Psychiatry (AREA)
  • Signal Processing (AREA)
  • Measurement And Recording Of Electrical Phenomena And Electrical Characteristics Of The Living Body (AREA)

Abstract

L'invention concerne des procédés et systèmes de décodage de signaux neuraux, par exemple des potentiels de champ locaux enregistrés à partir, par exemple, d'une surface corticale cérébrale. Dans certains modes de réalisation, l'invention concerne un système, un nécessaire et/ou un procédé de décodage de signaux neuraux. Le système ou le nécessaire peut comporter un récepteur configuré pour recevoir un signal neural provenant de chacune d'une pluralité d'électrodes. Chaque signal neural peut correspondre à une position spatiale différente du cerveau d'un patient et chaque signal neural peut émaner du cerveau, lorsque le patient bouge et/ou tente de bouger une partie du corps. Le système ou le nécessaire peut également comporter un processeur configuré pour convertir les signaux neuraux et pour appliquer un classificateur afin de déterminer la partie du corps et/ou le mouvement tenté et/ou réel de la partie du corps.
PCT/US2012/026400 2011-02-23 2012-02-23 Systèmes et procédés de décodage de signaux neuraux Ceased WO2012116232A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201161463797P 2011-02-23 2011-02-23
US61/463,797 2011-02-23
US201161546514P 2011-10-12 2011-10-12
US61/546,514 2011-10-12

Publications (1)

Publication Number Publication Date
WO2012116232A1 true WO2012116232A1 (fr) 2012-08-30

Family

ID=46721237

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2012/026400 Ceased WO2012116232A1 (fr) 2011-02-23 2012-02-23 Systèmes et procédés de décodage de signaux neuraux

Country Status (1)

Country Link
WO (1) WO2012116232A1 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014130571A1 (fr) 2013-02-19 2014-08-28 The Regents Of The University Of California Procédés de décodage de la parole depuis le cerveau et systèmes pour les mettre en pratique
US10653330B2 (en) 2016-08-25 2020-05-19 Paradromics, Inc. System and methods for processing neural signals
WO2022011260A1 (fr) * 2020-07-09 2022-01-13 Thomas Jefferson University Systèmes et procédés de facilitation de fonction motrice
CN116491960A (zh) * 2023-06-28 2023-07-28 南昌大学第一附属医院 脑瞬态监测设备、电子设备及存储介质
US12161443B2 (en) 2016-09-20 2024-12-10 Paradromics, Inc. Systems and methods for detecting corrupt or inaccurate sensory representations

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040249302A1 (en) * 2003-06-09 2004-12-09 Cyberkinetics, Inc. Methods and systems for processing of brain signals
US20070179396A1 (en) * 2005-09-12 2007-08-02 Emotiv Systems Pty Ltd Method and System for Detecting and Classifying Facial Muscle Movements
US7826894B2 (en) * 2004-03-22 2010-11-02 California Institute Of Technology Cognitive control signals for neural prosthetics

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040249302A1 (en) * 2003-06-09 2004-12-09 Cyberkinetics, Inc. Methods and systems for processing of brain signals
US7826894B2 (en) * 2004-03-22 2010-11-02 California Institute Of Technology Cognitive control signals for neural prosthetics
US20070179396A1 (en) * 2005-09-12 2007-08-02 Emotiv Systems Pty Ltd Method and System for Detecting and Classifying Facial Muscle Movements

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2014130571A1 (fr) 2013-02-19 2014-08-28 The Regents Of The University Of California Procédés de décodage de la parole depuis le cerveau et systèmes pour les mettre en pratique
EP2959280A4 (fr) * 2013-02-19 2016-10-19 Univ California Procédés de décodage de la parole depuis le cerveau et systèmes pour les mettre en pratique
US9905239B2 (en) 2013-02-19 2018-02-27 The Regents Of The University Of California Methods of decoding speech from the brain and systems for practicing the same
US10438603B2 (en) 2013-02-19 2019-10-08 The Regents Of The University Of California Methods of decoding speech from the brain and systems for practicing the same
EP4362016A3 (fr) * 2013-02-19 2024-07-31 The Regents of the University of California Procédés de décodage de la parole du cerveau et systèmes pour les mettre en uvre
US10653330B2 (en) 2016-08-25 2020-05-19 Paradromics, Inc. System and methods for processing neural signals
US12161443B2 (en) 2016-09-20 2024-12-10 Paradromics, Inc. Systems and methods for detecting corrupt or inaccurate sensory representations
WO2022011260A1 (fr) * 2020-07-09 2022-01-13 Thomas Jefferson University Systèmes et procédés de facilitation de fonction motrice
CN116491960A (zh) * 2023-06-28 2023-07-28 南昌大学第一附属医院 脑瞬态监测设备、电子设备及存储介质
CN116491960B (zh) * 2023-06-28 2023-09-19 南昌大学第一附属医院 脑瞬态监测设备、电子设备及存储介质

Similar Documents

Publication Publication Date Title
Bundy et al. Decoding three-dimensional reaching movements using electrocorticographic signals in humans
Liao et al. Decoding individual finger movements from one hand using human EEG signals
Blakely et al. Localization and classification of phonemes using high spatial resolution electrocorticography (ECoG) grids
Vallabhaneni et al. Brain—computer interface
Phinyomark et al. Fractal analysis features for weak and single-channel upper-limb EMG signals
Hebb et al. Creating the feedback loop: closed loop neurostimulation
Gage et al. Naive coadaptive cortical control
Velu et al. Single-trial classification of gait and point movement preparation from human EEG
US11647962B2 (en) System and method for classifying and modulating brain behavioral states
Jiang et al. Characterization and decoding the spatial patterns of hand extension/flexion using high-density ECoG
Goen et al. Review of surface electromyogram signals: its analysis and applications
EP3533392B1 (fr) Patch d'électrode, système et procédé de détection d'indicateur de la maladie de parkinson chez une personne
WO2012116232A1 (fr) Systèmes et procédés de décodage de signaux neuraux
WO2011127483A1 (fr) Décodage de mots à l'aide de signaux neuronaux
Krishnan et al. EEG-based brain-machine interface (BMI) for controlling mobile robots: The trend of prior studies
US10478086B2 (en) Measurement of cortical excitability
Guger et al. Electrocorticogram based brain–computer interfaces
Mhapankar et al. A Spectrotemporal EEG Mapping Approach for Decoding Imagined Marathi Language Phonemes
CN117838147B (zh) 一种基于chirp信号的意识状态检测方法及系统
Narayan et al. EEG Signals Classification for Motor Imagery Task Using Different KNN Algorithms
Ravikumar A 3D-printed Fat-I BC-enabled prosthetic arm: Control based on brain neuronal data
Kanas et al. Voice activity detection from electrocorticographic signals
Karakullukcu Perception estimation and torque control for hand prostheses using EEG and EMG signals
Blakely Decoding Coordinated Hand Movement in Human Primary Motor Cortex Using High Resolution Electrocorticography
Lupi The role of low-frequency activity in BCIs motor decoding: A comparison between invasive (ECoG) and non-invasive (EEG) brain recordings during repetitive finger movements

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12749186

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12749186

Country of ref document: EP

Kind code of ref document: A1